Electronic device and camera image processing method

Abstract

An electronic device (300) according to various embodiments of the present document may comprise: a display (830); a plurality of cameras (310) disposed adjacent to each other and having different optical characteristics; a memory (820); and at least one processor (850) operatively connected to the display, the plurality of cameras, and the memory. The memory may store instructions that may be executed by the at least one processor, and when executed, instruct the electronic device to: determine a first zoom magnification related to image capturing; acquire a first image by using a first camera corresponding to the first zoom magnification from among the plurality of cameras; if the first zoom magnification belongs to a digital zoom section of the first camera, generate a cropped image by cropping a part of the first image on the basis of the first zoom magnification; and on the basis of optical characteristics of the first camera and optical characteristics of a second camera, determine a bokeh level to be applied to a background subject included in the cropped image at the first zoom magnification. Various other embodiments are possible.

Description

Methods for processing electronic device and camera images

[0001] This article relates to an electronic device, and for example, to a method for an electronic device including multiple cameras to process images acquired from the cameras.

[0002] Portable electronic devices (hereinafter, "electronic devices"), such as smartphones and tablet PCs, may include cameras for capturing images of the surrounding environment to provide diverse user experiences. For example, the electronic device may include at least one camera positioned on the front and / or rear. The electronic device may capture images using the camera according to a zoom ratio selected by the user and display the images as preview images (or viewfinders) on the display.

[0003] Because electronic devices prioritize portability, the size of their cameras is inevitably limited. Therefore, it may be difficult to incorporate high-power optical zoom lenses with a wide focal length range, as found in general digital cameras (e.g., digital single-lens reflex (DSLR) cameras). Accordingly, electronic devices may incorporate multiple cameras with different optical characteristics, and provide a hybrid zoom function (hereinafter, “hybrid zoom”) that combines the optical zoom magnification of each camera with digital zoom. Here, digital zoom (or digital crop zoom) may involve cropping a portion of an image acquired by the camera and then enlarging it.

[0004] An electronic device providing hybrid zoom can digitally zoom in on an image acquired from one camera for a certain period of time when the zoom ratio changes according to user input, and provide an image acquired from another camera when the zoom ratio reaches a certain level.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0006] Users want to experience the natural, smooth variable focal length and bokeh effect of a standard digital camera with a single analog high-magnification zoom lens, even on electronic devices. However, when a hybrid zoom operation provides images captured by a different camera at a fixed magnification, the bokeh effect applied to each object within the image may vary due to differences in optical characteristics from the previous camera. Consequently, the bokeh effect may change abruptly during the zoom operation, failing to satisfy the user's desire for a natural, smooth bokeh effect.

[0007] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains.

[0008] An electronic device (300) according to the present disclosure (or specification, invention) may include a display (830), a plurality of cameras (310) arranged adjacent to each other and having different optical characteristics, a memory (820), and at least one processor (850) operatively connected to the display, the plurality of cameras, and the memory.

[0009] According to one embodiment, the memory may store instructions that are executable by at least one processor and, when executed, cause the electronic device to determine a first zoom ratio related to image capturing based on a user input and to acquire a first image using a first camera corresponding to the first zoom ratio among the plurality of cameras.

[0010] According to one embodiment, the memory may store instructions that cause the electronic device to generate a cropped image by cropping a portion of the first image based on the first zoom magnification when the first zoom magnification belongs to a digital zoom range of the first camera in which the first zoom magnification is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of the second camera among the plurality of cameras.

[0011] According to one embodiment, the memory may store instructions that cause the electronic device to determine a bokeh level to be applied to a background subject included in the crop image at the first zoom ratio based on optical characteristics of the first camera and optical characteristics of the second camera, correct the crop image by applying a bokeh effect to the background subject based on the determined bokeh level, and display the corrected image on the display.

[0012] A method performed in an electronic device according to various embodiments of the present document may include an operation of determining a first zoom magnification related to image capturing based on a user input, an operation of acquiring a first image using a first camera corresponding to the first zoom magnification among a plurality of cameras of the electronic device, an operation of generating a crop image by cropping a portion of the first image based on the first zoom magnification when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras, an operation of determining a bokeh level to be applied to a background subject included in the crop image at the first zoom magnification based on the optical characteristics of the first camera and the optical characteristics of the second camera, an operation of correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level, and an operation of displaying the corrected image.

[0013] According to various embodiments of the present document, a computer-readable non-transitory recording medium is provided, which performs the following operations: determining a first zoom magnification related to image capturing based on a user input; acquiring a first image using a first camera corresponding to the first zoom magnification among a plurality of cameras of an electronic device; generating a crop image by cropping a portion of the first image based on the first zoom magnification when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras; determining a bokeh level to be applied to a background subject included in the crop image at the first zoom magnification based on the optical characteristics of the first camera and the optical characteristics of the second camera; correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level; and displaying the corrected image. Instructions can be stored.

[0014] According to various embodiments of the present document, in an electronic device providing a hybrid zoom function, an electronic device and an image processing method of the electronic device can be provided, which can obtain a natural zoom transition effect by processing an image so that the angle of view and bokeh effect continuously vary in relation to the zoom ratio.

[0015] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains.

[0016] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0017] FIG. 2 is a block diagram of a camera module according to various embodiments.

[0018] FIG. 3 illustrates cameras arranged on the rear of an electronic device according to one embodiment.

[0019] FIG. 4 is a flowchart of a method for an electronic device to process camera zoom in a hybrid manner according to one embodiment.

[0020] FIG. 5 illustrates an image acquired from a camera of an electronic device and an image provided through a display when a zoom ratio is changed according to one embodiment.

[0021] FIG. 6 illustrates a front bokeh area and a rear bokeh area in which a depth of field and a bokeh effect appear in a captured image according to one embodiment.

[0022] FIG. 7 illustrates a bokeh level applied to a background subject when the zoom ratio is changed according to one embodiment.

[0023] FIG. 8 is a block diagram of an electronic device according to various embodiments.

[0024] FIG. 9 illustrates a method for determining a bokeh level to be applied to a background subject by interpolating a first optical bokeh level of a first camera and a second optical bokeh level of a second camera according to one embodiment.

[0025] FIG. 10 is a flowchart of a method for generating a LUT that defines a virtual bokeh level to be applied to a background subject based on the optical characteristics of a camera according to one embodiment.

[0026] FIG. 11 illustrates a method for determining a virtual bokeh level to be applied to a background subject from a LUT according to one embodiment.

[0027] FIG. 12 illustrates the bokeh levels to be applied to each background subject according to one embodiment.

[0028] FIG. 13 illustrates a bokeh level applied to a background subject when the zoom ratio is changed according to one embodiment.

[0029] FIG. 14 is a diagram for explaining a Gaussian blur filter for applying a bokeh effect according to one embodiment.

[0030] FIGS. 15a, 15b and 15c illustrate an operation of applying a bokeh effect to a background object from a camera image according to one embodiment.

[0031] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0032] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0033] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0034] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0035] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0036] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0037] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0038] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0039] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0040] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0041] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0042] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0043] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0044] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0045] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0046] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0047] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0048] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0049] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0050] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0051] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0052] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0053] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0054] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0055] FIG. 2 is a block diagram (200) illustrating a camera module (180) according to various embodiments.

[0056] Referring to FIG. 2, the camera module (180) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (260). The lens assembly (210) may collect light emitted from a subject that is a target of image capturing. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. A lens assembly (210) may include, for example, a wide-angle lens or a telephoto lens.

[0057] The flash (220) can emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) can include one or more light-emitting diodes (e.g., red-green-blue (RGB) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. The image sensor (230) can acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) can include one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0058] The image stabilizer (240) can move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operating characteristics of the image sensor (230) (e.g., adjusting the read-out timing, etc.) in response to the movement of the camera module (180) or the electronic device (101) including the same. This allows compensating for at least some of the negative effects of the movement on the captured image. In one embodiment, the image stabilizer (240) can detect such movement of the camera module (180) or the electronic device (101) using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module (180). In one embodiment, the image stabilizer (240) can be implemented as, for example, an optical image stabilizer. The memory (250) can temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing task. For example, when image acquisition is delayed due to the shutter, or when multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and a corresponding copy image (e.g., a low-resolution image) can be previewed through the display device (160). Thereafter, when a specified condition is satisfied (e.g., a user input or a system command), at least a portion of the original image stored in the memory (250) can be acquired and processed, for example, by the image signal processor (260). According to one embodiment, the memory (250) can be configured as at least a portion of the memory (130) or as a separate memory that operates independently therefrom.

[0059] The image signal processor (260) can perform one or more image processing operations on an image acquired through an image sensor (230) or an image stored in a memory (250). The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor (260) may perform control (e.g., exposure time control, read-out timing control, etc.) for at least one of the components included in the camera module (180) (e.g., image sensor (230)). The image processed by the image signal processor (260) may be stored back in the memory (250) for further processing or provided to an external component of the camera module (180) (e.g., memory (130), display device (160), electronic device (102), electronic device (104), or server (108)). According to one embodiment, the image signal processor (260) may be configured to perform a control operation of the processor (120). It may be configured as a separate processor that is at least partially composed of, or operates independently of, the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display device (160) as is or after undergoing additional image processing by the processor (120).

[0060] According to one embodiment, the electronic device (101) may include a plurality of camera modules (180), each having different properties or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera, and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera, and at least another may be a rear camera.

[0061] FIG. 3 illustrates cameras arranged on the rear of an electronic device according to one embodiment.

[0062] According to one embodiment, the electronic device (300) may include at least one front camera disposed on the front of a housing in which a display (e.g., display module (160) of FIG. 1) is disposed, and at least one rear camera disposed on the rear of the housing in which a rear cover is disposed.

[0063] Referring to FIG. 3, the electronic device (300) may include three cameras (e.g., a first camera (312), a second camera (314), and a third camera (316)) arranged adjacent to each other on the rear side (e.g., the upper left side of the rear side) of the housing, but the number and / or arrangement of the cameras is not limited thereto. The first camera (312), the second camera (314), and the third camera (316) may include at least some of the configurations and / or functions of the camera module (180) of FIG. 1 and / or the camera module (180) of FIG. 2. According to one embodiment, the first camera (312), the second camera (314), and the third camera (316) may be arranged adjacent to each other to capture images in substantially the same direction (or slightly different directions).

[0064] In one embodiment, the first camera (312), the second camera (314), and the third camera (316) may have different optical characteristics. For example, the first camera (312), the second camera (314), and the third camera (316) may differ in at least some of the lens optical characteristics, such as the angle of view, the focal length, the aperture, the lens material, the structure, the refractive index, or the refractive / diffraction characteristics, and / or may differ in at least some of the sensor characteristics, such as the sensor pitch and the number of pixels.

[0065] In one embodiment, the first camera (312), the second camera (314), and the third camera (316) can capture images with different angles of view. For example, the first camera (312) can be an ultra-wide camera (or UW (ultra-wide) camera) that captures images with a very wide angle of view (e.g., about 120 degrees), the second camera (314) can be a wide-angle camera (or W (wide) camera) that captures images with a wide angle of view (e.g., about 84 degrees), and the third camera (316) can be a telephoto camera (or T (telescope) camera) that captures images with a narrow angle of view (e.g., about 20 degrees).

[0066] According to one embodiment, the first camera (312), the second camera (314), and the third camera (316) may each have fixed optical characteristics. For example, the first camera (312), the second camera (314), and the third camera (316) may each have fixed values ​​of a lens focal length (f), an aperture number (Fno), and a sensor pitch. Since the electronic device (300) emphasizes portability and has size constraints, it may not be equipped with a high-magnification optical zoom lens with a large variable range of focal lengths like a general digital camera (e.g., a DSLR camera), and may include at least one camera with fixed optical characteristics to enable miniaturization.

[0067] According to one embodiment, the electronic device (300) can activate at least one camera to acquire images when a camera application (or an application using camera resources) is running. The electronic device (300) can display images captured by the camera in real time as a preview (or viewfinder) on the display.

[0068] According to one embodiment, the electronic device (300) may provide a user interface (UI) that can set a zoom ratio when a camera application is executed. For example, the UI may be provided with selectable items corresponding to various zoom ratios (e.g., x0.5, x1.0, x5.0), may be provided in the form of a scrollable bar that can select a specific zoom ratio, and / or may be provided so that the zoom ratio can be adjusted according to a user's multi-touch interaction (e.g., pitch and zoom). According to one embodiment, when the camera application is executed, the electronic device (300) may acquire an image with a default zoom ratio (e.g., x1.0) and change the zoom ratio based on a user input through the UI.

[0069] According to one embodiment, the electronic device (300) may provide a hybrid zoom function. In this document, the hybrid zoom function includes analog optical zoom and digital zoom (or digital crop zoom), and may be a method of using analog optical zoom in some zoom magnification ranges and using digital zoom in other zoom magnification ranges. Analog optical zoom may be a method of enlarging or reducing the angle of view using a camera lens and optical elements. According to one embodiment, when the first camera (312), the second camera (314), and the third camera (316) have fixed optical characteristics (e.g., sensor pitch, lens focal length (f), aperture number (Fno)), the analog optical zoom magnifications provided by each of the first camera (312), the second camera (314), and the third camera (316) may be fixed values ​​such as x0.5, x1.0, and x5.0, respectively. In this document, the first camera (312), the second camera (314), and the third camera (316) are explained as examples having fixed analog optical zoom ratios of x0.5, x1.0, and x5.0, respectively, but the present invention is not limited thereto, and the analog optical zoom ratio of each camera may have a different value. Digital zoom may be a method of enlarging or reducing the field of view of an image to be displayed or stored by cropping a portion of the original image through digital processing of the original image acquired from the image sensor without changing the optical elements.

[0070] According to one embodiment, the electronic device (300) can select one of the first camera (312), the second camera (314), or the third camera (316) to capture an image based on the section to which the zoom ratio currently set by the user (or set by default) belongs. For example, the electronic device (300) may acquire an image using the first camera (312) in a first section (e.g., x0.5 to x1.0) in which the current zoom magnification is less than the optical zoom magnification (e.g., x1.0) of the second camera (314), acquire an image using the second camera (314) in a second section (e.g., x1.0 to x5.0) in which the current zoom magnification is greater than or equal to the optical zoom magnification of the second camera (314) and less than the optical zoom magnification (e.g., x5.0) of the third camera (316), and acquire an image using the third camera (316) in a third section (e.g., x5.0 to x30.0) in which the current zoom magnification is greater than or equal to the optical zoom magnification of the third camera (316).

[0071] According to one embodiment, the electronic device (300) may enlarge or reduce an image using digital zoom when the zoom magnification changes according to a user input within each section of the zoom magnification. For example, when the zoom magnification changes from x0.5 to x0.9 in the first section according to a user input, the electronic device (300) may acquire an original image using the first camera (312) corresponding to the first section, and provide a portion of the original image by cropping it to a size corresponding to the changed zoom magnification x0.9. An example of enlargement or reduction of an image according to the hybrid zoom method will be described in more detail with reference to FIG. 5.

[0072] FIG. 4 is a flowchart of a method for an electronic device to process camera zoom according to one embodiment.

[0073] The illustrated method can be performed by an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (300) of FIG. 3). The electronic device can include a plurality of cameras (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3) having different optical characteristics.

[0074] In one embodiment, at step 410, the electronic device may receive user input for selecting a zoom factor. For example, the electronic device may receive user input for selecting a zoom factor through a UI, such as selectable items corresponding to the zoom factor, a scrollable bar for selecting a specific zoom factor, or a user multi-touch interaction.

[0075] According to one embodiment, in operation 420, the electronic device may select a camera corresponding to a zoom ratio and obtain an image of the selected camera. For example, if the zoom ratio selected by the user falls within a first range (e.g., x0.5 to x1.0), the electronic device may activate the first camera to obtain an image, if the zoom ratio falls within a second range (e.g., x1.0 to x5.0), the electronic device may activate the second camera to obtain an image, and if the zoom ratio falls within a third range (e.g., x5.0 to x30.0), the electronic device may activate the third camera to obtain an image.

[0076] According to one embodiment, in operation 430, the electronic device can determine whether the zoom factor selected by the user belongs to a digital zoom range. According to one embodiment, the electronic device can provide a hybrid zoom function including analog optical zoom and digital zoom, and if the zoom factor selected by the user corresponds to an analog optical zoom factor, the analog optical zoom can be used, and if it belongs to another digital zoom range, the digital zoom can be used to process the zoom of the image. For example, each camera of the electronic device (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3) can have fixed optical characteristics (e.g., lens focal length (f), aperture number (Fno)), and accordingly, the analog optical zoom factor provided by each camera can be fixed to one value (e.g., x0.5 for the first camera, x1.0 for the second camera, and x5.0 for the third camera). If the selected zoom factor is greater than the analog optical zoom factor of the currently activated camera, the electronic device can determine to process it as a digital zoom.

[0077] According to one embodiment, if the selected zoom ratio does not fall within the digital zoom range, in operation 440, the electronic device may display an analog optical zoom image. For example, if the zoom ratio selected by the user is equal to the optical zoom ratio of a specific camera (e.g., the zoom ratio of the first camera x0.5), the image acquired from the camera may be displayed on the display without performing a separate cropping process.

[0078] According to one embodiment, if the selected zoom factor falls within the digital zoom range, the electronic device may perform operations 450, 460, and 470.

[0079] According to one embodiment, at operation 450, the electronic device may determine the angle of view or focal length based on the selected zoom factor. As the zoom factor increases, the angle of view may narrow, and conversely, as the zoom factor decreases, the angle of view may widen. Additionally, as the zoom factor increases, the focal length of the camera may increase, and as the zoom factor decreases, the focal length of the camera may decrease. By determining the angle of view or focal length based on the zoom factor selected by the user, the electronic device may determine an area to be cropped from the original image corresponding to the zoom factor.

[0080] In one embodiment, at operation 460, the electronic device may crop an area of ​​the image corresponding to the determined angle of view (or focal length) and upscale the cropped area. For example, the electronic device may upscale the cropped area to the size of the original image based on an angle of view corresponding to a zoom factor in the original image.

[0081] According to one embodiment, at operation 470, the electronic device can display the digitally zoomed image generated at operation 460 through a display.

[0082] According to one embodiment, instructions for performing each operation constituting the method (400) may be stored in a tangible, non-transitory computer readable recording medium.

[0083] FIG. 5 illustrates an image acquired from a camera of an electronic device and an image provided through a display when a zoom ratio is changed according to one embodiment.

[0084] According to one embodiment, an electronic device (e.g., the electronic device (300) of FIG. 3) may provide a hybrid zoom function including analog optical zoom and digital zoom. According to one embodiment, the electronic device may acquire an image using a camera corresponding to a zoom ratio selected by a user (or a zoom ratio set as a default) among a plurality of cameras (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3), and display the image through a display by cropping and up-scaling at least a portion of the original image during the digital zoom section.

[0085] FIG. 5 illustrates examples of original images acquired through a camera and images provided through a display when a user increases the zoom magnification from the initial x0.5 to x0.9 and x1.0. According to one embodiment, the electronic device may include a first camera (e.g., the first camera (312) of FIG. 3), a second camera (e.g., the second camera (314) of FIG. 3), and a third camera (e.g., the third camera (316) of FIG. 3), and, depending on the optical characteristics of each camera, the first camera may capture images corresponding to a zoom magnification of x0.5, the second camera may capture images corresponding to a zoom magnification of x1.0, and the third camera may capture images corresponding to a zoom magnification of x5.0. In other words, the first optical zoom magnification of the first camera may be x0.5, the second optical zoom magnification of the second camera may be x1.0, and the third optical zoom magnification of the third camera may be x5.0.

[0086] The electronic device can capture images and process digital zoom using a first camera in a first range of zoom magnification (e.g., x0.5 to x1.0), a second camera in a second range (e.g., x1.0 to x5.0), and a third camera in a third range (e.g., x5.0 to x30.0). The number of cameras included in the electronic device, the optical zoom magnification of each camera, and / or the range of zoom magnifications are not limited to the examples described above.

[0087] Referring to (a) of FIG. 5, when a user sets the zoom ratio to x0.5, the electronic device can acquire a first original image (510) using the first camera corresponding to the set zoom ratio. Since the zoom ratio x0.5 corresponds to the analog optical zoom ratio of the first camera, the electronic device can display an image (530) of the entire area of ​​the first original image (510) through the display without cropping the first original image (510).

[0088] Referring to (b) of FIG. 5, when the user changes the zoom ratio to x0.9, the electronic device can determine that x0.9 belongs to the first section of the digital zoom section. Here, the first section may be equal to or greater than the optical zoom ratio of the first camera (e.g., x0.5) and less than the optical zoom ratio of the second camera (e.g., x1.0). For zoom-in processing, the electronic device can determine the angle of view (or focal length) corresponding to the changed zoom ratio x0.9 in the first original image (510), and determine an area (515) to be cropped in the first original image (510) according to the determined angle of view. The electronic device can crop a portion of the first original image (510), upscale the cropped area (515) to the size of the original image, and display the upscaled image (535) through the display.

[0089] Referring to (c) of FIG. 5, when the user changes the zoom magnification to x1.0, the electronic device can confirm that x1.0 corresponds to the analog optical zoom magnification of the second camera. In this case, the electronic device can activate the second camera to acquire the second original image (520) and deactivate the first camera. Since the zoom magnification x1.0 corresponds to the analog optical zoom magnification of the second camera, the electronic device can display an image (540) of the entire area of ​​the second original image (520) through the display without cropping the second original image (520). According to one embodiment, since the first camera and the second camera are arranged adjacent to each other, even if the camera that captures the image is changed from the first camera to the second camera, the capturing area can be substantially the same. That is, the image obtained by digitally zooming the first original image (510) corresponding to x1.0 and the second original image (520) can be images having substantially the same angle of view.

[0090] FIG. 6 illustrates a front bokeh area and a rear bokeh area in which a depth of field and a bokeh effect appear in a captured image according to one embodiment.

[0091] According to one embodiment, the electronic device (300) may include a plurality of cameras (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3) having different optical characteristics. In this case, each camera of the electronic device (300) may have a different depth of field (DoF) (610) in the captured image depending on the different optical characteristics.

[0092] Depth of field (610) refers to the distance between the front and back of the main subject being photographed (e.g., a book (592) in the image of FIG. 5) that is in focus, and may refer to the depth between the front depth of field (Dn) and the back depth of field (Df). When photographing a surrounding environment with a specific camera, subjects within the depth of field (610) may be in focus and captured clearly, while objects outside the depth of field (610) may be captured blurry because they are out of focus.

[0093] Depth of field (610) may be determined based on the optical characteristics of the camera and the shooting environment. More specifically, depth of field (610) may be determined based on the optical characteristics of the camera, such as the sensor pitch, lens focal length, and lens aperture, and the shooting environment, such as the distance to the subject.

[0094] Sensor pitch may refer to the spacing between pixels constituting the camera's image sensor. As the sensor pitch decreases, the amount of light incident through the camera lens reaches each pixel increases, which may result in a shallower depth of field (610). In other words, as the sensor pitch increases, the forward depth of field decreases toward the camera, and the rearward depth of field decreases in the opposite direction of the camera, thereby increasing the depth of field (610).

[0095] The aperture is a configuration for controlling the amount of light entering the image sensor, and the aperture number (or f-number) can correspond to the inverse of the aperture size. That is, as the aperture size becomes smaller, the aperture number can be assigned a larger value. For example, the higher the magnification of the camera, the smaller the aperture diameter (or the larger the aperture number), and accordingly, the depth of field of the high-magnification camera can be deeper than that of the low-magnification camera. The longer the focal length of the lens, the shallower the depth of field (610), and conversely, the shorter the focal length, the deeper the depth of field (610). That is, a telephoto camera with a long focal length (e.g., the third camera (316)) has a shallow depth of field (610), and a wide-angle camera (e.g., the second camera (314)) and an ultra-wide-angle camera (e.g., the first camera (312)) can have a relatively deeper depth of field (610).

[0096] Depth of field (610) may vary depending on the distance from the main subject (e.g., the book (592) in the shooting environment of FIG. 6) in the shooting environment. For example, if the distance between the camera and the main subject is far, the depth of field (610) may be deep, and if the distance between the camera and the subject is close, the depth of field (610) may be shallow.

[0097] According to the optical principles described above, the depth of field (DoF), forward depth (Dn), and back depth (Df) of a specific camera can be calculated as in mathematical expression 1.

[0098]

[0099] ,

[0100] In the above mathematical expression 1, F is the aperture number, f is the lens focal length, c is the size of the circle of confusion, and s is the distance to the main subject.

[0101] According to one embodiment, each camera included in the electronic device (300) (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3) may have fixed optical characteristics. That is, the sensor pitch, the lens focal length, and the aperture number of each camera may be fixed. Accordingly, the depth of field of each camera may depend only on the distance from the main subject, and when the distance from the main subject is the same, the depth of field (610) of each camera may have a fixed value. For example, the first camera (or UW camera), the second camera (or W camera), and the third camera (or T camera) in that order of wide angle of view may have a wide depth of field (610). When the shooting direction is fixed and the shooting camera is changed according to the adjustment of the zoom ratio, the optical depth of field (610) changes, so some subjects that were located within the depth of field (610) of the camera before the change may be located outside the depth of field (610) according to the change of the shooting camera.

[0102] Fig. 6 shows the depth of field (610) when capturing an image with the first camera of the electronic device (300) in a shooting environment similar to Fig. 5. In the case of the rear bokeh area (630) that is further than the rear depth of field (Df) and the front bokeh area (620) that is closer than the front depth of field (Dn), since they correspond to areas outside the depth of field (610), background subjects located in those areas may be captured blurry.

[0103] In the shooting environment of Fig. 5, among the subjects in front, a book (592), a flower pot (594), and a chair (596) can be placed in that order close to the electronic device (300), and the electronic device (300) can capture an image based on the book (592), which is the main subject, using each camera.

[0104] Referring to FIG. 6, depending on the optical characteristics of the first camera (e.g., sensor pitch, lens focal length, aperture number) and the distance from the main subject, the book (592), a front depth of field may be formed at a distance closer than the book (592), and a rear depth of field may be formed between the flowerpot (594) and the chair (596). Accordingly, when capturing an image with the first camera, the flowerpot (594) among the background subjects may be captured clearly because it falls within the depth of field (610), and the chair (596) may be captured blurrily because it falls within the rear bokeh area (630) outside the depth of field (610).

[0105] Bokeh can refer to a phenomenon in which subjects outside the depth of field (610) that are out of focus in a captured image appear blurred. The bokeh level can correspond to the degree to which a specific subject is blurred within the captured image. Referring to FIG. 6 , among the background subjects, a chair (596) may be captured blurry because it falls within the rear bokeh area (630) that is further than the rear depth of field.

[0106] In one embodiment, a second camera (or W camera) having different optical characteristics from the first camera (or UW camera) may have an optical depth of field (610) different from that of the first camera. For example, since the second camera has a longer focal length than the first camera, the optical depth of field (610) of the second camera may be shallower than that of the first camera. In this case, some background objects (e.g., flower pots (594)) that fall within the depth of field (610) when photographed by the first camera may fall outside the depth of field of the second camera when photographed by the second camera.

[0107] FIG. 7 illustrates a bokeh level applied to a background object when the zoom ratio is changed according to one embodiment.

[0108] According to one embodiment, an electronic device (e.g., the electronic device (300) of FIG. 3) includes a plurality of cameras (e.g., the first camera (312), the second camera (314), and the third camera (316) of FIG. 3) and may provide a hybrid zoom function including analog optical zoom and digital zoom. In this case, when the analog optical zoom magnification of another camera is changed in the digital zoom section, a phenomenon may occur in which the bokeh level (or degree of blur) of some background subjects (e.g., the flowerpot (594) of FIG. 6) changes discontinuously.

[0109] Fig. 7 shows a flower pot (594), which is a background subject, among the captured images in the same shooting environment as Figs. 5 and 6. In the shooting environment of Figs. 5 and 6, among the front subjects, a book (592), a flower pot (594), and a chair (596) may be arranged in that order to be closer to the electronic device, and in the case of the flower pot (594), which is a background subject, when shooting with the first camera, it may be shot with a bokeh level of 0 according to the optical depth of field according to the optical characteristics of the first camera, when shooting with the second camera, it may be shot with a bokeh level of 5, which is blurrier than when shooting with the first camera, and when shooting with the third camera, it may be shot with a bokeh level of 9, which is blurrier.

[0110] According to one embodiment, when the zoom magnification is set to x0.5 according to the user's selection, the electronic device can acquire the original image using the first camera. Since the zoom magnification x0.5 corresponds to the first optical zoom magnification (710), which is the analog optical zoom magnification of the first camera, the electronic device can display an image of the entire area of ​​the first original image through the display without cropping the first original image. In this case, as described in FIG. 6, the background subject flowerpot (792) can be captured clearly because it falls within the optical depth of field of the first camera (e.g., the depth of field (610) of FIG. 6), and the bokeh level can be represented as 0.

[0111] According to one embodiment, when the user changes the zoom ratio to x0.9, the electronic device can determine that the changed zoom ratio belongs to the digital zoom range (715) of the first camera. Here, the digital zoom range (715) of the first camera may mean a zoom ratio range that is greater than the first optical zoom ratio (710) of the first camera (e.g., x0.5) and less than the second optical zoom ratio (720) of the second camera (e.g., x1.0). In addition, the digital zoom range (725) of the second camera may mean a zoom ratio range that is greater than the second optical zoom ratio (720) of the second camera and less than the third optical zoom ratio (730) of the third camera (e.g., x5.0). In addition, the digital zoom range (735) of the third camera may mean a range that is greater than the third optical zoom ratio (730) of the third camera.

[0112] In one embodiment, the electronic device may display the first original image by cropping and upscaling it at a zoom ratio of x0.9 using a digital zoom method. In this case, since the flowerpot (794) was clearly captured in the first original image, even if the displayed size is enlarged, the bokeh level may still be 0.

[0113] In one embodiment, when the user changes the zoom ratio to x1.0, the electronic device can determine that the zoom ratio x1.0 corresponds to the second optical zoom ratio (720), which is the analog optical zoom ratio of the second camera, and can acquire a second original image using the second camera. In this case, the electronic device can deactivate the first camera and activate the second camera to acquire the second original image.

[0114] In one embodiment, the second camera may have different optical characteristics from the first camera, and thus the depth of field of the second camera may differ from that of the first camera. Since the optical depth of field of the second camera is shallower than that of the first camera, the flowerpot may not fall within the depth of field of the second camera. In this case, the flowerpot (796) may be captured blurry in the second original image depending on the optical characteristics (or depth of field) of the second camera, and the bokeh level, which is the degree of blurriness, may have a value of 5.

[0115] As described above, the bokeh level of the flower pot, which is a background subject, is maintained at 0 in the digital zoom section (715) of the first camera (e.g., x0.5 or more and less than x1.0), and when it reaches x1.0, which is the second optical zoom ratio (720) of the second camera, the bokeh level of the flower pot may be abruptly changed to 5. Similarly, the bokeh level of the flower pot, which is a background subject, is maintained at 5 in the digital zoom section (725) of the second camera (e.g., x1.0 or more and less than x5.0), and when it reaches x5.0, which is the third optical zoom ratio (730) of the third camera, the bokeh level of the flower pot may be abruptly changed to 9.

[0116] In this way, according to the embodiments of FIGS. 5 to 7, when the analog optical zoom ratio of another camera is changed in the digital zoom section (715, 725, 735), the bokeh level (or degree of blur) of some background subjects changes discontinuously, which may give a sense of incongruity to the user.

[0117] Hereinafter, various embodiments for providing a natural and gradual bokeh effect for background subjects, such as using a single analog optical zoom lens over the entire zoom magnification range of a hybrid zoom, will be described with reference to FIGS. 8 to 15.

[0118] FIG. 8 is a block diagram of an electronic device according to various embodiments.

[0119] Referring to FIG. 8, an electronic device (300) according to various embodiments may include a plurality of cameras (310), a display (830), a ToF sensor (840), a processor (850), and a memory (820). Even if at least some of the illustrated configurations are omitted or replaced with other configurations, various embodiments of the present document may be implemented. In addition to the illustrated configurations, the electronic device (300) may further include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1. At least some of the respective configurations of the illustrated (or not illustrated) electronic device (300) (e.g., the processor (850), the memory (820)) may be disposed within a housing of the electronic device (300), and at least some of the other configurations (e.g., the display (830), the cameras (310)) may be at least partially visually exposed to the outside of the housing. At least some of the components of the electronic device (300) may be operatively, functionally and / or electrically connected to one another.

[0120] According to one embodiment, the display (830) can display image information provided from the processor (850). The display (830) can be implemented as any one of a liquid crystal display (LCD), a light-emitting diode (LED) display, and an organic light-emitting diode (OLED) display, but is not limited thereto. The display (830) can be configured as a touch screen that detects a touch and / or proximity touch (or hovering) input using a part of a user's body (e.g., a finger) or an input device (e.g., a stylus pen). The display (830) can include at least some of the configurations and / or functions of the display module (160) of FIG. 1. At least a part of the display (830) can be flexible, and can also be implemented as a foldable display or a rollable display.

[0121] According to one embodiment, the electronic device (300) may include at least one camera on the front and / or the back. As described with reference to FIG. 3, the electronic device (300) may include three cameras (e.g., a first camera (312), a second camera (314), and a third camera (316)) on the back of the housing where the back cover is disposed; however, the number of cameras included in the electronic device (300) is not limited thereto. In this document, the electronic device (300) is described as including a first camera (312), a second camera (314), and a third camera (316); however, various embodiments of this document may be implemented even when the electronic device (300) includes two cameras or four or more cameras.

[0122] According to one embodiment, the cameras (310) may further include at least some of the components and / or functions of the camera module (180) of FIG. 2, such as a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (820) (250), and an image signal processor (260).

[0123] According to one embodiment, the cameras (310) are arranged adjacent to each other so that they can capture images in substantially the same direction. Accordingly, when switching from the digital zoom range of the first camera (312) to the analog optical zoom range of the second camera (314), and when switching from the digital zoom range of the second camera (314) to the analog optical zoom range of the third camera (316), the change in the angle of view of the image can be substantially continuous.

[0124] In one embodiment, the cameras (310) may have different optical characteristics. For example, the first camera (312), the second camera (314), and the third camera (316) may have different lens characteristics, such as angle of view, focal length, aperture, lens material, structure, refractive index, or refractive / diffraction characteristics, and / or may have different sensor characteristics, such as sensor pitch and number of pixels.

[0125] According to one embodiment, the cameras (310) may each have fixed values ​​for optical characteristics such as a lens focal length (f), an aperture number (Fno), and a sensor pitch. Accordingly, the optical zoom magnification of the first camera (312) (e.g., x0.5), the optical zoom magnification of the second camera (314) (e.g., x1.0), and the optical zoom magnification of the third camera (316) (e.g., x5.0) may each have fixed values.

[0126] According to one embodiment, since the optical characteristics of the cameras (310) have fixed values, the depth of field of each camera can be calculated depending on the distance between the camera and the main subject. For example, the depth of field (DoF) can be determined by the optical characteristics of the camera, such as the sensor pitch, lens focal length, and lens aperture, and the shooting environment, such as the distance to the subject. However, since the sensor pitch, lens focal length, and lens aperture of each camera have fixed values, the depth of field can depend only on the distance between the camera and the main subject.

[0127] According to one embodiment, the ToF sensor (840) can measure the distance to an external object. For example, the ToF sensor (840) can include at least one light emitting unit that outputs a signal (e.g., laser, infrared) and at least one light receiving unit that receives a signal output from the light emitting unit and reflected by an object. The ToF sensor (840) can be placed on the rear cover of the electronic device (300) on which the cameras (310) are placed, and can measure the distance to subjects of images captured by the cameras (310).

[0128] According to another embodiment, the electronic device (300) may not include a ToF sensor (840) and may determine the distance to each subject by analyzing an image captured by one camera or a stereo image acquired from two or more cameras.

[0129] According to one embodiment, the memory (820) may temporarily or permanently store various data, including volatile memory and non-volatile memory. The memory (820) may include at least some of the configuration and / or functions of the memory (130) of FIG. 1 and may store the program (140) of FIG. 1. The memory (820) may store various instructions that may be executed by the processor (850). Such instructions may include control commands such as arithmetic and logical operations, data movement, and input / output that may be recognized by the processor (850).

[0130] According to one embodiment, the memory (820) may store a lookup table (LUT) (or 3D LUT) that maps the zoom ratio of the camera, the distance from the main subject, the distance from the background subject, and the bokeh level to be applied to the background subject. According to one embodiment, the LUT may be pre-calculated according to the optical characteristics of each camera included in the electronic device (300). For example, electronic devices of a specific model having the same optical characteristics of each camera may have the same LUT. The electronic device (300) may generate and store the LUT based on the information about the optical characteristics of each camera stored in the memory (820), or an LUT generated by another device (e.g., an LUT generating device) in a process step of the electronic device (300) may be stored in the memory (820).

[0131] According to one embodiment, the processor (850) may be configured as one or more processors capable of performing calculations or data processing related to control and / or communication of each component of the electronic device (300). The processor (850) may include at least some of the configurations and / or functions of the processor (120) of FIG. 1. The calculation and data processing functions that the processor (850) may implement on the electronic device (300) are not limited, but this document will describe in detail various embodiments for providing continuity of the bokeh effect when the zoom ratio is changed by adding a virtual bokeh effect to the background subject of an image through image processing. The operations of the processor (850) described below may be performed by loading instructions stored in the memory (820).

[0132] In this document, the description that the processor (850) can perform a certain operation (or function, work, task) may be interpreted to mean substantially the same as that an instruction (or command, computer program) that causes the electronic device (300) (or the processor (850)) to perform the corresponding operation is stored in the memory (820) (e.g., non-volatile memory, storage). In addition, the description that the processor (850) can perform a certain operation may be interpreted to mean substantially the same as that at least one processor, without specifying the operation, can perform the corresponding operation.

[0133] According to one embodiment, the processor (850) can activate at least one camera to acquire an image when a camera application (or an application using camera resources) is executed. The processor (850) can provide a user interface (UI) that can set a zoom factor when the camera application is executed. For example, the UI can be provided as selectable items corresponding to various zoom factors (e.g., x0.5, x1.0, x5.0), or in the form of a scrollable bar that can select a specific zoom factor, and / or can be provided so that the zoom factor can be adjusted according to a user's multi-touch interaction (e.g., pitch and zoom).

[0134] According to one embodiment, the electronic device (300) may provide a hybrid zoom function. The hybrid zoom function may include analog optical zoom and digital zoom, and may utilize analog optical zoom in some zoom magnification ranges and digital zoom in other zoom magnification ranges. For example, the processor (850) uses the analog optical zoom of the first camera (312) at an optical zoom magnification of x0.5 of the first camera (312), crops the first original image acquired using the first camera (312) and processes it with digital zoom in a first section that is greater than the optical zoom magnification of the first camera (312) and less than x1.0 of the optical zoom magnification of the second camera (314), uses the analog optical zoom of the second camera (314) at an optical zoom magnification of x1.0 of the second camera (314), crops the second original image acquired using the second camera (314) and processes it with digital zoom in a second section that is greater than the optical zoom magnification of the second camera (314) and less than x5.0 of the optical zoom magnification of the third camera (316), and processes it with analog optical zoom of the third camera (316) at an optical zoom magnification of x5.0 of the third camera (316). By using the zoom, the third original image acquired using the third camera (316) can be cropped and processed with digital zoom in a section larger than the optical zoom magnification of the third camera (316).

[0135] In one embodiment, the processor (850) may capture an image using a camera corresponding to the current zoom ratio and display the image through the display (830), and deactivate other cameras.

[0136] According to one embodiment, the processor (850) may crop and upscale an original image acquired through a specific camera when the current zoom ratio falls within a digital zoom range (e.g., a first range, a second range, a third range). In this case, as described with reference to FIGS. 4 to 7 , when the zoom ratio changes from a digital zoom range of a specific camera to an analog optical zoom ratio of another camera according to a user's selection, a phenomenon in which the bokeh level of a background subject changes discontinuously may occur. To prevent this phenomenon, the processor (850) may apply a virtual bokeh effect to the background subject of the cropped image through image processing.

[0137] According to one embodiment, the processor (850) may measure the distance between the main subject and at least one background subject included in a cropped image obtained by cropping the original image according to the zoom magnification. For example, the processor (850) may measure the distance to each subject based on the sensing value of the ToF sensor (840), and / or may measure the distance to each subject through analysis of a single image or stereo image. The processor (850) may convert the shooting environment in a three-dimensional space into an actual distance, and then determine a virtual depth of field and / or a virtual bokeh level of each background subject by considering the zoom magnification and the distance to the subject.

[0138] According to one embodiment, the processor (850) may divide the crop image into regions including the main subject and each background subject based on the measured distances to each subject. For example, after dividing the region of the main subject, the processor (850) may divide the region of the background subjects whose distances are within a predetermined range based on the shooting direction of the camera, and designate the remaining background subjects that are further away from the main subject (e.g., 3.0 m) as a single region and divide them.

[0139] According to one embodiment, the processor (850) may determine a bokeh level (or virtual bokeh level) to be applied to background subjects. Bokeh may refer to a phenomenon in which subjects outside the depth of field that are not in focus in a captured image appear blurred. The bokeh level may correspond to the degree to which a specific subject is blurred in the captured image. Although the bokeh effect is a phenomenon that occurs based on the optical characteristics of the camera, the electronic device (300) providing hybrid zoom may enable the bokeh effect to be continuously recognized even in a section where the digital zoom changes to an analog optical zoom through image processing.

[0140] According to one embodiment, the processor (850) may determine a bokeh level (or virtual bokeh level) to be applied to each background subject at the zoom ratio based on the optical characteristics of the first camera (312) and the optical characteristics of the second camera (314) when the zoom ratio selected by the user falls within the digital zoom range (or first range) of the first camera (312). In addition, the processor (850) may determine a bokeh level (or virtual bokeh level) to be applied to each background subject at the zoom ratio based on the optical characteristics of the second camera (314) and the optical characteristics of the third camera (316) when the zoom ratio selected by the user falls within the digital zoom range (or second range) of the second camera (314). The processor (850) may determine a bokeh level to be applied to each background subject when the captured image includes a plurality of background subjects.

[0141] According to one embodiment, the processor (850) may determine the bokeh level to be applied to the background subject based on the first optical bokeh level applied to the background subject in the first original image captured by the first camera (312) according to the optical characteristics of the first camera (312), and the second optical bokeh level applied to the background subject when capturing the background subject with the second camera (314) according to the optical characteristics of the second camera (314), when the zoom ratio selected by the user falls within the digital zoom range of the first camera (312). For example, the depth of field is determined based on the optical characteristics of the camera (e.g., sensor pitch, lens focal length, aperture number) and the distance from the main subject, and since the optical characteristics of the cameras (310) of the electronic device (300) have fixed values, if the distance from the main subject has a specific value, the optical bokeh level to be applied to the background subject can be mathematically calculated based on the optical characteristics of the camera. The processor (850) can determine a virtual bokeh level to be applied to a corresponding background subject by using a first optical bokeh level that appears optically when shooting with the first camera (312) for a specific subject in an image and a second optical bokeh level that appears optically when shooting with the second camera (314).

[0142] According to one embodiment, the processor (850) may determine a bokeh level to be applied to a background subject based on a second optical bokeh level applied to a background subject in a second original image captured by the second camera (314) according to the optical characteristics of the second camera (314), and a third optical bokeh level applied to a background subject when capturing the background subject with the third camera (316) according to the optical characteristics of the third camera (316), when the zoom ratio selected by the user falls within the digital zoom range of the second camera (314).

[0143] According to one embodiment, when the zoom ratio changes from the digital zoom range of the first camera (312) to the optical zoom ratio of the second camera (314), the processor (850) may apply a bokeh effect to at least one background subject included in the crop image that falls within the depth of field of the first original image and does not fall within the depth of field of the second original image.

[0144] According to one embodiment, the processor (850) may interpolate the first optical bokeh level of the first camera (312) and the second optical bokeh level of the second camera (314) applied to a specific background subject based on a zoom factor selected by the user, a first optical zoom factor of the first camera (312) and a second optical zoom factor of the second camera (314) to determine a bokeh level to be applied to the corresponding background subject. In addition, the processor (850) may interpolate the second optical bokeh level of the second camera (314) and the third optical bokeh level of the third camera (316) applied to the specific background subject based on a zoom factor selected by the user, a second optical zoom factor of the second camera (314) and a third optical zoom factor of the third camera (316) to determine a bokeh level to be applied to the corresponding background subject. The method of interpolating optical bokeh levels is described in more detail with reference to Fig. 9.

[0145] According to one embodiment, the processor (850) can check the virtual bokeh level mapped to the current zoom ratio, the distance to the main subject, and the distance to the background subject from the LUT stored in the memory (820), and apply the checked virtual bokeh level to the corresponding background subject. A method for determining the virtual bokeh level using the 3D LUT will be described in more detail with reference to FIGS. 10 to 12.

[0146] According to one embodiment, the processor (850) may correct the crop image by applying a bokeh effect to the background subject based on the determined bokeh level for the background subject. For example, the processor (850) may apply the determined virtual bokeh level to each segmented image including the background subject. According to one embodiment, the processor (850) may adjust pixel data of pixels in the background subject area using a Gaussian blur filter. The processor (850) may determine the kernel size (x, y) and standard deviation (σ) of the Gaussian blur filter based on the determined bokeh level. For example, as the virtual bokeh level increases, a larger kernel size and / or standard deviation may be applied to provide a stronger blur effect.

[0147] According to one embodiment, the processor (850) may utilize other filters used for image blurring, such as a median filter or an average filter. The processor (850) may determine the parameters of the filter based on the virtual bokeh level determined for each background subject.

[0148] According to one embodiment, the processor (850) may apply the determined virtual bokeh level to a background subject, setting the virtual bokeh level to the maximum bokeh level for the background subject, and may provide a gradual blur effect to areas within the image depending on the distance from the camera.

[0149] According to one embodiment, the processor (850) may synthesize the segmented images with the bokeh effect applied to generate a final image and display the final image through the display (830). In the final image, background subjects estimated to be outside the depth of field at the given zoom ratio may be blurred by applying a virtual bokeh effect, similar to when analog optical zoom is used at the given zoom ratio.

[0150] Instructions for performing the operations of the electronic device (300) (or processor (850)) described above may be stored on a computer-readable recording medium. The recording medium may be tangible and non-transitory. The recording medium may store one or more computer programs including the instructions.

[0151] FIG. 9 illustrates a method for determining a virtual bokeh level to be applied to a background subject by interpolating a first optical bokeh level of a first camera and a second optical bokeh level of a second camera according to one embodiment.

[0152] According to one embodiment, an electronic device (e.g., the electronic device (300) of FIG. 8) may generate a cropped image by cropping a portion of an image captured by a camera corresponding to a zoom ratio selected by a user, if the zoom ratio falls within a digital zoom range. The electronic device may determine a bokeh level to be applied to each background subject included in the cropped image at the selected zoom ratio.

[0153] According to one embodiment, the electronic device may determine a bokeh level to be applied to a background subject based on a first optical bokeh level applied to a background subject in a first original image captured by the first camera according to the optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when capturing the background subject with the second camera according to the optical characteristics of the second camera (e.g., the second camera (314) of FIG. 8), when the zoom factor selected by the user falls within the digital zoom range of the first camera (e.g., the first camera (312) of FIG. 8). The electronic device may determine a bokeh level to be applied to the background subject by interpolating the first optical bokeh level and the second optical bokeh level of the first camera applied to a specific background subject, according to the zoom factor selected by the user, the first optical zoom factor, and the second optical zoom factor of the first camera.

[0154] Referring to FIG. 9, in the shooting environment of FIG. 9, a flower pot (994), an animal (992), a tree number 1 (996), and a tree number 2 (998) may be arranged in the order of proximity from the camera. The first camera of the electronic device (e.g., the first camera (312) of FIGS. 3 and 8) may be a camera with a short focal length and a wide angle of view (e.g., a UW camera), and depending on the optical characteristics of the first camera, the animal (992), which is the main subject, the flower pot (994) among the background subjects, and the tree number 1 (996) may be located in the optical depth of field (DoF 1) of the first camera, and the tree number 2 (998) may be located outside the depth of field (DoF 1). The second camera of the electronic device (e.g., the second camera (314) of FIGS. 3 and 8) may be a camera (e.g., a W camera) having a relatively longer focal length and narrower angle of view than the first camera, and depending on the optical characteristics of the second camera, the optical depth of field (DoF 2) of the second camera may be shallower than the optical depth of field (DoF 1) of the first camera. Accordingly, only the main subject, the animal (992), may be located in the optical depth of field (DoF 2) of the second camera, and the background subjects, the flowerpot (994), the first tree (996), and the second tree (998), may be located outside the depth of field (DoF 2).

[0155] The graph of FIG. 9 illustrates a first optical bokeh level (910) and a second optical bokeh level (920) applied to background subjects depending on their distance from the camera or main subject (or focal plane), and a virtual bokeh level (930) calculated by interpolating the first optical bokeh level (910) and the second optical bokeh level (920) for a selected zoom factor (e.g., x0.9).

[0156] Referring to the graph of Fig. 9, in the case of the main subject, an animal (992), both the first optical bokeh level (910) when photographed with the first camera and the second optical bokeh level (920) when photographed with the second camera may have values ​​close to 0. Since the optical depth of field (DoF 2) of the second camera is shallower than the optical depth of field (DoF 1) of the first camera, the second optical bokeh level (920) of the background subjects may have a higher value than the first optical bokeh level (910). That is, each background subject may be expressed more blurrily in the second original image photographed with the second camera.

[0157] According to one embodiment, the first optical bokeh level (910) of the first camera applied to a background subject located at a specific distance from the camera may be determined based on the optical characteristics of the first camera. For example, when the distance to the main subject is the same, since the optical characteristics of the first camera are fixed values, the depth of field also has a fixed depth, and the bokeh effect (or degree of blur) applied to the background subject located at a specific distance may have a fixed value. Similarly, the second optical bokeh level (920) of the second camera applied to a specific background subject may be determined based on the optical characteristics of the second camera. Accordingly, the electronic device may estimate the bokeh effect (or optical bokeh level) to be applied to each background subject by confirming the distances to the main subject and each background subject without capturing an actual image through the first camera and the second camera.

[0158] According to one embodiment, the electronic device can interpolate the first optical bokeh level (910) and the second optical bokeh level (920) according to the selected zoom factor to determine a virtual bokeh level (930) to be applied to each background subject.

[0159] According to one embodiment, the electronic device can determine the bokeh level to apply to each background subject according to the following mathematical expression 2.

[0160]

[0161] In the above mathematical expression 2, R1 may be a first optical zoom factor of the first camera, R2 may be a second optical zoom factor of the second camera, Lv1 may be a first optical bokeh level applied when shooting a background subject with the first camera, Lv2 may be a second optical bokeh level applied when shooting a background subject with the second camera, and Rs may be a zoom factor selected by the user.

[0162] Referring to FIG. 9, the first optical zoom magnification of the first camera is fixed at x0.5, the second optical zoom magnification of the second camera is fixed at x1.0, and in the case of the background subject, tree number 1 (996), the first optical bokeh level (915) at the corresponding distance can be calculated as 0.9, and the second optical bokeh level (925) can be calculated as 5.2, according to the optical characteristics of the first and second cameras. When the current zoom magnification is set to x0.9, the virtual bokeh level (935) to be applied to the background subject, tree number 1 (996), can be calculated as approximately 4.3 according to the above mathematical expression 2.

[0163] According to one embodiment, the electronic device can calculate a virtual bokeh level to be applied to each background subject in the above manner, and apply a bokeh effect to the background subjects in an image cropped from the original image according to the virtual bokeh level.

[0164] FIG. 10 is a flowchart (1000) of a method for generating a LUT defining a virtual bokeh level to be applied to a background subject based on optical characteristics of a camera according to one embodiment.

[0165] According to one embodiment, an electronic device (e.g., electronic device (300) of FIG. 8) may store a lookup table (LUT) (or 3D LUT, virtual bokeh table) that maps a zoom ratio of a camera, a distance to a main subject, and a distance to a background subject and a bokeh level to be applied to the background subject in a memory (e.g., memory (820) of FIG. 8).

[0166] In one embodiment, the LUT may be pre-calculated based on the optical characteristics of each camera included in the electronic device. For example, electronic devices of a specific model with identical camera optical characteristics may have identical LUTs. The electronic device may generate and store the LUT based on information about the optical characteristics of each camera stored in memory, or the LUT may be pre-stored during the manufacturing process of the electronic device.

[0167] The method of FIG. 10 may be performed by an electronic device, or may be performed by another device (hereinafter, LUT generation device) in a process step of the electronic device.

[0168] According to one embodiment, in operation 1010, the LUT generation device (or electronic device) may obtain information on each camera of the electronic device that is a target of LUT generation. For example, the LUT generation device may obtain model information of a plurality of cameras included in the electronic device.

[0169] According to one embodiment, in operation 1020, the LUT generation device may determine optical characteristics for each camera. For example, the LUT generation device may determine optical characteristics such as sensor pitch, focal length, and / or aperture number from model information of each camera.

[0170] In one embodiment, the LUT generation device can determine the analog optical zoom ratio of a specific camera among a plurality of cameras mounted on the electronic device. Each camera may have fixed optical characteristics, and thus the optical zoom ratio of each camera may be fixed.

[0171] The analog optical zoom ratio of each camera is shown in Table 1 as an example.

[0172] 1st camera (UW) 2nd camera (W) 3rd camera (T) Optical zoom ration x 0.5 x 1.0 x 5.0 FoV 120 degrees 84 degrees 20 degrees f.length 12 mm 24 mm 120 mm

[0173] Mathematical expression 3 mathematically expresses the operation of selecting the optical zoom ratio of a specific camera among the cameras of an electronic device.

[0174]

[0175] According to one embodiment, in operation 1030, the LUT generation device may set a shooting distance of the main subject. For example, the LUT generation device may set at least one shooting distance of the main subject for each camera.

[0176] The shooting distance for each camera is shown in Table 2 as an example.

[0177] Camera 1 (UW) Camera 2 (W) Camera 3 (T) Close range 0.5m 0.5m 0.5m 1.0m 1.0m 2.0m 2.0m 2.0m Far range 3.0m 3.0m 3.0m

[0178] The distance from the main subject can be a variable that determines the depth of field of the camera and the bokeh level to be applied to the background subject, and the shooting reference distance of the main subject can be a reference value for calculating the virtual bokeh level to be applied to the background subject when the main subject is located at that distance. For example, as shown in Table 2 above, the shooting reference distance of the first camera can be set to 0.5 m, 1.0 m, 2.0 m, and 3.0 m, and the LUT generation device can calculate the virtual bokeh level to be applied to the background subject when the main subject is located at 0.5 m, 1.0 m, 2.0 m, and 3.0 m from the first camera. In Table 2 above, the shooting reference distances of the first camera, the second camera, and the third camera are shown as being the same, but the shooting reference distances of each camera may be set to different values.

[0179] According to one embodiment, the LUT generation device can set a shooting reference distance of a main subject for each camera module.

[0180] Mathematical expression 4 mathematically expresses the operation of selecting one of the reference distances of the main subject.

[0181]

[0182] According to one embodiment, in operation 1040, the LUT generation device can estimate a virtual depth of field for each distance of the main subject for the optical characteristics of each camera.

[0183] According to one embodiment, the optical depth of field (DoF) can be obtained as the difference between the rear depth of field (Df) and the front depth of field (Dn), and the four variables that determine the depth of field can include the sensor pitch, the lens focal length, the aperture number, and the distance to the main subject. The LUT generation device can obtain the four variables that can determine the optical depth of field of each camera through operations 1020 and 1030. The formula for obtaining the optical depth of field has been described above through mathematical expression 1.

[0184] Depth of field according to the distance from the main subject is shown in Table 3 as an example.

[0185] Reference distance Camera 1 (UW) Camera 2 (W) Camera 3 (T) 0.5m 1231521.0m 4936082.0m 1971240333.0m 443554074

[0186] Referring to Table 3 above, it can be seen that the farther the main subject is from the camera, the deeper the depth of field is, and the deeper the depth of field is in the order of cameras with a wide angle of view (or a short focal length). According to one embodiment, the LUT generation device can convert the calculated depth of field to a logarithmic scale to limit the range of bokeh levels to be applied to the background subject by reducing the relative difference between the depth of field of each camera.

[0187] Table 4 shows the depth of field in Table 3 converted to a logarithmic scale.

[0188] Reference distance Camera 1 (UW) Camera 2 (W) Camera 3 (T) 0.5 m 2.1 1.2 0.3 1.0 m 2.7 1.8 0.9 2.0 m 3.3 2.4 1.5 3.0 m 3.6 2.7 1.9

[0189] In Table 4, the maximum value of the logarithmic value of the depth of field, max (L.DoF), is 3.6, and the minimum value, min (L.DoF), is 0.3. Mathematical expression 5 shows a formula for calculating the logarithmic scale of the depth of field.

[0190]

[0191] According to one embodiment, in operation 1050, the LUT generation device can construct a LUT (or virtual bokeh table) based on a two-dimensional virtual depth of field and a distance of a background subject.

[0192] In one embodiment, the LUT generation device can calculate a maximum bokeh level to apply to a background subject for each distance.

[0193] Mathematical expression 6 shows the formula for calculating the maximum bokeh level.

[0194]

[0195] When calculated as in the above mathematical expression 6, the bokeh level can have a value between 0 and 9 depending on the distance from the background subject.

[0196] Table 5 shows examples of virtual bokeh levels to be applied when the background subject is 3.0 m away from the main subject.

[0197] Main subjectBackground subject1 Camera (UW)2 Camera (W)3 Camera (T)0.5m3.0m4.26.69.01.0m3.0m2.65.07.42.0m3.0m0.93.45.73.0m3.0m0.02.54.8

[0198] Referring to Table 5 above, for a background subject at a distance of 3.0 m from an electronic device, when shooting with the first camera, if the main subject is at a distance of 0.5 m, a bokeh level of 4.2 may be applied, if the main subject is at a distance of 1.0 m, a bokeh level of 2.6 may be applied, and if the main subject is at a distance of 2.0 m, a bokeh level of 0.9 may be applied. According to one embodiment, the LUT generation device may generate a two-dimensional LUT using the distance to the main subject and the camera as variables at various distances of the background subject (e.g., in units of 0.1 m between 0.5 m and 3.0 m), and when the two-dimensional LUTs are combined, a three-dimensional LUT that maps the distance to the main subject, the distance to the camera, and the background subject and a virtual bokeh level may be generated.

[0199] FIG. 11 illustrates a method for determining a virtual bokeh level to be applied to a background subject from a LUT according to one embodiment. FIG. 12 illustrates a bokeh level to be applied to each background subject according to one embodiment.

[0200] According to one embodiment, the electronic device (300) may store a lookup table (LUT) that maps the zoom ratio of the camera, the distance from the main subject, the distance from the background subject, and the bokeh level to be applied to the background subject. According to one embodiment, the LUT may be pre-calculated according to the optical characteristics of each camera included in the electronic device (300). For example, electronic devices (300) of a specific model having the same optical characteristics of each camera may have the same LUT. The electronic device (300) may generate and store the LUT based on information about the optical characteristics of each camera stored in the memory, or the LUT may be pre-stored during a manufacturing step of the electronic device (300).

[0201] Fig. 11 is a visual representation of a 3D LUT stored in an electronic device (300). In Fig. 11, the x-axis may represent the zoom ratio of the camera, the y-axis may represent the distance to the main subject, and the z-axis may represent the distance to the background subject. In the 3D LUT, each bokeh level may be mapped and stored to a plurality of points that constitute 3D.

[0202] In a shooting environment such as Fig. 12, the distance to the current main subject, an animal (1292), may be 0.8 m, and the distances to the background subjects, a flowerpot (1294), tree number 1 (1296), and tree number 2 (1298), may be 0.5 m, 2.3 m, and 3.0 m, respectively. The user may set the zoom ratio to x0.9 through the UI, and the zoom ratio x0.9 may fall within the digital zoom range of the first camera (or UW camera).

[0203] According to one embodiment, the electronic device (300) may determine a bokeh level to be applied to a flower pot (1294) among background subjects by checking a virtual bokeh level mapped to coordinates of (x0.9, 0.8m, 0.5m) in the 3D LUT, determine a bokeh level to be applied to a tree number 1 (1296) among background subjects by checking a virtual bokeh level mapped to coordinates of (x0.9, 0.8m, 2.3m), and determine a bokeh level to be applied to a tree number 2 (1298) among background subjects by checking a virtual bokeh level mapped to coordinates of (x0.9, 0.8m, 3.0m).

[0204] According to one embodiment, if a virtual bokeh level mapped to a distance from a main subject and / or a distance from a specific background subject is not stored in the 3D LUT, the electronic device (300) may check two or more adjacent virtual bokeh levels for the distance from the main subject and / or the distance from a specific background subject, and interpolate the checked values ​​to determine a virtual bokeh level to be applied to the background subject.

[0205] According to one embodiment, as verified in the 3D LUT, the virtual bokeh levels mapped to the background subjects, i.e., flower pot (1294), tree number 1 (1296), and tree number 2 (1298), may be 0.9, 4.0, and 5.2, respectively. The electronic device (300) may apply a bokeh effect (or blurring) based on the verified virtual bokeh levels for each background subject.

[0206] FIG. 13 illustrates a bokeh level applied to a background subject when the zoom ratio is changed according to one embodiment.

[0207] According to one embodiment, an electronic device (e.g., electronic device (300) of FIG. 8) includes a plurality of cameras (e.g., first camera (312), second camera (314), and third camera (316) of FIG. 8) and may provide a hybrid zoom function including analog optical zoom and digital zoom.

[0208] In the embodiment of FIG. 7, when the zoom ratio changes from the digital zoom range of a specific camera to the analog optical zoom ratio of another camera according to the user's selection, a phenomenon may occur in which the bokeh level of the background subject changes discontinuously. According to one embodiment, the electronic device can provide a continuous bokeh effect when the zoom ratio changes, unlike the embodiment of FIG. 7, by adding a virtual bokeh effect to the background subject based on the optical characteristics of each camera.

[0209] Referring to FIG. 13, when the zoom ratio is set to x0.5 according to the user's selection, the zoom ratio x0.5 corresponds to the first optical zoom ratio (1310), which is the analog optical zoom ratio of the first camera, and thus the electronic device can acquire the first original image using the first camera. In this case, since the zoom ratio x0.5 corresponds to the analog optical zoom ratio and not the digital zoom range, the electronic device may not perform an operation of applying a virtual bokeh level to the flowerpot (1392), which is a background subject.

[0210] In one embodiment, when a user changes the zoom ratio to x0.9, the electronic device can determine that the changed zoom ratio falls within the digital zoom range (1315) of the first camera. In one embodiment, the electronic device can crop and upscale the first original image according to the zoom ratio x0.9 using a digital zoom method.

[0211] According to one embodiment, the electronic device can determine the virtual bokeh level to be applied to each background subject included in the first original image. For example, the electronic device can determine virtual bokeh levels mapped to the currently set zoom ratio of x0.9 in the 3D LUT, the distance to the main subject, and the distance to each background subject. Referring to FIG. 13, the virtual bokeh level mapped to the background subject flowerpot (1394) may be 4.

[0212] In one embodiment, the electronic device can correct an image by applying the identified virtual bokeh level to each background subject. For example, the electronic device can segment the image based on the distance from the main subject and each subject, apply a bokeh effect corresponding to the virtual bokeh level to each segmented area, and then synthesize the segmented images.

[0213] In one embodiment, when the user changes the zoom factor to x1.0, the electronic device can determine that the zoom factor x1.0 corresponds to the second optical zoom factor, which is the analog optical zoom factor (1320) of the second camera, and can acquire a second original image using the second camera. In the second original image, the bokeh level applied to the background subject flowerpot (1396) may be 5.0, depending on the optical characteristics of the second camera.

[0214] Referring to FIG. 13, a flower pot as a background subject can have a virtual bokeh level of 4 applied for a zoom ratio of x0.9, and accordingly, in the section where the zoom ratio changes from x0.5 to x0.9 to x1.0, the bokeh effect for the background subject can gradually increase to provide continuity.

[0215] In one embodiment, the electronic device can check and apply the virtual bokeh level even in the digital zoom range (1325) of the second camera to provide continuity when switching to the optical zoom ratio (1330) of the third camera.

[0216] According to one embodiment, the electronic device can provide a continuous effect, such as in FIG. 13, through image processing for other image parameters, such as brightness and color. For example, the electronic device can acquire an image by activating a second camera in the background while cropping and displaying an image captured in a digital zoom range of the first camera, and can interpolate the brightness of the image acquired from the first camera and the brightness of the image acquired from the second camera based on the current zoom ratio and apply the interpolated result to the cropped image.

[0217] FIG. 14 is a diagram for explaining a Gaussian blur filter for applying a bokeh effect according to one embodiment.

[0218] According to one embodiment, an electronic device (e.g., electronic device (800) of FIG. 8) may determine a virtual bokeh level to be applied to a background subject based on the optical characteristics of each camera, and / or, if a virtual bokeh level mapped to a zoom factor, a distance to a main subject, and a distance to a virtual subject is identified from a 3D LUT, the electronic device may correct an image (e.g., an image cropped and up-scaled from an original image) to apply the virtual bokeh level. For example, the electronic device may blur an area of ​​the image that includes a background subject.

[0219] According to one embodiment, an electronic device may adjust pixel data of pixels in a background subject area using a Gaussian blur filter. A Gaussian blur filter may be a filter that blurs a portion of an image by adjusting pixel data of a specific pixel using surrounding pixels. The electronic device may calculate weights between pixels using a Gaussian function, and assign a higher weight to pixels closer to the center of a specific area and a lower weight to pixels further from the center, thereby applying a blur effect to the area.

[0220] The Gaussian function used in the Gaussian blur filter is as shown in mathematical formula 7.

[0221]

[0222] An electronic device may determine a kernel size (x, y) and a standard deviation (σ) of a Gaussian blur filter based on a virtual bokeh level for a specific area including a background subject. When the kernel size in the Gaussian blur filter is fixed to an arbitrary value, the standard deviation (or sigma) may be a parameter indicating the intensity of the blur. Accordingly, the intensity of the bokeh effect applied to the corresponding kernel may be strengthened in proportion to the value of the standard deviation. When the virtual bokeh level determined for the area including the background subject is high, the electronic device may apply a high standard deviation to increase the intensity of the bokeh effect.

[0223] According to one embodiment, the electronic device may utilize other filters used for image blurring, such as a median filter or an average filter. The electronic device may determine the parameters of the filter based on the virtual bokeh level determined for each background subject.

[0224] FIGS. 15a, 15b and 15c illustrate an operation of applying a bokeh effect to a background object from a camera image according to one embodiment.

[0225] According to one embodiment, an electronic device (e.g., the electronic device (300) of FIG. 8) can acquire an image using a camera corresponding to a zoom ratio set according to a user input. Referring to (a) of FIG. 15A, when a user selects a zoom ratio of x0.9 through a UI of a camera application, an analog optical zoom ratio of x0.5 is activated, and a first camera used in a digital zoom range of x0.5 to x1.0 is activated to acquire a first original image (1510).

[0226] According to one embodiment, when the selected zoom ratio falls within the digital zoom range, the electronic device may crop an area corresponding to the zoom ratio in an original image acquired using the camera, and upscale the area to correspond to the size of the original image. Referring to (b) of FIG. 15a, the electronic device may crop an area (1515) corresponding to a zoom ratio x0.9 in a first original image (1510), and upscale the cropped image (1520).

[0227] According to one embodiment, the electronic device can sense the distance to each subject included in the cropped image. For example, the electronic device can measure the distance to each subject in the shooting direction using a ToF sensor (e.g., the ToF sensor (840) of FIG. 8), and / or can measure the distance to each subject through analysis of a single image or stereo image. Referring to (c) of FIG. 15a, in the cropped image (1520), the distance of the book (1532), which is the main subject, from the electronic device is measured as 0.8 m, the distance of the flowerpot (1534) among the background subjects is measured as 2.3 m, and the remaining background subjects (1536) behind the flowerpot can be sensed to be located at a distance of 3.0 m or more from the electronic device.

[0228] According to one embodiment, the electronic device may segment a region including a main subject and each background subject in a crop image based on a distance from the electronic device. Referring to (d) of FIG. 15b, the electronic device may segment the crop image (1520) into a segmented image (1542) including a main subject book located at a distance of 0.8 m, a segmented image (1544) including a background subject flower pot located at a distance of 2.3 m, and a segmented image (1546) including the remaining background subjects located at a distance of 3.0 m or more.

[0229] According to one embodiment, the electronic device can determine a bokeh level to be applied to an image including a background subject among the segmented images (1542, 1544, 1546). The electronic device can determine the bokeh level to be applied to the background subject based on a first optical bokeh level applied to the background subject in a first original image captured by the first camera according to the optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when capturing the background subject with the second camera according to the optical characteristics of the second camera, when the zoom magnification selected by the user falls within the digital zoom range of the first camera. The electronic device can interpolate the first optical bokeh level and the second optical bokeh level of the first camera applied to a specific background subject according to the zoom magnification selected by the user, the first optical zoom magnification and the second optical zoom magnification of the first camera, and determine the bokeh level to be applied to the corresponding background subject.

[0230] According to one embodiment, the electronic device may store a lookup table (LUT) that maps a zoom factor of the camera, a distance to a main subject, and a distance to a background subject and a bokeh level to be applied to the background subject in a memory (e.g., memory (820) of FIG. 8). The electronic device may determine a virtual bokeh level mapped to the currently selected zoom factor, the measured distance to the main subject, and each measured distance to the background subject from the 3D LUT.

[0231] Referring to (e) of FIG. 15b, the electronic device can confirm that the current zoom magnification is x0.9, the distance from the main subject is 0.8 m, and the virtual bokeh level of the background subject (1534) (e.g., a flower pot) located at a distance of 2.3 m is 4.0, and the virtual bokeh levels of the remaining background subjects (1536) located at a distance of 3.0 m or more are 5.2. The electronic device can add a bokeh effect to the images based on the confirmed virtual bokeh levels for each of the segmented images (1552, 1554, 1556). For example, the electronic device can blur each of the segmented images using a Gaussian blur filter, and determine parameters of the Gaussian blur filter based on the virtual bokeh levels.

[0232] According to one embodiment, the electronic device may generate a final image by synthesizing segmented images with added bokeh effect and display the final image through a display. Referring to (f) of FIG. 15c, in the final image (1560), the main subject, a book (1562), may be clearly displayed, and background subjects (1564, 1566) may be provided with a blur effect. Since the current zoom ratio is x0.9, which is the digital zoom range of the first camera, the blur effect applied to each background subject may be greater than the first optical bokeh level applied to the corresponding background subject when shooting with the first camera, and may be lower than the second optical bokeh level applied to the corresponding background subject when shooting with the second camera.

[0233] Users want to experience the natural, smooth variable focal length and bokeh effect of a standard digital camera with a single analog high-magnification zoom lens, even on electronic devices. However, when a hybrid zoom operation provides images captured by a different camera at a fixed magnification, the bokeh effect applied to each object within the image may vary due to differences in optical characteristics from the previous camera. Consequently, the bokeh effect may change abruptly during the zoom operation, failing to satisfy the user's desire for a natural, smooth bokeh effect.

[0234] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains.

[0235] An electronic device (300) according to various embodiments of the present document may include a display (830), a plurality of cameras (310) arranged adjacent to each other and having different optical characteristics, a memory (820), and at least one processor (850) operatively connected to the display, the plurality of cameras, and the memory.

[0236] According to one embodiment, the memory may be executed by at least one processor, and when executed, the electronic device determines a first zoom magnification related to image capturing based on a user input, acquires a first image using a first camera corresponding to the first zoom magnification among the plurality of cameras, and when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of the second camera among the plurality of cameras, generates a crop image by cropping a portion of the first image based on the first zoom magnification, determines a bokeh level to be applied to a background subject included in the crop image at the first zoom magnification based on the optical characteristic of the first camera and the optical characteristic of the second camera, and corrects the crop image by applying a bokeh effect to the background subject based on the determined bokeh level, and displays the corrected image on the display. You can store instructions that cause it to be displayed.

[0237] According to one embodiment, the memory may store instructions that cause the electronic device to determine a bokeh level to be applied to the background subject based on a first optical bokeh level applied to the background subject in the first image according to optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when the background subject is captured by the second camera according to optical characteristics of the second camera.

[0238] According to one embodiment, the memory may store instructions that cause the electronic device to interpolate the first optical bokeh level and the second optical bokeh level based on the first zoom magnification, the first optical zoom magnification, and the second optical zoom magnification to determine a bokeh level to be applied to the background subject.

[0239] According to one embodiment, the memory may store instructions that cause the electronic device to apply the bokeh effect to at least one background subject included in the crop image, the background subject falling within the depth of field of the first image and not falling within the depth of field of the second image.

[0240] According to one embodiment, the optical characteristics of the first camera may include at least one of a focal length, a sensor pitch, or an aperture number of the first camera, and the optical characteristics of the second camera may include at least one of a focal length, a sensor pitch, or an aperture number of the second camera.

[0241] In one embodiment, the first camera and the second camera may have fixed optical characteristics.

[0242] According to one embodiment, the memory may store instructions that cause the electronic device to determine a distance to each of the background subjects included in the crop image, and determine a bokeh level to apply to each of the background subjects based on the distance to the main subject and the distance to each of the background subjects.

[0243] According to one embodiment, the memory stores a lookup table (LUT) that maps a zoom ratio of a camera, a distance from a main subject, and a distance from a background subject to a bokeh level to be applied to the background subject, and the memory can store instructions that cause the processor to determine a bokeh level to be applied to each of the background subjects of the cropped image based on the LUT.

[0244] According to one embodiment, the LUT may be preset based on properties of each camera included in the electronic device.

[0245] According to one embodiment, the memory may store instructions causing the processor to divide the cropped image into a main subject area including a main subject, and at least one background subject area each including a background subject, apply a bokeh effect to each of the at least one background subject area based on a bokeh level determined according to a distance from the background subject, and synthesize each of the background subject areas to which the bokeh effect is applied and the main subject area, thereby generating the corrected image.

[0246] According to one embodiment, the memory may store instructions that cause the processor to deactivate the first camera and acquire a second image using the second camera when a zoom ratio related to image capturing is changed from the first zoom ratio to a second zoom ratio within a digital zoom range of the second camera based on a user input.

[0247] According to one embodiment, the electronic device may further include a ToF sensor for measuring the distance between the main subject and the background subject.

[0248] A method performed in an electronic device according to various embodiments of the present document may include an operation of determining a first zoom magnification related to image capturing based on a user input, an operation of acquiring a first image using a first camera corresponding to the first zoom magnification among a plurality of cameras of the electronic device, an operation of generating a crop image by cropping a portion of the first image based on the first zoom magnification when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras, an operation of determining a bokeh level to be applied to a background subject included in the crop image at the first zoom magnification based on the optical characteristics of the first camera and the optical characteristics of the second camera, an operation of correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level, and an operation of displaying the corrected image.

[0249] According to one embodiment, the operation of determining the bokeh level to be applied to the background subject may include an operation of determining the bokeh level to be applied to the background subject based on a first optical bokeh level applied to the background subject in the first image according to the optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when the background subject is captured by the second camera according to the optical characteristics of the second camera.

[0250] According to one embodiment, the operation of determining the bokeh level to be applied to the background subject may include an operation of determining the bokeh level to be applied to the background subject by interpolating the first optical bokeh level and the second optical bokeh level according to the first zoom magnification, the first optical zoom magnification, and the second optical zoom magnification.

[0251] According to one embodiment, the operation of determining a bokeh level to be applied to the background subject may include an operation of determining a distance to each of the background subjects included in the crop image, and an operation of determining a bokeh level to be applied to each of the background subjects based on the distance to the main subject and the distance to each of the background subjects.

[0252] According to one embodiment, the electronic device may store a lookup table (LUT) that maps the zoom ratio of the camera, the distance to the main subject, and the distance to the background subject and the bokeh level to be applied to the background subject.

[0253] According to one embodiment, the operation of determining a bokeh level to be applied to the background subject may include an operation of determining a bokeh level to be applied to each of the background subjects of the crop image based on the LUT.

[0254] According to one embodiment, the LUT may be preset based on properties of each camera included in the electronic device.

[0255] According to one embodiment, the method may include an operation of dividing the cropped image into a main subject area including a main subject, and at least one background subject area each including a background subject, an operation of applying a bokeh effect to each of the at least one background subject area based on a bokeh level determined according to a distance from the background subject, and an operation of synthesizing each of the background subject areas to which the bokeh effect is applied and the main subject area to generate the corrected image.

[0256] According to various embodiments of the present document, a computer-readable non-transitory recording medium is provided, which performs the following operations: determining a first zoom magnification related to image capturing based on a user input; acquiring a first image using a first camera corresponding to the first zoom magnification among a plurality of cameras of an electronic device; generating a crop image by cropping a portion of the first image based on the first zoom magnification when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras; determining a bokeh level to be applied to a background subject included in the crop image at the first zoom magnification based on the optical characteristics of the first camera and the optical characteristics of the second camera; correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level; and displaying the corrected image. Instructions can be stored.

[0257] According to various embodiments of the present document, in an electronic device providing a hybrid zoom function, an electronic device and an image processing method of the electronic device can be provided, which can obtain a natural zoom transition effect by processing an image so that the angle of view and bokeh effect continuously vary in relation to the zoom ratio.

[0258] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains.

[0259] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0260] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0261] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0262] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0263] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0264] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (300), Display (830); A plurality of cameras (310) arranged adjacent to each other and having different optical characteristics; memory (820); and At least one processor (850) operatively connected to the display, the plurality of cameras, and the memory, The above memory is executable by at least one processor, and when executed, the electronic device: Based on user input, determine the first zoom factor associated with image capture, A first image is acquired using a first camera corresponding to the first zoom ratio among the plurality of cameras, When the first zoom ratio belongs to a digital zoom range of the first camera that is greater than a first optical zoom ratio corresponding to an optical characteristic of the first camera and lower than a second optical zoom ratio corresponding to an optical characteristic of the second camera among the plurality of cameras, a cropped image is generated by cropping a portion of the first image based on the first zoom ratio, Based on the optical characteristics of the first camera and the optical characteristics of the second camera, a bokeh level to be applied to a background subject included in the crop image at the first zoom ratio is determined, Based on the determined bokeh level, the crop image is corrected by applying a bokeh effect to the background subject, and An electronic device storing instructions for displaying the corrected image on the display.

2. In paragraph 1, The above memory, the electronic device, An electronic device storing instructions for determining a bokeh level to be applied to the background subject based on a first optical bokeh level applied to the background subject in the first image according to the optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when the background subject is photographed with the second camera according to the optical characteristics of the second camera.

3. In paragraph 2, The above memory, the electronic device, An electronic device storing instructions for determining a bokeh level to be applied to the background subject by interpolating the first optical bokeh level and the second optical bokeh level according to the first zoom magnification, the first optical zoom magnification, and the second optical zoom magnification.

4. In any one of paragraphs 1 to 3, The above memory, the electronic device, An electronic device storing instructions for applying the bokeh effect to at least one background subject included in the crop image, the background subject falling within the depth of field of the first image and not falling within the depth of field of the second image.

5. In any one of paragraphs 1 to 4, The optical characteristics of the first camera include at least one of a focal length, a sensor pitch, or an aperture number of the first camera, and An electronic device wherein the optical characteristics of the second camera include at least one of a focal length, a sensor pitch, or an aperture number of the second camera.

6. In any one of paragraphs 1 to 5, The first camera and the second camera are electronic devices having fixed optical characteristics.

7. In any one of paragraphs 1 to 6, The above memory, the electronic device, Determine the distance of each background subject included in the above crop image, An electronic device storing instructions for determining a bokeh level to be applied to each of the background subjects based on a distance from the main subject and a distance from each of the background subjects.

8. In any one of paragraphs 1 to 7, The above memory is, Save a lookup table (LUT) that maps the camera's zoom ratio, distance to the main subject, and distance to the background subject, and the bokeh level to be applied to the background subject. The above memory, the above processor, An electronic device storing instructions for determining a bokeh level to be applied to each of the background subjects of the cropped image based on the LUT.

9. In paragraph 8, The above LUT is an electronic device that is preset based on the properties of each camera included in the electronic device.

10. In any one of paragraphs 1 to 9, The above memory, the above processor, Divide the crop image into a main subject area including a main subject, and at least one background subject area each including a background subject, Applying a bokeh effect to each of the at least one background subject area based on a bokeh level determined according to the distance from the background subject, and An electronic device storing instructions for generating the corrected image by synthesizing each background subject area to which the bokeh effect is applied and the main subject area.

11. In a method performed in an electronic device, An operation for determining a first zoom ratio related to image capturing based on user input; An operation of acquiring a first image using a first camera corresponding to the first zoom ratio among a plurality of cameras of the electronic device; An operation of generating a cropped image by cropping a portion of the first image based on the first zoom magnification, when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras; An operation of determining a bokeh level to be applied to a background subject included in the crop image at the first zoom ratio based on the optical characteristics of the first camera and the optical characteristics of the second camera; An operation of correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level; and A method comprising the action of displaying the above-mentioned corrected image.

12. In paragraph 11, The action of determining the bokeh level to be applied to the above background subject is: A method comprising an operation of determining a bokeh level to be applied to the background subject based on a first optical bokeh level applied to the background subject in the first image according to the optical characteristics of the first camera, and a second optical bokeh level applied to the background subject when the background subject is photographed with the second camera according to the optical characteristics of the second camera.

13. In paragraph 11 or 12, The action of determining the bokeh level to be applied to the above background subject is: An operation of determining the distance of each background subject included in the above crop image; and A method comprising an action of determining a bokeh level to be applied to each of the background subjects based on the distance to the main subject and the distance to each of the background subjects.

14. In any one of paragraphs 11 to 13, The above electronic device, Save a lookup table (LUT) that maps the camera's zoom ratio, distance to the main subject, and distance to the background subject, and the bokeh level to be applied to the background subject. The action of determining the bokeh level to be applied to the above background subject is: A method comprising an operation of determining a bokeh level to be applied to each of the background subjects of the cropped image based on the LUT.

15. In a non-transitory computer-readable recording medium, An operation for determining a first zoom ratio related to image capturing based on user input; An operation of acquiring a first image using a first camera corresponding to the first zoom ratio among a plurality of cameras of an electronic device; An operation of generating a cropped image by cropping a portion of the first image based on the first zoom magnification, when the first zoom magnification belongs to a digital zoom range of the first camera that is greater than a first optical zoom magnification corresponding to an optical characteristic of the first camera and lower than a second optical zoom magnification corresponding to an optical characteristic of a second camera among the plurality of cameras; An operation of determining a bokeh level to be applied to a background subject included in the crop image at the first zoom ratio based on the optical characteristics of the first camera and the optical characteristics of the second camera; An operation of correcting the crop image by applying a bokeh effect to the background subject based on the determined bokeh level; and A recording medium storing instructions for performing an operation of displaying the above-mentioned corrected image.

Images