Method for capturing image including dynamic scene, and electronic device therefor

Abstract

An electronic device according to one embodiment of the present disclosure may comprise: one or more cameras; a display; at least one processor; and a memory for storing instructions. When executed by the at least one processor, the instructions enable the electronic device to: analyze the movement of the electronic device or an object in a preview image while displaying, on the display, the preview image acquired from the one or more cameras; calculate movement information related to the preview image on the basis of the analysis; set at least one capturing parameter on the basis of the calculated movement information in response to the reception of a user input for image capturing; and acquire an image captured on the basis of the set capturing parameter. Other various embodiments identified through the present document are possible.

Description

Method for capturing images including dynamic scenes and electronic device thereof

[0001] The present disclosure relates to a method for capturing an image including a dynamic scene and an electronic device thereof.

[0002] As the possession of portable electronic devices has become more common in daily life, the use of cameras on these devices has increased significantly. Taking pictures with these devices has become an essential function, to the point where camera functions are becoming a criterion for selecting a portable electronic device. Recently, as camera functions implemented in portable electronic devices have become more sophisticated, technologies are being applied to capture clear images in various shooting environments. For example, an automatic shooting mode can be provided that automatically adjusts shooting parameters such as ISO (International Organization for Standardization) sensitivity, shutter speed, exposure time, white balance, or focus to obtain optimal images depending on the user's shooting situation.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] An electronic device according to an embodiment of the present disclosure may include one or more cameras, a display, at least one processor, and a memory storing instructions. The instructions, when executed by the at least one processor, may cause the electronic device to analyze movement of the electronic device or an object within the preview image while displaying a preview image acquired from the one or more cameras on the display. The instructions, when executed by the at least one processor, may cause the electronic device to compute movement information related to the preview image based on the analysis. The instructions, when executed by the at least one processor, may cause the electronic device to set at least one shooting parameter based on the computed movement information in response to receiving a user input for capturing an image. The instructions, when executed by the at least one processor, may cause the electronic device to acquire a captured image based on the set shooting parameter.

[0005] A method according to one embodiment of the present disclosure may include an operation of analyzing a movement of an electronic device or an object within a preview image while displaying a preview image acquired from one or more cameras. The method may include an operation of calculating movement information related to the preview image based on the analysis. The method may include an operation of setting at least one shooting parameter based on the calculated movement information in response to receiving a user input for capturing an image. The method may include an operation of acquiring a captured image based on the set shooting parameter.

[0006] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0007] FIG. 2 is a drawing illustrating the configuration of an electronic device according to one embodiment.

[0008] FIG. 3 is a drawing illustrating a detailed configuration of an electronic device according to one embodiment.

[0009] FIG. 4 is a diagram illustrating a method for capturing an image including a dynamic scene in an electronic device, according to one embodiment.

[0010] FIG. 5 is a flowchart illustrating a method of operating an electronic device according to one embodiment.

[0011] FIG. 6 is a diagram illustrating a method for analyzing movement while displaying a preview screen on an electronic device, according to one embodiment.

[0012] FIG. 7 is a diagram illustrating a method of setting shooting parameters and acquiring an image in an electronic device according to one embodiment.

[0013] FIG. 8 is a flowchart illustrating operations performed by a third application according to one embodiment.

[0014] FIG. 9 is a flowchart illustrating operations performed by a first application according to one embodiment.

[0015] FIG. 10 is a flowchart illustrating operations performed by a second application according to one embodiment.

[0016] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0017] Hereinafter, various embodiments disclosed in this document will be described with reference to the attached drawings. It should be understood that this is not intended to limit the various embodiments of the present invention to a specific form, but rather encompasses various modifications, equivalents, and / or alternatives of the present invention.

[0018] Among the camera's photo shooting modes, the auto shooting mode calculates shooting parameters (e.g., ISO sensitivity, shutter speed, exposure time, or white balance) based on the surrounding environmental data collected from the sensor to obtain images with optimal image quality in general situations, and performs shooting based on the calculated shooting parameters. When set to auto shooting mode, the electronic device can determine shooting parameter values ​​based on the lighting environment, assuming little subject or camera movement, to address noise issues. For example, the electronic device can reduce noise by setting the minimum ISO sensitivity value based on the current illuminance value in auto shooting mode. This noise-priority policy can lower the quality of the captured image because it is vulnerable to motion blur when shooting sports events or scenes with moving subjects.

[0019] In various embodiments of this document, when capturing an image in an automatic capture mode, capturing parameters are calculated based on movement detected from a preview image, and image capturing is performed based on the calculated capturing parameters, thereby providing various embodiments for obtaining clearer capturing results even when capturing a moving scene.

[0020] FIG. 1 is a diagram illustrating an electronic device within a network environment (100) according to one embodiment.

[0021] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0022] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0023] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0024] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0025] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0026] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0027] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0028] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0029] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0030] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0031] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0032] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0033] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0034] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0035] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0036] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0037] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, Wi-Fi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0038] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0039] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0040] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0041] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0042] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0043] FIG. 2 is a drawing illustrating the configuration of an electronic device (200) according to one embodiment.

[0044] Referring to FIG. 2, an electronic device (200) is a device that determines motion information from a preview image acquired through a camera when taking an image using an automatic shooting mode, and sets shooting parameters in consideration of the determined motion information, and may include one or more cameras (210), a display (220), one or more sensors (230), at least one processor (240), or a memory (250). In FIG. 2, the electronic device (200) may correspond to the electronic device (101) illustrated in FIG. 1.

[0045] In one embodiment, one or more cameras (210) (e.g., camera module (180) of FIG. 1) may capture a subject according to a user's operation. For example, when a camera application is executed, one or more cameras (210) may acquire a preview image including the subject on a frame-by-frame basis, and when a user input for capturing an image (e.g., shutter button input) is received while displaying the preview image, the camera may capture an image including the subject. According to various embodiments, when one or more cameras (210) are set to an automatic capturing mode, when a user input for capturing an image is received, the camera may automatically set one or more capturing parameters (e.g., ISO sensitivity, shutter speed, exposure time, or white balance) based on ambient environment data, and perform capturing based on the set one or more capturing parameters.

[0046] In one embodiment, the display (220) (e.g., the display module (160) of FIG. 1) may display images captured by one or more cameras (210). For example, the display (220) may display preview images captured from one or more cameras (210) when a camera application is running.

[0047] In one embodiment, the display (220) may be configured with at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED), a light emitting diode (LED), an active matrix organic LED (AMOLED), a flexible display, and a 3-dimensional display. In addition, some of these displays may be configured as transparent or light-transmitting so that the outside can be viewed through them. This may be configured in the form of a transparent display including a TOLED (transparent OLED).

[0048] In one embodiment, one or more sensors (230) (e.g., the sensor module (176) of FIG. 1) may detect physical movement or posture of the electronic device (200). For example, the one or more sensors (230) may detect whether movement of the electronic device (200) has occurred while a preview image is displayed through the display (220). According to various embodiments, the one or more sensors (230) may include a gyro sensor, an acceleration sensor, or various other types of sensors capable of detecting movement of the electronic device (200).

[0049] In one embodiment, the memory (250) (e.g., the memory (130) of FIG. 1) may store instructions that, when executed, control at least one processor (240) (e.g., the processor (120) of FIG. 1) to perform various operations. For example, the at least one processor (240) may perform operations for setting at least one shooting parameter based on motion information determined from a preview image.

[0050] In one embodiment, at least one processor (240) can determine whether movement of the electronic device (200) or movement of an object within the preview image has occurred while displaying a preview image acquired from one or more cameras (210) on the display (220).

[0051] In one embodiment, at least one processor (240) may obtain data regarding physical movement of the electronic device (200) from one or more sensors (230) while the preview image is displayed, and determine whether movement has occurred in the electronic device (200) based on the data. For example, the one or more sensors (230) may include various types of sensors capable of detecting movement of the electronic device (200), including a gyro sensor and / or an acceleration sensor.

[0052] In one embodiment, when at least one processor (240) determines that movement has occurred in the electronic device (200), it may determine motion information indicating the degree of movement that occurred while the preview image was displayed based on the amount of movement of the electronic device (200) obtained from the data. In this case, at least one processor (240) may determine that motion blur may occur due to shaking of the electronic device (200) or the camera (210) when capturing an image, and may determine shooting parameters by considering the motion information calculated for the electronic device (200).

[0053] In one embodiment, when at least one processor (240) determines that no movement has occurred in the electronic device (200), the processor may analyze the movement of pixels included in the preview image. According to one embodiment, the at least one processor (240) may estimate pixel movement for the entire area of ​​the preview image. The at least one processor (240) may extract optical flow for the entire area within the preview image and estimate the pixel movement based on the extracted optical flow. The at least one processor (240) may extract the optical flow from data acquired using hardware components included in the electronic device (200) and / or data acquired using a software program stored in the memory (250). The optical flow may be obtained based on dense optical flow that estimates movement in the entire area of ​​the preview image, and in cases where it is difficult to extract dense optical flow, it may be obtained based on movement of feature points such as edges or corner points detected within the preview image.

[0054] In one embodiment, at least one processor (240) may identify one or more objects from the preview image. When the at least one processor (240) receives the preview image, the at least one processor (240) may determine whether the preview image is a first frame acquired from one or more cameras (210) after the camera application is executed. When the at least one processor (240) determines that the preview image is a first frame received after the camera application is executed, the at least one processor (240) may generate a background model based on a probability distribution of pixels included in the preview image. For example, the at least one processor (240) may generate the background model based on a color or brightness value of the preview image, such as a single Gaussian model or a Gaussian mixture model. When the at least one processor (240) determines that the preview image is not a first frame received after the camera application is executed, the at least one processor (240) may update the generated background model based on the preview image. At least one processor (240) can distinguish a background area and a foreground area from the preview image based on the background model, and extract the foreground area. At least one processor (240) can identify one or more objects based on the foreground area within the preview image.

[0055] In one embodiment, at least one processor (240) may determine motion information related to the preview image based on pixel motion estimated based on optical flow for the entire area of ​​the preview image and / or objects identified in the preview image. For example, at least one processor (240) may calculate an amount of motion for each object by combining the pixel motion estimated for the entire area of ​​the preview image and one or more objects identified in the preview image, and may determine motion information indicating the degree of motion that occurred while the preview image was displayed based on the result of the calculation. In this case, at least one processor (240) may determine that motion blur may occur due to movement of the object when capturing an image, and may determine capturing parameters by considering motion information calculated for the object (or foreground area) in the preview image. According to various embodiments, when there are multiple foreground areas extracted from the preview image, at least one processor (240) may assign a weight according to the area of ​​each of the multiple foreground areas, and may calculate the motion information by combining the multiple foreground areas (or objects included in each foreground area) and the estimated pixel movement based on the weight. As another example, when there is no foreground area (or object) extracted from the preview image, at least one processor (240) may determine that motion blur may occur due to full-screen movement (or change) during image capturing, and may determine motion information indicating the degree of motion that occurred while the preview image was displayed based on the estimated pixel movement. In this case, at least one processor (240) may determine the shooting parameters by considering the motion information determined for the entire area of ​​the preview image.

[0056] In one embodiment, at least one processor (240) may receive a user input for capturing an image while displaying the preview image via the display (220). For example, the user input may correspond to a shutter button input received via a camera application.

[0057] In one embodiment, at least one processor (240) may, in response to receiving the user input, determine at least one shooting parameter for obtaining a clear image based on motion information determined while the preview image is displayed. The at least one parameter may include at least one of an ISO sensitivity or a shutter speed of one or more cameras (210). For example, when the at least one processor (240) detects movement of the electronic device (200) or the preview image while the preview image is provided in a low-light environment with a brightness value below a specified value, the at least one processor (240) may determine to have a higher ISO sensitivity value than when no motion is detected. In another example, when the at least one processor (240) detects movement of the electronic device (200) or the preview image while the preview image is provided in a low-light environment, the at least one processor (240) may determine to have a faster shutter speed than when no motion is detected. The at least one processor (240) may set the determined shooting parameter to one or more cameras (210) and obtain a captured image based on the setting.

[0058] According to various embodiments, at least one processor (240) may determine to set shooting parameters without considering motion information based on scene analysis included in the preview image while providing the preview image. For example, if the scene of the preview image is analyzed to include a moving shooting background such as a propeller, at least one processor (240) may determine the motion detected from the preview image as an environmental factor of the scene and may exclude the motion or reflect it with a lower weight when determining the shooting parameters. In addition, at least one processor (240) may also determine whether to reflect motion by considering other contexts such as location, time, or object type of the scene of the preview image when determining the shooting parameters.

[0059] FIG. 3 is a drawing illustrating a detailed configuration of an electronic device (200) according to an embodiment. The functions or operations described with reference to FIG. 3 may be understood as functions performed by at least one processor (240) (e.g., application processor) of the electronic device (200) of FIG. 2. For example, the object detection module (311), the optical flow extraction module (312), the motion score calculation module (313) of the first application (310), the shooting condition determination module (321) of the second application (320), the camera control module (322), the motion determination module (331) of the third application (330), the camera operation control module (332), and the image generation module (333) illustrated in FIG. 3 may be implemented as a software module including at least one command. At least one processor (240) may execute commands (e.g., instructions) stored in the memory (250) to implement the software modules illustrated in FIG. 3, and control hardware related to the function (e.g., one or more cameras (210), a display (220), one or more sensors (230), or the memory (250) of FIG. 2). According to various embodiments, the electronic device (200) is not limited to the components illustrated in FIG. 3, and may additionally include components corresponding to the functions required in the electronic device (200) among the components illustrated in FIG. 1.

[0060] Referring to FIG. 3, the electronic device (200) may include a hardware layer (301), a software framework layer (302), or a software application layer (303).

[0061] In one embodiment, the hardware layer (301) may include one or more cameras (210), a display (220), one or more sensors (230), and at least one processor (240) or memory (250). The components included in the hardware layer (301) may each correspond to the components illustrated in FIG. 2.

[0062] In one embodiment, the software framework layer (302) may be understood as a hardware abstraction layer (HAL) or a camera software framework to enhance portability to various hardware, and may include a first application (310) that performs acquisition, analysis, or judgment of data related to the movement of the preview image, and a second application (320) that performs image capture.

[0063] In one embodiment, the first application (310) detects an object from a preview image and performs motion analysis of the object within the preview image, and may include an object detection module (311), an optical flow extraction module (312), or a motion score calculation module (313).

[0064] In one embodiment, the object detection module (311) may generate and update a background model for the preview image, and extract a foreground region within the preview image based on the background model. For example, the background model may be generated and / or updated based on a single Gaussian model or a Gaussian mixture model. The object detection module (311) may detect one or more objects based on the foreground region extracted from the preview image.

[0065] In one embodiment, the optical flow extraction module (312) can extract optical flow for the entire area of ​​the preview image. The optical flow extraction module (312) can estimate the movement of pixels in the preview image based on the extracted optical flow and calculate motion vectors.

[0066] In one embodiment, the motion score calculation module (313) can calculate the motion of one or more objects within the preview image by combining one or more objects (or foreground areas) detected from the preview image and the optical flow extracted from the preview image. The motion score calculation module (313) can quantify the motion of one or more objects included in the preview image as a motion score, thereby quantitatively analyzing in which area within the preview image and at what level the motion occurred.

[0067] In one embodiment, the second application (320) performs image capturing by setting the shooting parameters of the camera based on the movement information analyzed while the preview image is provided, and may include a shooting condition determination module (321) or a camera control module (322).

[0068] In one embodiment, the shooting condition determination module (321) may determine shooting parameters based on the type of movement that occurred while the preview image was being provided, the analyzed motion score, and / or the lighting environment. The shooting parameters may include at least one of ISO sensitivity and shutter speed, which are values ​​set in the camera (210) to obtain a clear image.

[0069] In one embodiment, when the camera control module (322) receives a user input requesting image capture (e.g., shutter button input), the camera control module (322) can set the determined capture parameters to the camera (210) and capture an image based on the set capture parameters.

[0070] In one embodiment, the software application layer (303) controls the overall operational flow and may include a third application (330).

[0071] In one embodiment, the third application (330) may be understood as a camera application and may include a motion determination module (331), a camera motion control module (332), or an image generation module (333).

[0072] In one embodiment, the motion determination module (331) may determine whether movement of the electronic device (200) has occurred while the preview image is provided. The motion determination module (331) may determine whether movement has occurred in the electronic device (200) based on data obtained from one or more sensors (230). The one or more sensors (230) may include various types of sensors capable of detecting movement of the electronic device (200), including a gyro sensor and / or an acceleration sensor. For example, if it is determined that movement has occurred in the electronic device (200) while the preview image is provided, the motion determination module (331) may transmit information about the movement of the electronic device (300) to the first application (310) for generation and / or update of a background model. According to various embodiments, the operation of the motion determination module (331) may also be performed by the first application (310).

[0073] In one embodiment, the camera operation control module (332) can control overall operations of the hardware and software of the electronic device (200) while the third application (330) is running.

[0074] In one embodiment, the image generation module (333) may obtain an image captured from the second application (320) and store it in the memory (250).

[0075] FIG. 4 is a diagram illustrating a method of capturing an image including a dynamic scene in an electronic device (200), according to one embodiment.

[0076] Referring to FIG. 4, when a third application (330) (e.g., a camera application) is executed, the electronic device (200) may acquire preview images (400) through one or more cameras (e.g., the camera (210) of FIG. 2) and provide the acquired preview images (400) through a display (e.g., the display (220) of FIG. 2). According to various embodiments, the preview images (400) may be acquired frame by frame at a specified time interval.

[0077] In one embodiment, the electronic device (200) may analyze motion related to the preview image using the first application (310). For example, the electronic device (200) may detect (411) a motion mask of the preview image using the first application (310). While the preview image is provided, the electronic device (310) may distinguish a background area and a foreground area based on a background model for the preview image, and detect a motion mask based on the foreground area. As another example, the electronic device (200) may extract (413) an optical flow for the entire area of ​​the preview image, and estimate a pixel motion of the preview image based on the extracted optical flow. According to various embodiments, motion mask detection (411) and optical flow extraction (413) for the preview image may be performed in parallel through the first application (310).

[0078] In one embodiment, the electronic device (200) may obtain (415) a final motion mask for the motion of each foreground area within the preview image by combining the detected motion mask and the extracted optical flow using the first application (310). The electronic device (200) may calculate (417) motion information for the preview image based on the obtained final motion mask. According to various embodiments, if the foreground area is not detected in the preview image, the electronic device (200) may calculate (417) motion information for the preview image based on the extracted optical flow for the entire area of ​​the preview image. According to various embodiments, if movement of the electronic device (200) is detected while the preview image is provided, the electronic device (200) may calculate (417) movement information about the preview image based on data about the movement of the electronic device (200) detected from one or more sensors (e.g., sensor (230) of FIG. 2) without considering the acquired final motion mask and the extracted optical flow. Whether the electronic device (200) is moving may be determined by a third application (330) and transmitted to the first application, or the first application may directly determine the presence of movement based on sensing data acquired from one or more sensors (230) (e.g., a gyro sensor or an acceleration sensor).

[0079] In one embodiment, when the electronic device (200) receives a shooting request from a user while providing the preview image, the electronic device (200) may set the shooting parameters of the camera using the second application (320). For example, the electronic device (200) may determine (421) shooting parameters for obtaining a clear image of high quality based on the motion information regarding the calculated preview image using the second application (320). The shooting parameters may include at least one of ISO sensitivity and shutter speed. The electronic device (200) may set the determined shooting parameters to one or more cameras (210) and perform image capturing (423). According to various embodiments, the electronic device (200) may perform (425) a post-processing operation on the captured image using the second application (320). For example, when obtaining an image using a multi-frame synthesis technique, the electronic device (200) may capture and synthesize a plurality of preview images obtained in frame units to create a single image. For another example, the electronic device (200) may correct brightness, color, or contrast to improve the quality of the captured image.

[0080] In one embodiment, the electronic device (200) may use the third application (330) to obtain an image that has been photographed and post-processed by the second application (320) and store (431) it in a memory (e.g., the memory (250) of FIG. 2).

[0081] FIG. 5 is a flowchart illustrating an operating method of an electronic device according to an embodiment. According to an embodiment, the electronic device (200) is a device that determines motion information from a preview image acquired through a camera when taking an image using an automatic shooting mode, and sets shooting parameters in consideration of the determined motion information, and may correspond to the electronic device (101) illustrated in FIG. 1. The operations of FIG. 5 may be performed by at least one processor included in the electronic device (200) (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2).

[0082] Referring to FIG. 5, in operation 510, the electronic device (200) may analyze movement of the electronic device (200) or an object within the preview image while displaying a preview image acquired from one or more cameras (e.g., the camera (210) of FIG. 2) on a display (e.g., the display (220) of FIG. 2) after a camera application is executed. In operation 510, the electronic device (200) may determine whether movement of the electronic device (200) or movement of an object within the preview image has occurred while displaying the preview image. For example, the electronic device (200) may determine whether movement has occurred in the electronic device (200) based on data regarding physical movement of the electronic device (200) acquired from one or more sensors (e.g., the sensor (230) of FIG. 2) while displaying the preview image. One or more sensors (230) may include various types of sensors capable of detecting movement of the electronic device (200), including a gyro sensor and / or an acceleration sensor. When the electronic device (200) determines that movement has occurred in the electronic device (200), the type of movement that occurred while displaying the preview image may be determined as movement of the electronic device (200) (or movement of the camera (210).

[0083] According to one embodiment, in operation 510, if it is determined that no movement has occurred in the electronic device (200), the electronic device (200) may analyze the movement of pixels included in the preview image. For example, the electronic device (200) may extract an optical flow of the entire area within the preview image and estimate the pixel movement of the entire area of ​​the preview image based on the extracted optical flow. The optical flow may be obtained based on a dense optical flow that estimates movement in the entire area of ​​the preview image, and if it is difficult to extract the dense optical flow, the optical flow may be obtained based on the movement of a feature point, such as an edge or corner point, detected within the preview image. According to one embodiment, the electronic device (200) may identify one or more objects from the preview image. The electronic device (200) may distinguish a background area and a foreground area from the preview image based on a background model for the preview image, and may extract the foreground area. The electronic device (200) may identify one or more objects based on the foreground area in the preview image. According to one embodiment, the electronic device (200) may determine the movement of an object in the preview image and / or the pixel movement of the entire area while displaying the preview image based on the estimated pixel movement based on the optical flow for the entire area of ​​the preview image and / or the object identified in the preview image.

[0084] According to various embodiments, when the electronic device (200) receives a first frame of a preview image from one or more cameras (210) after a camera application is executed, the electronic device (200) may generate a background model based on a probability distribution of pixels included in the preview image. For example, the electronic device (200) may generate the background model based on color or brightness values ​​of the preview image, such as a single Gaussian model or a Gaussian mixture model. The electronic device (200) may update the background model each time it receives a subsequent frame after receiving the first frame of the preview image.

[0085] According to one embodiment, in operation 520, the electronic device (200) may calculate motion information related to the preview image based on an analysis of motion detected while displaying the preview image. According to various embodiments, when it is determined as a result of the analysis that motion of the electronic device (200) has occurred, the electronic device (200) may determine the amount of motion of the electronic device (200) based on data acquired from one or more sensors (230), and calculate motion information based on the determined amount of motion of the electronic device (200). The motion information may numerically indicate how much motion occurred while the preview image was displayed.

[0086] According to various embodiments, in operation 520, if it is determined that no movement of the electronic device (200) has occurred as a result of the analysis, the electronic device (200) may calculate the amount of movement for each object by combining the estimated pixel movement for the entire area of ​​the preview image with one or more objects identified in the preview image, and determine the motion information based on the calculated amount of movement for each object. According to various embodiments, if there are multiple foreground areas extracted from the preview image, the electronic device (200) may assign a weight according to the area of ​​each of the multiple foreground areas, and may calculate the motion information by combining the multiple foreground areas (or objects included in each foreground area) and the estimated pixel movement based on the weight. According to various embodiments, if there is no foreground area (or object) extracted from the preview image, the electronic device (200) may calculate motion information indicating the degree of motion that occurred while the preview image was displayed based on the estimated pixel movement.

[0087] According to one embodiment, in operation 530, when the electronic device (200) receives a user input for capturing an image while displaying the preview image, the electronic device (200) may set at least one capturing parameter for obtaining a clear image of high quality based on the calculated motion information. Here, the user input may correspond to a shutter button input received through a camera application. The at least one parameter may include at least one of an ISO sensitivity or a shutter speed of one or more cameras (210). For example, when the electronic device (200) detects movement of the electronic device (200) or the preview image while the preview image is provided in a low-light environment where the brightness value is lower than a specified value, the electronic device (200) may set the electronic device (200) to have a higher ISO sensitivity value than when no motion is detected. As another example, when the electronic device (200) detects movement of the electronic device (200) or the preview image while the preview image is provided in a low-light environment, the electronic device (200) may set the electronic device (200) to have a faster shutter speed than when no motion is detected.

[0088] According to one embodiment, in operation 540, the electronic device (200) can perform image capturing based on the set shooting parameters and obtain the captured image.

[0089] FIG. 6 is a diagram illustrating a method for analyzing movement while displaying a preview screen on an electronic device, according to one embodiment. In FIG. 6, the operations of the first application (310) and / or the third application may be understood as functions performed by at least one processor included in the electronic device (200) (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2).

[0090] Referring to FIG. 6, when a third application (330) (e.g., a camera application) is executed, the first application (310) and / or the third application (330) may acquire data detected from one or more sensors (e.g., sensor (230) of FIG. 2) (operations 611 and 621). The data acquired from the one or more sensors (230) may include data regarding physical movement of the electronic device (200) and / or the current lighting environment.

[0091] In one embodiment, the third application (330) may analyze the movement of the electronic device (200) based on the data (operation 622) and determine whether camera movement has occurred based on the analysis (operation 623). For example, if there is movement of the electronic device (200), the third application (330) may determine that camera movement has occurred. If the determination result indicates that camera movement has occurred (operation 623—Yes), the third application may obtain movement information based on the camera movement (operation 625). If the determination result indicates that camera movement has not occurred (operation 623—No), the third application may request the first application to provide the analyzed movement information for the preview image (operation 624).

[0092] In one embodiment, the first application (310) can obtain a preview image from one or more cameras (e.g., the camera (210) of FIG. 2) (operation 612). The first application (310) can generate a background model based on a probability distribution of pixels included in the preview image if the preview image is a first frame received after the third application (330) is executed. The first application (310) can update the background model based on subsequent frames received after the first frame. The first application (310) can extract a foreground area from the preview image based on the background model (operation 613). The first application (310) can measure an optical flow for the entire area of ​​the preview image and extract a movement of pixels within the preview image based on the measured optical flow (operation 614). The first application (310) may calculate motion information for the foreground region by combining the foreground region of the preview image and the motion of pixels within the preview image (operation 615). According to various embodiments, when there are multiple foreground regions extracted from the preview image, the first application (310) may assign a weight according to the area of ​​each of the multiple foreground regions, and calculate motion information by combining the multiple foreground regions and the estimated pixel motion based on the weight. According to various embodiments, when there is no foreground region extracted from the preview image, the first application (310) may calculate motion information based on pixel motion extracted based on optical flow for the entire area of ​​the preview image.

[0093] In one embodiment, the third application (330) may obtain analyzed motion information for the preview image from the first application (310) (operation 625). The third application (330) may determine whether the obtained motion information is a valid (or meaningful) motion with respect to the preview image (operation 626). If the determination result indicates that the obtained motion information is an invalid motion (operation 626—No), the motion information may be ignored (operation 628). If the determination result indicates that the obtained motion information is a valid motion (operation 626—Yes), the third application may store the information in a memory (e.g., the memory (250) of FIG. 2).

[0094] According to various embodiments, the operations described in FIG. 6 may be performed repeatedly each time the third application (330) is executed to acquire a preview image frame by frame from one or more cameras (210).

[0095] FIG. 7 is a diagram illustrating a method of setting shooting parameters and acquiring images in an electronic device, according to an embodiment. In FIG. 7, the operations of the second application (320) and / or the third application may be understood as functions performed by at least one processor included in the electronic device (200) (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2).

[0096] Referring to FIG. 7, a third application (330) (e.g., a camera application) may receive an image capture request from a user (operation 721). The image capture request may correspond to a capture button (or shutter button) input through the execution screen of the third application (330).

[0097] In one embodiment, in response to receiving the image capture request, the second application (320) may determine capture parameters based on movement information analyzed while a preview image is provided from the third application (330) (operation 711). The capture parameters may include at least one of an ISO sensitivity or a shutter speed of one or more cameras (e.g., the camera (210) of FIG. 2). The second application (320) may set one or more cameras based on the determined capture parameters (operation 712) and acquire a capture image (operation 713). The second application may perform a post-processing operation on the capture image (operation 714) and transmit the post-processed image to the third application (330). The post-processing operation may include an operation for improving the image quality of the capture image. For example, the second application (320) may capture and synthesize multiple preview images acquired in frame units to create a single image, or may perform a task of correcting the brightness, color, or contrast of the captured image.

[0098] In one embodiment, the third application (330) may receive the image for which the post-processing operation has been completed from the second application (320) (operation 724) and store it in a memory (e.g., memory (250) of FIG. 2) (operation 725).

[0099] FIG. 8 is a flowchart illustrating operations performed by a third application (330), according to one embodiment. The operations of the third application described in FIG. 8 may be understood as functions performed by at least one processor included in an electronic device (200) (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2).

[0100] Referring to FIG. 8, in operation 810, a third application (330) may obtain data regarding physical movement of the electronic device (200) from one or more sensors (e.g., the sensor (230) of FIG. 2) while the third application (330) is running and a preview image is provided. For example, the one or more sensors (230) may include various types of sensors capable of detecting movement of the electronic device (200), including a gyro sensor and / or an acceleration sensor.

[0101] According to one embodiment, in operation 820, the third application (330) may determine the type of motion detected while the preview image is being provided based on the acquired data. For example, if the third application (330) determines that motion has occurred in the electronic device (200) based on the acquired data, the third application (330) may determine the type of motion that has occurred while the preview image is being displayed as motion of the electronic device (200) (or motion of the camera (210). As another example, if the third application (330) determines that motion has not occurred in the electronic device (200) based on the acquired data, the third application (330) may request the first application (310) to analyze the motion of pixels included in the preview image.

[0102] According to one embodiment, in operation 830, the third application (330) can obtain motion information regarding the preview image analyzed by the first application (310).

[0103] According to one embodiment, in operation 840, the third application (330) can determine whether a user's request to take a picture has been received. The request to take a picture may correspond to a picture button (or shutter button) input through the execution screen of the third application (330).

[0104] If the user's shooting request is received as a result of the determination in operation 840 (operation 840-Yes), the third application (330) can obtain the captured image from the second application (320) in operation 850. The image obtained in operation 850 may be an image that has undergone post-processing to improve image quality after being captured by the second application (320) in response to the shooting request. The third application (330) can store the obtained image in the memory of the electronic device (200) (e.g., the memory (130) of FIG. 1 or the memory (250) of FIG. 2) in operation 860.

[0105] If the user's shooting request is not received as a result of the judgment of action 840 (action 840-No), the third application may wait for the user's input without performing any other action.

[0106] According to one embodiment, in operation 870, the third application (330) can check whether an application termination request has been received from the user. If the confirmation result of operation 870 indicates that an application termination request has been received (operation 870-Yes), the execution of the third application (330) can be terminated. If the confirmation result of operation 870 indicates that an application termination request has not been received (operation 870-No), the third application (330) can repeatedly display and analyze motion of preview images acquired from one or more cameras (e.g., the camera module (180) of FIG. 1 or the camera (210) of FIG. 2) while the execution of the third application (330) is maintained.

[0107] FIG. 9 is a flowchart illustrating operations performed by a first application according to an embodiment. The operations of the first application described in FIG. 9 may be understood as functions performed by at least one processor (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2) included in an electronic device (200).

[0108] Referring to FIG. 9, in operation 910, the first application (310) may obtain sensor data and a preview image in response to the execution of a third application (e.g., a camera application). For example, the first application (310) may obtain data regarding physical movement of the electronic device (200) from one or more sensors (e.g., the sensor (230) of FIG. 2). For example, the one or more sensors (230) may include various types of sensors capable of detecting movement of the electronic device (200), including a gyro sensor and / or an acceleration sensor. As another example, the first application (310) may obtain the preview image from one or more cameras (e.g., the camera module (180) of FIG. 1 or the camera (210) of FIG. 2).

[0109] According to one embodiment, in operation 912, the first application (310) may perform preprocessing on the acquired preview image. For example, the first application (310) may adjust the preview image acquired from one or more cameras (210) to a size that can be displayed on a display (e.g., the display module (160) of FIG. 1 or the display (220) of FIG. 2), and / or remove noise included in the preview image.

[0110] According to one embodiment, in operation 915, the first application (310) can determine whether the acquired preview image is the first frame received after the execution of the third application (330). If the determination result of operation 915 indicates that the preview image is the first frame received after the execution of the third application (330) (operation 915-Yes), the first application (310) can generate a background model based on a probability distribution of pixels included in the preview image in operation 920. The background model can be generated based on a single Gaussian model or a Gaussian mixture model. If the determination result of operation 915 indicates that the preview image is not the first frame received after the execution of the third application (operation 915-No), the first application (310) can update the background model based on the preview image in operation 925.

[0111] According to one embodiment, in operation 930, the first application (310) can extract optical flow for the entire area of ​​the preview image. The first application (310) can estimate the movement of pixels within the preview image based on the extracted optical flow.

[0112] According to one embodiment, in operation 935, the first application (310) can determine the type of movement detected while the preview image is provided. For example, the first application (310) can determine the type of movement based on data regarding the movement of the electronic device (200) received from the third application (330). As another example, the first application (310) can determine whether the electronic device (200) is moving based on sensor data acquired from one or more sensors (230), and determine the type of movement based on the determination.

[0113] According to one embodiment, in operation 940, the first application (310) can determine whether camera movement has occurred based on the identified movement type. If movement of the electronic device (200) is identified, the first application (310) can determine that camera movement has occurred. If the determination result of operation 940 indicates that camera movement has occurred (operation 940-Yes), the first application (310) can calculate a global motion score for the preview image based on data regarding the movement of the electronic device (200) in operation 965.

[0114] If the judgment result of operation 940 is that no camera movement has occurred (operation 940-No), the first application (310) can distinguish the background area and the foreground area in the preview image based on the background model in operation 950 and extract the foreground area.

[0115] According to one embodiment, in operation 955, the first application (310) can determine whether there is a foreground region extracted from the preview image. If there is no foreground region extracted from the preview image as a result of the determination in operation 955 (operation 955-No), the first application (310) can calculate a global motion score for the preview image based on the movement of pixels in the preview image estimated based on the optical flow of the preview image as a result of the determination in operation 965. If there is one or more foreground regions extracted from the preview image as a result of the determination in operation 955 (operation 955-Yes), the first application (310) can calculate a motion score for each foreground region in the preview image by combining the one or more foreground regions and the movement of the estimated pixels as a result of the determination in operation 960.

[0116] According to one embodiment, in operation 970, the first application (310) may calculate a final score based on the calculated motion score for each foreground region and / or the global motion score for the preview image and store the calculation result in a memory (e.g., the memory (130) of FIG. 1 or the memory (250) of FIG. 2). The final score may be utilized in a process of determining shooting parameters by considering the motion of the preview image after a user's shooting request is input.

[0117] According to one embodiment, in operation 980, the first application (310) can check whether a request to terminate the third application (330) has been received from the user. If the confirmation result of operation 980 indicates that an application termination request has been received (operation 980-Yes), the first application (310) can stop determining the movement of the preview image. If the confirmation result of operation 980 indicates that an application termination request has not been received (operation 980-No), the first application (310) can repeatedly perform acquisition of the preview image and determination of the movement while the execution of the third application (330) is maintained.

[0118] FIG. 10 is a flowchart illustrating operations performed by a second application according to an embodiment. The operations of the second application described in FIG. 10 may be understood as functions performed by at least one processor (e.g., the processor (120) of FIG. 1 or at least one processor (240) of FIG. 2) included in the electronic device (200).

[0119] Referring to FIG. 10, in operation 1010, the second application (320) may receive motion information calculated while a preview image is provided from the third application (330) in response to a user's shooting request received through the third application (330). The motion information may be calculated in the form of a motion score that numerically represents the global motion for the preview image and / or the motion for each foreground area of ​​the preview image.

[0120] According to one embodiment, in operation 1020, the second application (320) may set at least one shooting parameter based on motion information received from the third application (330). The shooting parameter may include at least one of an ISO sensitivity or a shutter speed. For example, if the second application (320) detects motion of the electronic device (200) or the preview image while a preview image is provided in a low-light environment with a brightness value below a specified value, the second application (320) may set the electronic device (200) to have a higher ISO sensitivity value than when no motion is detected. As another example, if the second application (320) detects motion of the electronic device (200) or the preview image while a preview image is provided in a low-light environment, the second application (320) may set the electronic device (200) to have a faster shutter speed than when no motion is detected. According to various embodiments, the second application (320) may also apply a weight for setting the shooting parameter based on a motion score calculated while the preview image is provided.

[0121] According to one embodiment, in operation 1030, the second application (320) can perform image capturing based on the set shooting parameters and obtain the captured image.

[0122] According to one embodiment, in operation 1040, the second application (320) may perform image post-processing on the captured image. For example, when acquiring an image using multi-frame synthesis technology, the second application (320) may capture and synthesize multiple preview images acquired frame by frame to create a single image. As another example, the second application (320) may correct brightness, color, or contrast to improve the image quality of the captured image.

[0123] According to one embodiment, in operation 1050, the second application (320) may transfer the final image buffer, on which post-processing has been completed, to the third application (330) and terminate the process related to image capture.

[0124] An electronic device (e.g., electronic device (200)) according to one embodiment includes one or more cameras (e.g., camera (210)), a display (e.g., display (220)), at least one processor (e.g., processor (240)) operatively connected to the one or more cameras and the display, and a memory (e.g., memory (250)) operatively connected to the at least one processor, wherein the memory may store instructions that, when executed, cause the at least one processor to analyze a movement of the electronic device or an object in the preview image while displaying a preview image acquired from the one or more cameras on the display, calculate motion information related to the preview image based on the analysis, and, in response to receiving a user input for capturing an image, set at least one capturing parameter based on the calculated motion information, and obtain a captured image based on the set capturing parameter.

[0125] In one embodiment, the electronic device further comprises one or more sensors (e.g., sensor (230)) for detecting movement, and the instructions may cause the at least one processor to determine whether movement of the electronic device has occurred based on data obtained from the one or more sensors while the preview image is displayed on the display, and if it is determined that movement of the electronic device has occurred, determine movement information related to the preview image based on data obtained from the one or more sensors.

[0126] In one embodiment, the instructions may cause the at least one processor to estimate pixel motion for an entire area of ​​the preview image, identify one or more objects from the preview image, and determine motion information associated with the preview image based on the estimated pixel motion and the identified one or more objects, if it is determined that no motion of the electronic device has occurred.

[0127] In one embodiment, the instructions may cause the at least one processor to extract optical flow of an entire area within the preview image and estimate pixel motion based on the extracted optical flow.

[0128] In one embodiment, the instructions may cause the at least one processor to determine motion information associated with the preview image based on the estimated pixel motion if no object is identified from the preview image.

[0129] In one embodiment, the instructions may cause the at least one processor to distinguish a background area and a foreground area in the preview image, and to identify the one or more objects based on the foreground area in the preview image.

[0130] In one embodiment, the instructions may cause the at least one processor to determine a weight based on an area of ​​one or more foreground regions included in the preview image, and to combine an object included in each of the one or more foreground regions with the estimated pixel motion based on the determined weight to determine motion information associated with the preview image.

[0131] In one embodiment, the instructions may cause the at least one processor to, when receiving the preview image, determine whether the preview image is a first frame acquired from the one or more cameras after a camera application is executed, and, if the preview image is determined to be the first frame, generate a background model based on a probability distribution of pixels included in the preview image, and extract a foreground area of ​​the preview image based on the generated background model.

[0132] In one embodiment, the instructions may cause the at least one processor to update the background model based on the preview image if the preview image is determined to not be a first frame.

[0133] In one embodiment, the shooting parameters may include at least one of an ISO sensitivity (International Organization for Standardization film speed) or a shutter speed of the one or more cameras.

[0134] A method of operating an electronic device (e.g., electronic device (200)) according to one embodiment may include an operation of analyzing a movement of the electronic device or an object in the preview image while displaying a preview image acquired from one or more cameras, an operation of calculating movement information related to the preview image based on the analysis, an operation of setting at least one shooting parameter based on the calculated movement information in response to receiving a user input for capturing an image, and an operation of obtaining a captured image based on the set shooting parameter.

[0135] In one embodiment, the act of analyzing the movement of the electronic device or the object within the preview image may include the act of determining whether movement of the electronic device has occurred based on data obtained from one or more sensors while the preview image is displayed on the display, and, if it is determined that movement of the electronic device has occurred, the act of determining movement information related to the preview image based on data obtained from the one or more sensors.

[0136] In one embodiment, the operation of analyzing the movement of the electronic device or an object within the preview image may include, if it is determined that no movement of the electronic device has occurred, estimating pixel movement for an entire area of ​​the preview image, identifying one or more objects from the preview image, and determining motion information related to the preview image based on the estimated pixel movement and the identified one or more objects.

[0137] In one embodiment, the operation of estimating pixel motion for the entire area of ​​the preview image may include the operation of extracting optical flow for the entire area within the preview image, and the operation of estimating the pixel motion based on the extracted optical flow.

[0138] In one embodiment, the method may further include determining motion information associated with the preview image based on the estimated pixel motion if no object is identified from the preview image.

[0139] In one embodiment, the operation of identifying one or more objects from the preview image may include the operation of distinguishing a background area and a foreground area in the preview image, and the operation of identifying the one or more objects based on the foreground area in the preview image.

[0140] In one embodiment, the operation of determining motion information related to the preview image may include an operation of determining a weight according to an area of ​​one or more foreground regions included in the preview image, and an operation of determining motion information related to the preview image by combining an object included in each of the one or more foreground regions with the estimated pixel motion based on the determined weight.

[0141] In one embodiment, the method may include, when the preview image is received, an operation of determining whether the preview image is a first frame acquired from the one or more cameras after the camera application is executed, an operation of generating a background model based on a probability distribution of pixels included in the preview image if the preview image is determined to be the first frame, and an operation of extracting a foreground area of ​​the preview image based on the generated background model.

[0142] In one embodiment, the method may further include an operation of updating the background model based on the preview image if it is determined that the preview image is not the first frame.

[0143] In one embodiment, the shooting parameters may include at least one of an ISO sensitivity (International Organization for Standardization film speed) or a shutter speed of the one or more cameras.

[0144] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0145] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0146] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0147] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0148] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smartphones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0149] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (200), One or more cameras (210); display (220); at least one processor (240); and Contains a memory (250) for storing instructions, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: While displaying a preview image acquired from one or more cameras (210) on the display (220), analyzing the movement of the electronic device (220) or an object within the preview image, Based on the above analysis, motion information related to the preview image is calculated, In response to receiving a user input for capturing an image, setting at least one capturing parameter based on the computed motion information; An electronic device that acquires a photographed image based on the above-set photographing parameters.

2. In claim 1, Further comprising one or more sensors (230) for detecting movement of the electronic device; The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: While the above preview image is displayed on the display (220), it is determined whether movement of the electronic device occurs based on data obtained from the one or more sensors (230), An electronic device that determines movement information related to the preview image based on data acquired from one or more sensors (230) when it is determined that movement of the electronic device has occurred.

3. In claim 2, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: If it is determined that no movement of the above electronic device has occurred, the pixel movement for the entire area of ​​the above preview image is estimated, Identifying one or more objects from the above preview image, An electronic device that determines motion information associated with the preview image based on the estimated pixel motion and the identified one or more objects.

4. In claim 3, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: Extract the optical flow of the entire area within the above preview image, An electronic device for estimating the pixel movement based on the extracted optical flow.

5. In claim 3, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: An electronic device that determines motion information related to the preview image based on the estimated pixel motion if no object is identified from the preview image.

6. In claim 3, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: In the above preview image, the background area and the foreground area are distinguished, An electronic device that identifies one or more objects based on the foreground area within the preview image.

7. In claim 6, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: Determine weights based on the area of ​​one or more foreground regions included in the above preview image, An electronic device that determines motion information related to the preview image by combining the estimated pixel motion with an object included in each of the one or more foreground areas based on the determined weights.

8. In claim 1, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: Upon receiving the above preview image, the camera application is run and checks whether it is the first frame acquired from one or more cameras, When the above preview image is confirmed as the first frame, a background model is generated based on the probability distribution of pixels included in the above preview image, An electronic device for extracting a foreground area of ​​the preview image based on the generated background model.

9. In claim 8, The above instructions, when executed by the at least one processor (240), cause the electronic device (200) to: An electronic device that updates the background model based on the preview image if it is determined that the preview image is not the first frame.

10. In claim 1, The above shooting parameters are, An electronic device comprising at least one of an ISO sensitivity (International Organization for Standardization film speed) or a shutter speed of said one or more cameras (210).

11. In the method, An operation of analyzing movement of the electronic device (200) or an object within the preview image while displaying a preview image acquired from one or more cameras (210); An operation of calculating motion information related to the preview image based on the above analysis; In response to receiving a user input for capturing an image, an operation for setting at least one capturing parameter based on the computed motion information; and A method comprising an operation of acquiring a photographed image based on the above-set photographing parameters.

12. In claim 11, The action of analyzing the movement of the object in the electronic device or the preview image, An operation of determining whether movement of the electronic device has occurred based on data acquired from one or more sensors while the preview image is displayed on the display; and A method comprising: determining motion information related to the preview image based on data acquired from the one or more sensors, when it is determined that movement of the electronic device has occurred.

13. In claim 12, The action of analyzing the movement of the object in the electronic device or the preview image, An operation of estimating pixel movement for the entire area of ​​the preview image, if it is determined that no movement of the electronic device has occurred; An operation of identifying one or more objects from the above preview image; and A method comprising the operation of determining motion information associated with the preview image based on the estimated pixel motion and the identified one or more objects.

14. In claim 13, The operation of estimating pixel motion for the entire area of ​​the above preview image is: An operation of extracting optical flow of the entire area within the above preview image; and A method comprising an operation of estimating the pixel movement based on the extracted optical flow.

15. In claim 11, Upon receiving the above preview image, an action is taken to determine whether it is the first frame acquired from the one or more cameras after the camera application is launched; When the above preview image is confirmed as the first frame, an operation of generating a background model based on a probability distribution of pixels included in the preview image and extracting a foreground area of ​​the preview image based on the background model; and A method further comprising an action of updating the background model based on the preview image if it is determined that the preview image is not the first frame.

Images