Electronic device and control method therefor

Abstract

This electronic device comprises: a plurality of image sensors including a first image sensor and a second image sensor having different aperture values; a memory storing instructions; and at least one processor including processing circuitry, wherein the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: receive a user input for long-exposure photographing; obtain a first photographing frame set on the basis of a first exposure value through the first image sensor for a preset time; obtain a second photographing frame set on the basis of a second exposure value through the second image sensor for a preset time; obtain a first brightness change amount in the first photographing frame set; obtain a second brightness change amount in the second photographing frame set; if at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, identify a plurality of target frames in the first photographing frame set and the second photographing frame set; and merge the plurality of target frames to generate a long-exposure frame.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more specifically, to an electronic device for generating a long exposure frame and a method for controlling the same.

[0002] Electronic devices including cameras can generate long-exposure frames through a long-exposure function. A long-exposure function may record the trajectory of a moving subject or details in a low-light environment through long-duration exposure.

[0003] An electronic device can acquire multiple images over a set period of time and merge the multiple images to generate a single long-exposure frame.

[0004] Unintended light may be detected at a set time. Light from the external environment may be detected while taking long-exposure photographs. For example, unintended light from the external environment may include car headlights or external flashes.

[0005] If unintended light is detected, changes in that light may be reflected during the merging process. If strong light intensity is detected, long-exposure frames may contain pixel data that is brighter overall than what the user expects.

[0006] The present disclosure is designed to improve upon the aforementioned problems, and the purpose of the present disclosure is to provide an electronic device and a control method thereof that compare exposure values ​​of a plurality of image sensors to select a shooting frame and generate a long exposure frame with the selected shooting frame.

[0007] According to one embodiment, the electronic device comprises a plurality of image sensors including a first image sensor and a second image sensor having different aperture values, a memory for storing instructions, and at least one processor including processing circuitry, wherein when the instructions are executed individually or collectively by the at least one processor, the electronic device receives user input for long exposure shooting, acquires a first set of shooting frames based on a first exposure value through the first image sensor for a preset time, acquires a second set of shooting frames based on a second exposure value through the second image sensor for a preset time, acquires a first brightness change amount in the first set of shooting frames, acquires a second brightness change amount in the second set of shooting frames, and if at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, identifies a plurality of target frames in the first set of shooting frames and the second set of shooting frames, and merges the plurality of target frames to generate a long exposure frame.

[0008] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify the first image sensor having a large aperture value among the plurality of image sensors, determine the exposure value of the plurality of image sensors, acquire a plurality of shooting frames through the first image sensor and the second image sensor during the preset time, and identify a plurality of target frames among the plurality of shooting frames based on the exposure value.

[0009] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may determine a first exposure value corresponding to the first image sensor and determine a second exposure value corresponding to the second image sensor, acquire a first shooting frame at a first time point with the first exposure value through the first image sensor, and acquire a second shooting frame at a first time point with the second exposure value through the second image sensor.

[0010] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may determine a third exposure value corresponding to the first image sensor when a first shooting frame is acquired, determine a fourth exposure value corresponding to the second image sensor when a second shooting frame is acquired, acquire a third shooting frame with the third exposure value at a second time point through the first image sensor, acquire a fourth shooting frame with the fourth exposure value at the second time point through the second image sensor, determine the first shooting frame as a first target frame, determine one of the third shooting frame and the fourth shooting frame as the second target frame, and merge the first target frame and the second target frame to generate the long exposure frame.

[0011] When the above instructions are executed individually or collectively by at least one processor, the electronic device may determine the first exposure value as the target exposure value, obtain a first difference value which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value, and if the first difference value is less than the first threshold value, determine the third shooting frame as the second target frame.

[0012] When the above instructions are executed individually or collectively by at least one processor, the electronic device may obtain a second difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value, if the first difference value is greater than or equal to the first threshold value, and determine the fourth shooting frame as the second target frame if the first difference value exceeds the second difference value.

[0013] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify a fixed object based on the plurality of captured frames and generate the long exposure frame by merging the first target frame and the second target frame based on the fixed object.

[0014] The electronic device includes an inertial sensor, and when the instructions are executed individually or collectively by the at least one processor, the electronic device acquires a first inertial vector at a first time point through the inertial sensor, acquires a second inertial vector at a second time point through the inertial sensor, acquires a first difference vector between the first inertial vector and the second inertial vector, and if the first difference vector is less than a threshold size, merges the first target frame and the second target frame to generate the long exposure frame, and the first difference vector may be a vector representing the movement of the electronic device.

[0015] When the above instructions are executed individually or collectively by the above at least one processor, the electronic device identifies a target object included in the first target frame and the second target frame when the first difference vector is greater than or equal to a threshold size, identifies a first position of the target object in the first target frame at the first time point, identifies a second position of the target object in the second target frame at the second time point, obtains a first motion vector of the target object based on the first position and the second position, and generates the long exposure frame based on the correlation value of the first difference vector and the first motion vector, wherein the first motion vector represents the movement of the target object.

[0016] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may determine the target object as a fixed object if the correlation value is below a second threshold value, determine the target object as a non-fixed object if the correlation value exceeds the second threshold value, and generate the long exposure frame by merging the first target frame and the second target frame based on the position of the fixed object.

[0017] According to one embodiment, a control method for an electronic device comprising a plurality of image sensors including a first image sensor and a second image sensor having different aperture values ​​comprises: receiving user input for long exposure shooting; acquiring a first set of shooting frames based on a first exposure value through the first image sensor for a preset time; acquiring a second set of shooting frames based on a second exposure value through the second image sensor for the preset time; acquiring a first brightness change amount in the first set of shooting frames; acquiring a second brightness change amount in the second set of shooting frames; if at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, identifying a plurality of target frames in the first set of shooting frames and the second set of shooting frames; and merging the plurality of target frames to generate a long exposure frame.

[0018] The above control method includes the step of identifying the first image sensor having a large aperture value among the plurality of image sensors and the step of determining the exposure value of the plurality of image sensors, and the step of acquiring the first shooting frame set and the second shooting frame set acquires a plurality of shooting frames through the first image sensor and the second image sensor during the preset time, and the step of identifying the plurality of target frames can identify a plurality of target frames among the plurality of shooting frames based on the exposure value.

[0019] The step of acquiring the first shooting frame set and the second shooting frame set may involve determining a first exposure value corresponding to the first image sensor and determining a second exposure value corresponding to the second image sensor, acquiring a first shooting frame at a first time point with the first exposure value through the first image sensor, and acquiring a second shooting frame at a first time point with the second exposure value through the second image sensor.

[0020] The step of acquiring the first shooting frame set and the second shooting frame set comprises determining a third exposure value corresponding to the first image sensor when the first shooting frame is acquired, determining a fourth exposure value corresponding to the second image sensor when the second shooting frame is acquired, acquiring a third shooting frame with the third exposure value at a second time point through the first image sensor, acquiring a fourth shooting frame with the fourth exposure value at a second time point through the second image sensor, and the step of identifying the plurality of target frames comprises determining the first shooting frame as the first target frame and determining one of the third shooting frame and the fourth shooting frame as the second target frame, and the step of generating the long exposure frame may generate the long exposure frame by merging the first target frame and the second target frame.

[0021] The step of identifying the plurality of target frames can determine the first exposure value as the target exposure value, obtain a first difference value which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value, and if the first difference value is less than the first threshold value, determine the third shooting frame as the second target frame.

[0022] The step of identifying the plurality of target frames can be to obtain a second difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value, if the first difference value is greater than or equal to the first threshold value, and if the first difference value exceeds the second difference value, the fourth shooting frame can be determined as the second target frame.

[0023] The above control method includes a step of identifying a fixed object based on the plurality of shooting frames, and the step of generating the long exposure frame may generate the long exposure frame by merging the first target frame and the second target frame based on the fixed object.

[0024] The electronic device includes an inertial sensor, and the control method includes the steps of obtaining a first inertial vector at a first time point through the inertial sensor, obtaining a second inertial vector at a second time point through the inertial sensor, and obtaining a first difference vector of the first inertial vector and the second inertial vector, and the step of generating the long exposure frame includes merging the first target frame and the second target frame to generate the long exposure frame if the first difference vector is less than a threshold size, and the first difference vector may be a vector representing the movement of the electronic device.

[0025] A control method wherein the step of generating the long exposure frame comprises, if the first difference vector is greater than or equal to a threshold size, identifying a target object included in the first target frame and the second target frame, identifying a first position of the target object in the first target frame at the first time point, identifying a second position of the target object in the second target frame at the second time point, obtaining a first motion vector of the target object based on the first position and the second position, and generating the long exposure frame based on the correlation value of the first difference vector and the first motion vector, wherein the first motion vector represents the movement of the target object.

[0026] The step of generating the long exposure frame above may generate the long exposure frame by determining the target object as a fixed object if the correlation value is less than or equal to a second threshold value, determining the target object as a non-fixed object if the correlation value exceeds the second threshold value, and merging the first target frame and the second target frame based on the position of the fixed object.

[0027] FIG. 1 is a drawing for explaining a long exposure frame according to one embodiment.

[0028] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.

[0029] FIG. 3 is a block diagram illustrating the specific configuration of the electronic device of FIG. 2 according to one embodiment.

[0030] FIG. 4 is a drawing for explaining a camera module according to one embodiment.

[0031] FIG. 5 is a drawing for explaining the operation of generating a long exposure frame according to one embodiment.

[0032] FIG. 6 is a diagram illustrating the operation of generating a long exposure frame based on light change and motion analysis according to one embodiment.

[0033] FIG. 7 is a diagram illustrating the operation of acquiring a plurality of shooting data according to one embodiment.

[0034] FIG. 8 is a diagram illustrating the operation of acquiring an exposure value according to one embodiment.

[0035] FIG. 9 is a drawing for explaining the operation of acquiring an exposure value according to one embodiment.

[0036] FIG. 10 is a drawing for explaining the operation of acquiring a target frame set according to one embodiment.

[0037] FIG. 11 is a diagram illustrating the operation of acquiring a target frame set based on an exposure value according to one embodiment.

[0038] FIG. 12 is a drawing for explaining motion analysis operations according to one embodiment.

[0039] FIG. 13 is a drawing for explaining motion analysis operations according to one embodiment.

[0040] FIG. 14 is a drawing for explaining exposure values ​​according to one embodiment.

[0041] FIG. 15 is a drawing for explaining exposure values ​​according to one embodiment.

[0042] FIG. 16 is a drawing for explaining a shooting frame acquired from a plurality of image sensors according to one embodiment.

[0043] FIG. 17 is a drawing for explaining the operation of determining a target frame according to one embodiment.

[0044] FIG. 18 is a diagram illustrating the operation of generating a merged image according to one embodiment.

[0045] FIG. 19 is a diagram illustrating the operation of generating a merged image according to one embodiment.

[0046] FIG. 20 is a drawing for explaining motion analysis according to one embodiment.

[0047] FIG. 21 is a drawing for explaining the operation of generating a long exposure frame through motion analysis according to one embodiment.

[0048] FIG. 22 is a diagram illustrating the operation of analyzing correlations according to one embodiment.

[0049] FIG. 23 is a drawing for explaining a screen showing a long exposure function according to one embodiment.

[0050] FIG. 24 is a drawing for explaining the operation of generating a long exposure frame according to one embodiment.

[0051] FIG. 25 is a drawing for explaining a method of controlling an electronic device according to one embodiment.

[0052] The present disclosure will be described in detail below with reference to the attached drawings.

[0053] The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0054] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, operations, or components such as parts) and do not exclude the presence of additional features.

[0055] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0056] Expressions such as "first," "second," "first," or "second" used in this specification may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0057] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0058] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0059] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0060] In this specification, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0061] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0062] FIG. 1 is a drawing for explaining a long exposure frame according to one embodiment.

[0063] Referring to FIG. 1, the electronic device (100) may include at least one of a plurality of image sensors (151, 152, 153, 154) and a laser autofocus module (130).

[0064] The aperture value (f-number) of each of the multiple image sensors (151, 152, 153, 154) may be different.

[0065] The electronic device (100) can perform a long exposure function using at least one of a plurality of image sensors (151, 152, 153, 154). The long exposure function may include the operation of generating a long exposure frame (10) by absorbing light for a preset time. The long exposure function may sense the movement of a subject for a preset time or sense the trajectory of light. The preset time may refer to the total shooting time according to the long exposure function.

[0066] A long exposure frame (10) may include a light trajectory over a preset time. An electronic device (100) may generate a long exposure frame (10) by combining a plurality of frames acquired over a preset time.

[0067] Overexposure (11) may occur during a set time, in which strong light is temporarily detected in a specific area. Overexposure of light may refer to a light flare. If there is overexposure of light, there may be a problem in that the brightness of the long exposure frame (10) is not constant.

[0068] FIG. 2 is a block diagram illustrating an electronic device (100) according to one embodiment.

[0069] The electronic device (100) may include at least one of a plurality of image sensors, a memory (120), and at least one processor (110).

[0070] The electronic device (100) may include a plurality of image sensors, including a first image sensor and a second image sensor. The electronic device (100) may include a memory (120) for storing instructions. The electronic device (100) may include at least one processor (110) including processing circuitry.

[0071] At least one processor (110) can receive user input for long exposure shooting. At least one processor (110) can receive user input including a long exposure shooting command.

[0072] At least one processor (110) can acquire a first set of shooting frames based on a first exposure value through a first image sensor for a preset time.

[0073] At least one processor (110) can acquire a second set of shooting frames based on a second exposure value through a second image sensor for a preset time.

[0074] At least one processor (110) can obtain a first brightness change amount in a first set of shooting frames. The first brightness change amount can be determined according to a change in exposure values ​​(e.g., a first exposure value, a third exposure value) that changes over a preset time.

[0075] At least one processor (110) can obtain a second brightness change amount in a second set of shooting frames. The second brightness change amount can be determined according to a change in exposure values ​​(e.g., a second exposure value, a fourth exposure value) that changes over a preset time.

[0076] The pre-set time may refer to the total shooting time based on the long exposure function.

[0077] If at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, at least one processor (110) can identify a plurality of target frames with constant brightness change in the first shooting frame set and the second shooting frame set. At least one processor (110) can merge the plurality of target frames to generate a long exposure frame.

[0078] At least one processor (110) can identify a first image sensor having a large aperture value among a plurality of image sensors. At least one processor (110) can determine the exposure value of the plurality of image sensors. The exposure value may differ for each sensor. The exposure value may include at least one value.

[0079] Each of the plurality of image sensors may have a different aperture value. For example, a first image sensor may have a first aperture value. A second image sensor may have a second aperture value. At least one processor (110) may determine the image sensor having the largest aperture value among the plurality of aperture values ​​as the primary image sensor. The primary image sensor may be described as the main image sensor.

[0080] The basic image sensor is assumed to be the first image sensor. At least one processor (110) can acquire a first exposure value of the first image sensor. At least one processor (110) can acquire a second exposure value of the second image sensor.

[0081] The description of the aperture value and exposure value is shown in FIGS. 14 and FIGS. 15.

[0082] At least one processor (110) can acquire a plurality of shooting frames through a first image sensor and a second image sensor for a preset time. At least one processor (110) can identify a plurality of target frames among the plurality of shooting frames based on exposure values. At least one processor (110) can merge the plurality of target frames to generate a long exposure frame. The plurality of target frames may represent frames with constant brightness change. The plurality of target frames may limit brightness change to within a certain level, thereby representing frames with uniform quality (brightness value within a threshold range) despite the occurrence (or detection) of sudden light. At least one processor (110) can provide a user with an output image of uniform quality by generating a plurality of target frames with a brightness change amount within a certain level.

[0083] The operation of generating a long exposure frame is described in FIGS. 5 and FIGS. 6.

[0084] At least one processor (110) can determine a first exposure value corresponding to a first image sensor. At least one processor (110) can determine a second exposure value corresponding to a second image sensor. At least one processor (110) can acquire a first shooting frame with a first exposure value at a first time point through the first image sensor. At least one processor (110) can acquire a second shooting frame with a second exposure value at a first time point through the second image sensor.

[0085] At least one processor (110) can acquire a first shooting frame and a second shooting frame at the same time. Since the setting values ​​of the first image sensor and the second image sensor are different, the first shooting frame and the second shooting frame may be different despite being at the same time. The setting values ​​may include physical characteristics of the lens itself or software characteristics applied to the lens. For example, the exposure value may differ for each lens. The exposure value may be determined based on at least one of the aperture or shutter speed. For example, the exposure value may change based on the surrounding environment (e.g., brightness).

[0086] For example, the shutter speed of the first image sensor and the shutter speed of the second image sensor may be the same. The aperture ratio of the first image sensor and the aperture ratio of the second image sensor may be different. Therefore, even if the shutter speeds are the same, the exposure value of the first image sensor and the exposure value of the second image sensor may be different.

[0087] As another example, the shutter speed of the first image sensor and the shutter speed of the second image sensor may be different. The aperture ratio of the first image sensor and the aperture ratio of the second image sensor may be different. Therefore, the exposure value of the first image sensor and the exposure value of the second image sensor may be different.

[0088] At least one processor (110) can determine an exposure value corresponding to a real-time environment. At least one processor (110) can acquire a shooting frame based on an exposure value corresponding to a real-time environment.

[0089] When a first shooting frame is acquired, at least one processor (110) can determine a third exposure value corresponding to the first image sensor. When a second shooting frame is acquired, at least one processor (110) can determine a fourth exposure value corresponding to the second image sensor.

[0090] At least one processor (110) can acquire a third shooting frame with a third exposure value at a second time point through a first image sensor. At least one processor (110) can acquire a fourth shooting frame with a fourth exposure value at a second time point through a second image sensor.

[0091] The operation of acquiring a shooting frame according to multiple viewpoints is described in FIGS. 7 to 10.

[0092] At least one processor (110) can determine a first shooting frame as a first target frame. At least one processor (110) can determine one of a third shooting frame and a fourth shooting frame as a second target frame. At least one processor (110) can merge the first target frame and the second target frame to generate a long exposure frame.

[0093] At least one processor (110) can determine a first exposure value as a target exposure value. At least one processor (110) can obtain a first difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value. If the first difference value is less than a first threshold value, at least one processor (110) can determine the third shooting frame as a second target frame.

[0094] If the first difference value is greater than or equal to the first threshold value, at least one processor (110) can obtain a second difference value which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value. If the first difference value exceeds the second difference value, at least one processor (110) can determine the fourth shooting frame as the second target frame.

[0095] The operation of determining the first target frame and the second target frame is described in FIGS. 11, 16, and 17.

[0096] The electronic device (100) may include an inertial sensor. For example, the inertial sensor may include an IMU (Inertial Measurement Unit). The inertial sensor may sense the movement of the electronic device (100). The inertial sensor may include at least one of an accelerometer and a gyroscope.

[0097] At least one processor (110) can obtain a first inertia vector at a first time point through an inertia sensor. At least one processor (110) can obtain a second inertia vector at a second time point through an inertia sensor.

[0098] At least one processor (110) can obtain a first difference vector of a first inertia vector and a second inertia vector.

[0099] If the first difference vector is less than the threshold size, at least one processor (110) can merge the first target frame and the second target frame to generate a long exposure frame.

[0100] The first difference vector may be a vector representing the movement of the electronic device (100).

[0101] If the first difference vector is greater than or equal to a threshold size, at least one processor (110) can identify target objects included in the first target frame and the second target frame.

[0102] At least one processor (110) can identify a first location of a target object in a first target frame at a first time point. At least one processor (110) can identify a second location of a target object in a second target frame at a second time point.

[0103] At least one processor (110) can acquire a first motion vector of a target object based on a first position and a second position. At least one processor (110) can generate a long exposure frame based on the correlation value of the first difference vector and the first motion vector.

[0104] The first motion vector can represent the movement of the target object.

[0105] The correlation value may be a value representing the relationship between the movement of the electronic device (100) and the movement of the target object.

[0106] An explanation of the correlation is provided in FIGS. 12, FIGS. 20, and FIGS. 22.

[0107] If the correlation value is less than or equal to the second threshold, at least one processor (110) can determine the target object as a fixed object.

[0108] If the correlation value exceeds the second threshold, at least one processor (110) can determine the target object as a non-fixed object.

[0109] At least one processor (110) can generate a long exposure frame by merging a first target frame and a second target frame based on the position of a fixed object.

[0110] The operation of generating a long exposure frame by considering a fixed object can be described in FIG. 13 and FIG. 21.

[0111] At least one processor (110) can display various screens related to the function of generating long exposure frames. Descriptions of the various screens are described in FIGS. 23 and 24.

[0112] The electronic device (100) may not generate a long exposure frame using all captured frames acquired through a plurality of image sensors. The electronic device (100) may select only the captured frames corresponding to an exposure value within a threshold range as target frames based on a reference exposure value (first exposure value) acquired at an initial point in time. When generating a long exposure frame with a captured frame that is close to or within a range similar to the reference exposure value, the long exposure frame may reflect unintended changes in light relatively less.

[0113] FIG. 3 is a block diagram illustrating the specific configuration of the electronic device (100) of FIG. 2 according to one embodiment.

[0114] FIG. 3 is a block diagram illustrating the specific configuration of the electronic device (100) of FIG. 2 according to one embodiment.

[0115] FIG. 3 is a block diagram of an exemplary electronic device (100) capable of performing the operations described in this document.

[0116] Referring to FIG. 3, the electronic device (100) may be one of various forms of electronic devices, such as a notebook (190), smartphones (191) having various form factors (e.g., a bar-type smartphone (191-1), a foldable-type smartphone (191-2), or a sliderable (or rollable)-type smartphone (191-3)), a tablet (192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 3 are illustrative only and are not intended to limit the implementations described or claimed herein. The electronic device (100) may be referred to as a mobile device, a user device, a multifunction device, a portable device, or a server.

[0117] The electronic device (100) may include components comprising at least one processor (110) (hereinafter referred to as processor (110)), at least one memory (120) (hereinafter referred to as memory (120)), at least one display (140) (hereinafter referred to as display (140)), at least one image sensor (150) (hereinafter referred to as image sensor (150)), at least one communication circuit (160) (hereinafter referred to as communication circuit (160)), and / or at least one sensor (170) (hereinafter referred to as sensor (170)). The components are merely exemplary. For example, the electronic device (100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuit, antenna, rechargeable battery, or input / output interface). For example, some components may be omitted from the electronic device (100). For example, some components may be integrated into a single component.

[0118] The processor (110) may be implemented as one or more integrated circuit (or circuitry) chips and may perform various data processing operations. The processor (110) may include at least one electrical circuit and may process instructions (or programs, data) stored in memory (120) individually or collectively in a distributed manner. The processor (110) may include a processor assembly comprising one or more processing circuits. The processor (110) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (100) (e.g., memory (120), display (140), image sensor (150), communication circuit (160), and / or sensor (170)). For example, the processor (110) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (110) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (110) may include one or more processing circuits. For example, the processor (110) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (110) may be included in a first chip of the electronic device (100), and at least another portion of the processor (110) may be included in a second chip of the electronic device (100) different from the first chip of the electronic device (100).

[0119] For example, the processor (110) may include a central processing unit (111), a graphics processing unit (112), a neural processing unit (113), an image signal processor (114), a display controller (115), a memory controller (116), a storage controller (117), a communication processor (118), and / or a sensor interface (119). These components of the processor (110) are merely exemplary. For example, the processor (110) may include other components. For example, some components of the processor (110) may be omitted from the processor (110). For example, some components of the processor (110) may be included as separate components of the electronic device (100) outside of the processor (110). For example, some components of the processor (110) (e.g., memory controller (116)) may be included in other components (e.g., at least part of memory (120), an interface (e.g. available for connection to at least one component of the electronic device (100)), a display (140) and / or an image sensor (150)).

[0120] The processor (110) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (120). The CPU (111) (or central processing circuit) may be configured to control the components of the processor (110) based on the execution of instructions stored in memory (120) (e.g., volatile memory (121) and / or non-volatile memory (122)). The GPU (112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (113) (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). An ISP (114) (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor (150) into a format suitable for a component within the electronic device (100) or a component of the processor (110). A display controller (115) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from a CPU (111), GPU (112), ISP (114), or memory (120) (e.g., volatile memory (121)) into a format suitable for a display (140). A memory controller (116) (or memory control circuit) may be configured to control reading data from the volatile memory (121) and writing data to the volatile memory (121). A storage controller (117) (or storage control circuit) may be configured to control reading data from the non-volatile memory (122) and writing data to the non-volatile memory (122).The CP (118) (communication processing circuit) may be configured to process data obtained from a component of the processor (110) into a format suitable for transmitting to another electronic device via the communication circuit (160), or to process data obtained from another electronic device via the communication circuit (160) into a format suitable for processing by the component of the processor (110). For example, the communication circuit (160) may include one or more communication circuits. The sensor interface (119) (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (100) and / or the state around the electronic device (100), obtained through the sensor (170), into a format suitable for the component of the processor (110).

[0121] Memory (120) may include one or more storage media (or one or more storage devices). For example, memory (120) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, a permanent memory such as flash memory, read-only memory (ROM) (e.g., non-volatile memory (122)), a semi-permanent memory such as random access memory (RAM) (e.g., volatile memory (121)), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (120) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As an example not limited to, the cache memory may be included within the processor (110). The memory (120) may be fixedly embedded within the electronic device (100) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (100).

[0122] For example, memory (120) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (110). For example, memory (120) may store instructions that can be called by an application programming interface (API). For example, memory (120) may store instructions within a library.

[0123] FIG. 4 is a drawing for explaining a camera module according to one embodiment.

[0124] FIG. 4 is a block diagram (200) illustrating the configuration of a camera module included in an electronic device (100) according to one embodiment.

[0125] In one embodiment, the camera included in the electronic device (100) may include a lens assembly (201), an image sensor (203) (e.g., the image sensor (150) of FIG. 3), a sensor interface (205), an ISP (211), a controller (215), an auto focus (AF) controller (209), a flash (217), and an optical image stabilizer (OIS) (223).

[0126] In one embodiment, light from an object incident through a lens assembly (201) can be converted into an electrical signal by an image sensor (203). The signal output from the image sensor (203) can be input to an image signal processor (ISP) (211) through a sensor interface (205). An infrared cut filter (IR cut filter) can be placed on the upper surface of the image sensor (203). Light from an object passing through the lens assembly (201) can be detected by the image sensor (203) after a portion of it is filtered by the infrared cut filter.

[0127] In one embodiment, the ISP (211) can perform operations related to an image signal output from an image sensor (203). The ISP (211) may include at least one of an ISP chain or a Pre-ISP. An ISP chain may mean, for example, a plurality of function blocks connected to perform the functions of an ISP. An ISP function block may mean a unit of hardware and / or software that performs any one of image signal processing functions. An ISP function block may perform at least one of image signal processing functions. For example, a function block may perform at least one of image signal processing functions such as noise reduction, edge enhancement, gamma correction, or color interpolation. An ISP chain may be implemented as a chip having the structure of an ISP chain, or as a software module executed by a processor (e.g., the processor (110) of FIG. 3). The ISP may perform image signal processing to obtain desired image data from the image signal. Pre-ISP can perform operations related to the video signal before performing video signal processing in the ISP chain. For example, AWB (auto white balance), auto exposure (AE) control, and auto focusing (AF) operations can be performed in Pre-ISP. The ISP (211) can store data in memory (213) or use data stored in memory (213) to perform operations related to the video signal.

[0128] In one embodiment, the ISP chain of the ISP (211) can process an image signal obtained through the image sensor (203). For example, the ISP chain can perform at least one of lens shading correction, dead pixel correction, noise control, tone curve adjustment, color correction and adjustment, edge enhancement, demosaic, or remosaic.

[0129] In one embodiment, the ISP (211) may be implemented as at least part of a processor (e.g., processor (110) of FIG. 3) (or integrated circuit) constituting the controller (215), but is not limited thereto. In one embodiment, the ISP (211) may be implemented as a separate processor (or integrated circuit). In one embodiment, the ISP (211) may be implemented through a processing unit included in the image sensor (203). In one embodiment, the ISP (211) may be distributed among a plurality of components (e.g., a processing unit of the image sensor (203), a separate processor, and the controller (215)).

[0130] In one embodiment, the controller (215) may control the display (140) to display an execution screen of an application executed by the controller (215) or a screen stored in memory (213). Memory (213) may include at least one recording (or storage) medium. For example, memory (213) may include at least one of volatile memory (121), such as RAM (random access memory), non-volatile memory (122), such as flash memory, or buffer memory.

[0131] In one embodiment, the optical image stabilizer (223) may move at least a part of the lens assembly (201) or the image sensor (203) in response to the movement of the electronic device (100) to eliminate or reduce shaking of the captured image. At least a part of the lens assembly (201) or the image sensor (203) may move to offset the movement of the electronic device (100). In one embodiment, the optical image stabilizer (223) may obtain information about the movement of the electronic device (100) through a motion sensor (221) (e.g., sensor (170) of FIG. 3). For example, the motion sensor (221) may include a gyro sensor.

[0132] In one embodiment, the autofocus adjuster (209) can adjust the distance between at least one lens of the lens assembly (201) and the image sensor (203) so that an image is formed on the image sensor (203) by light passing through the lens assembly (201). For example, the ISP (211) can determine the phase difference between pixels (or the phase difference between subpixels) from image data obtained through the image sensor (203). The controller (215) can control the autofocus adjuster (209) to adjust the focus based on the determined phase difference. However, the method of operation of the autofocus adjuster (209) is not limited thereto. For example, the autofocus adjuster (209) may perform focus adjustment based on the position of the lens assembly (201) or image sensor (203) where an image exhibiting maximum contrast is obtained while moving the lens assembly (201) or image sensor (203).

[0133] FIG. 5 is a drawing for explaining the operation of generating a long exposure frame according to one embodiment.

[0134] Referring to FIG. 5, the electronic device (100) can identify an event to start a long exposure function (S510). The event to start a long exposure function may include at least one of an event in which a user input to start a long exposure function is received or an event in which the ambient light is below a threshold light level.

[0135] The electronic device (100) can identify whether an event to start the long exposure function occurs. When an event to start the long exposure function is identified (S510-Y), the electronic device (100) can set an exposure value (S520). The electronic device (100) can determine the exposure value required to perform the shooting function. The electronic device (100) can set the exposure value required for the image sensor (150).

[0136] The electronic device (100) can acquire multiple shooting frames based on a set exposure value (S530). The electronic device (100) can acquire multiple shooting frames for a preset time.

[0137] The electronic device (100) can generate a long exposure frame by merging multiple shooting frames (S540). The electronic device (100) can obtain a long exposure frame, which is a result frame of the long exposure function.

[0138] FIG. 6 is a diagram illustrating the operation of generating a long exposure frame based on light change and motion analysis according to one embodiment.

[0139] Referring to FIG. 6, the electronic device (100) can identify whether a camera application is running (S610). The electronic device (100) can run the camera application according to user input. When the camera application is running (S610-Y), the electronic device (100) can identify the primary image sensor among a plurality of image sensors. The primary image sensor may be described as the main image sensor. The primary image sensor may be the image sensor with the largest aperture value (f-number).

[0140] The electronic device (100) can determine a basic exposure value corresponding to a basic image sensor (S630). An explanation regarding the exposure value is described in FIG. 15.

[0141] The electronic device (100) can identify whether an event to start a long exposure function occurs (S640). The event to start a long exposure function may include at least one of an event in which user input to start a long exposure function is received or an event in which the ambient light is below a threshold light level.

[0142] When an event to start the long exposure function is identified (S640-Y), the electronic device (100) can acquire multiple shooting frames (S650). The electronic device (100) can acquire multiple shooting frames for a preset time. The preset time may represent the exposure time associated with the long exposure function. The preset time may be changed by the user's settings.

[0143] The electronic device (100) can determine whether an event for terminating the long exposure function is identified (S660). The event for terminating the long exposure function may include at least one of an event in which a threshold time elapses from the time the long exposure function starts, or an event in which a preset number of shooting frames are acquired from the time the long exposure function starts.

[0144] If no event is identified to terminate the long exposure function (S660-N), the electronic device (100) can re-acquire the captured frame (S650).

[0145] When an event to terminate the long exposure function is identified (S660-Y), the electronic device (100) can acquire a target frame set according to the light change (S670). The target frame set may include representative frames determined at each unit time point. The target frame set may represent a set of representative frames acquired during a preset time period. An explanation related to this is described in FIG. 16.

[0146] The electronic device (100) can generate a long exposure frame by merging target frames based on motion analysis (S680). Motion analysis may include an operation to analyze motion identified in a captured frame. An explanation related to motion analysis is described in FIG. 20.

[0147] The electronic device (100) can provide a long exposure frame (S690). For example, the electronic device (100) can display a long exposure frame. For example, the electronic device (100) can store a long exposure frame.

[0148] In the description of FIG. 6, it is stated that operations S620 and S630 are performed before identifying an event for starting the long exposure function (S640). According to another embodiment, operations S620 and S630 may be performed after identifying an event for starting the long exposure function (S640).

[0149] FIG. 7 is a diagram illustrating the operation of acquiring a plurality of shooting data according to one embodiment.

[0150] The operations S710, S720, S730, and S740 of FIG. 7 may correspond to the operations S610, S620, S630, and S640 of FIG. 6. The operations S751 and S752 of FIG. 7 may correspond to the operation S650 of FIG. 6. Redundant descriptions are omitted.

[0151] The electronic device (100) may include a first image sensor and a second image sensor.

[0152] When the camera application is executed (S710-Y), the electronic device (100) can identify the first image sensor having the largest aperture value among the plurality of image sensors (S720). An explanation regarding the aperture value is described in FIG. 15.

[0153] For example, when a camera application for acquiring a shooting frame is executed, the electronic device (100) can identify an aperture value for each of a plurality of image sensors included in the electronic device (100). The electronic device (100) can identify the image sensor corresponding to the largest aperture value among the plurality of aperture values ​​as the first image sensor (or primary image sensor).

[0154] For example, the electronic device (100) can store aperture values ​​of a plurality of image sensors in advance in memory (120). The electronic device (100) can identify the image sensor corresponding to the largest aperture value among the plurality of aperture values ​​as the first image sensor (or basic image sensor).

[0155] The electronic device (100) can obtain a first exposure value corresponding to a first image sensor and a second exposure value corresponding to a first image sensor (S730).

[0156] An explanation regarding the exposure value is described in FIG. 15. An example of obtaining the first exposure value and the second exposure value is described in FIG. 8 and FIG. 9.

[0157] The electronic device (100) can determine whether an event for starting a long exposure function is identified (S740). If an event for starting a long exposure function is identified (S740-Y), the electronic device (100) can acquire a first shooting frame through a first image sensor with a first exposure value at a first time point and acquire a second shooting frame through a second image sensor with a second exposure value at a first time point (S751).

[0158] The electronic device (100) can acquire first matching data including a first time point, a first exposure value, and a first shooting frame, and can acquire second matching data including a first time point, a second exposure value, and a second shooting frame (S752).

[0159] The first matching data may be data obtained by grouping information acquired through the first image sensor at the first time point. The second matching data may be data obtained by grouping information acquired through the second image sensor at the first time point.

[0160] In the description of FIG. 7, it is stated that operations S720 and S730 are performed before identifying an event for starting the long exposure function (S740). According to another embodiment, operations S720 and S730 may be performed after identifying an event for starting the long exposure function (S740).

[0161] FIGS. 8 and 9 describe the process of determining the exposure value of the S730 operation of FIG. 7.

[0162] FIG. 8 is a diagram illustrating the operation of acquiring an exposure value according to one embodiment.

[0163] The operation S810 of FIG. 8 may correspond to the operation S610 of FIG. 6 and the operation S710 of FIG. 7. Redundant descriptions are omitted.

[0164] When the camera application is executed (S810-Y), the electronic device (100) can acquire a first initial frame from the first image sensor and a second initial frame from the second image sensor (S831). The electronic device (100) can acquire the first initial frame and the second initial frame at the same time.

[0165] The initial frame may represent a frame acquired before an event occurs to start the long exposure function. The electronic device (100) can determine the exposure value by analyzing the initial frame.

[0166] The electronic device (100) can acquire first pixel data of a first initial frame. The electronic device (100) can acquire second pixel data of a second initial frame.

[0167] The electronic device (100) can obtain a first luminance value based on the first pixel data of the first initial frame and identify a second luminance value based on the second pixel data of the second initial frame (S832).

[0168] For example, an electronic device (100) can obtain an average value of all pixel data of an initial frame and obtain a luminance value based on the average value. The electronic device (100) can store a first mapping table in which the average value of the pixel data and the luminance value are mapped in memory (120). The electronic device (100) can identify a luminance value corresponding to the pixel data based on the first mapping table stored in memory (120).

[0169] The electronic device (100) can identify a first luminance value corresponding to the first pixel data based on a first mapping table. The electronic device (100) can identify a second luminance value corresponding to the second pixel data based on the first mapping table.

[0170] The electronic device (100) can determine a first exposure value based on a first brightness value and determine a second exposure value based on a second brightness value (S833).

[0171] For example, an electronic device (100) can acquire a luminance value of an initial frame and acquire an exposure value based on the luminance value. The electronic device (100) can store a second mapping table in which the luminance value and the exposure value are mapped in memory (120). The electronic device (100) can determine an exposure value corresponding to the luminance value based on the second mapping table stored in memory (120).

[0172] The electronic device (100) can determine a first exposure value corresponding to a first luminance value based on a second mapping table. The electronic device (100) can determine a second exposure value corresponding to a second luminance value based on a second mapping table.

[0173] FIG. 9 is a drawing for explaining the operation of acquiring an exposure value according to one embodiment.

[0174] The operation S910 of FIG. 9 may correspond to the operation S610 of FIG. 6 and the operation S710 of FIG. 7. Redundant descriptions are omitted.

[0175] When the camera application is executed (S910-Y), the electronic device (100) can acquire first illuminance data through the first image sensor and acquire second illuminance data through the second image sensor (S931). The electronic device (100) can acquire the first illuminance value and the second illuminance value at the same time.

[0176] The illuminance value may be a value representing the amount of light physically acquired by the image sensor. The luminance value described in FIG. 8 represents brightness obtained by analyzing pixel data, and the illuminance value described in FIG. 9 may represent brightness obtained by sensing the amount of light by the image sensor.

[0177] The electronic device (100) can determine a first exposure value based on a first illuminance value and determine a second exposure value based on a second illuminance value (S932).

[0178] For example, an electronic device (100) can acquire an illuminance value and acquire an exposure value based on the illuminance value. The electronic device (100) can store a third mapping table in which the illuminance value and the exposure value are mapped in memory (120). The electronic device (100) can determine an exposure value corresponding to the illuminance value based on the third mapping table stored in memory (120).

[0179] The electronic device (100) can obtain a first exposure value corresponding to a first illuminance value based on a third mapping table. The electronic device (100) can obtain a second exposure value corresponding to a second illuminance value based on a third mapping table.

[0180] FIG. 10 is a drawing for explaining the operation of acquiring a target frame set according to one embodiment.

[0181] The operation S1060 of FIG. 10 may correspond to the operation S660 of FIG. 6. Redundant explanation is omitted.

[0182] FIG. 10 may show an embodiment following the S751 and S752 operations of FIG. 7. The electronic device (100) may determine a third exposure value corresponding to the first image sensor and determine a fourth exposure value corresponding to the second image sensor (S1053).

[0183] The exposure value may be changed based on the real-time environment (brightness environment). The electronic device (100) can determine an exposure value corresponding to each of the first image sensor and the second image sensor.

[0184] The electronic device (100) can acquire a third shooting frame through the first image sensor with a third exposure value at a second time point, and acquire a fourth shooting frame through the second image sensor with a fourth exposure value at a second time point (S1054). The electronic device (100) can acquire the third shooting frame and the fourth shooting frame at the same second time point.

[0185] The electronic device (100) can acquire third matching data including a second time point, a third exposure value, and a third shooting frame, and can acquire fourth matching data including a second time point, a fourth exposure value, and a fourth shooting frame (S1055).

[0186] The third matching data may be data obtained by grouping information acquired through the first image sensor at the second time point. The fourth matching data may be data obtained by grouping information acquired through the second image sensor at the second time point.

[0187] The electronic device (100) can identify whether an event for ending the long exposure function has occurred (S1060). If an event for ending the long exposure function is not identified (S1060-N), the electronic device (100) can acquire a new shooting frame at a new point in time (next point in time). When a new frame is acquired, the electronic device (100) can acquire and store new matching data.

[0188] When an event for ending the long exposure function is identified (S1060-Y), the electronic device (100) can acquire a set of target frames according to the light change (S1070). The light intensity may change in real time depending on the surrounding environment. The electronic device (100) can determine a representative frame according to the light intensity. An explanation related to this is described in FIG. 11.

[0189] FIG. 11 is a diagram illustrating the operation of acquiring a target frame set based on an exposure value according to one embodiment.

[0190] Referring to FIG. 11, the electronic device (100) can determine a first shooting frame at a first time point as a first target frame (S1171-1). The first target frame may be described as a basic frame or a reference frame. The target frame may be described as an optimal frame.

[0191] The electronic device (100) can determine a first exposure value corresponding to a first shooting frame as a target exposure value (S1171-2). The target exposure value may be described as a basic exposure value or a reference exposure value.

[0192] The electronic device (100) can acquire a third shooting frame and a fourth shooting frame at a second time point. When the third shooting frame and the fourth shooting frame are acquired, the electronic device (100) can compare the third exposure value at the second time point with the first exposure value at the first time point.

[0193] The higher the exposure value, the brighter the surrounding environment can be indicated. The electronic device (100) can identify the degree to which the brightness of the surrounding environment changes by comparing exposure values ​​at different points in time.

[0194] The electronic device (100) can obtain a first difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value (S1172). The first difference value may be a value representing the difference between the exposure value at the second time point and the exposure value at the first time point. As the first difference value increases, it may indicate that the brightness of the surrounding environment has changed significantly.

[0195] The electronic device (100) can identify whether the first difference value is greater than or equal to the first threshold value (S1173). The first threshold value may be changed according to the user's settings. The first threshold value may be a value for comparing brightness changes. The first threshold value may be described as the first threshold exposure value. The first threshold value may be described as a preset exposure value related to brightness.

[0196] If the first difference value is less than the first threshold value (S1173-N), the electronic device (100) can determine the third captured frame acquired through the first image sensor as the second target frame (S1174). If the first difference value is less than the first threshold value, the electronic device (100) can determine that the change in brightness is constant. The electronic device (100) can determine the target frame using only the captured frame acquired through the first image sensor.

[0197] The electronic device (100) can acquire a target frame set including a first target frame and a second target frame (S1179). If the second exposure value is less than the target exposure value, the electronic device (100) can acquire a target frame set including a first shooting frame acquired through the first image sensor and a third shooting frame acquired through the first image sensor.

[0198] If the first difference value is greater than or equal to the first threshold value (S1173-Y), the electronic device (100) can obtain a second difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value (S1175). If the first difference value is greater than or equal to the first threshold value, the electronic device (100) can determine that the change in brightness has exceeded a certain level. The electronic device (100) can determine that compensation for the change in brightness is required.

[0199] The electronic device (100) can identify whether the first difference value is less than or equal to the second difference value (S1176).

[0200] If the first difference value is less than the second difference value (S1176-N), the electronic device (100) can determine the fourth shooting frame corresponding to the fourth exposure value as the second target frame (S1177).

[0201] If the first difference value is greater than or equal to the second difference value (S1176-Y), the electronic device (100) can determine the third shooting frame corresponding to the third exposure value as the second target frame (S1178).

[0202] The electronic device (100) can acquire a set of target frames including a first target frame and a second target frame (S1179).

[0203] If the first difference value is less than the first threshold value (S1173-N), the electronic device (100) determines that the brightness of the surrounding environment is constant and can obtain a target frame set including only the captured frame obtained through the first image sensor.

[0204] If the first difference value is greater than or equal to the first threshold value (S1173-Y), the electronic device (100) determines that the brightness of the surrounding environment has changed and can acquire a target frame set including a shooting frame having an exposure value most similar to the target exposure value (first exposure value) at each point in time. An explanation related to this is described in FIG. 16.

[0205] When a target frame set is acquired, the electronic device (100) can perform a motion analysis function for the captured frames included in the target frame set.

[0206] FIG. 12 is a drawing for explaining motion analysis operations according to one embodiment.

[0207] Referring to FIG. 12, the electronic device (100) can acquire first inertial data through an inertial sensor at a first time point and acquire second inertial data through an inertial sensor at a second time point (S1281).

[0208] The inertial sensor can sense the three-dimensional movement of the electronic device (100). As an example, the inertial sensor may include an IMU (Inertial Measurement Unit).

[0209] While acquiring a first shooting frame and a second shooting frame at a first time point, the electronic device (100) can acquire first inertial data through an inertial sensor. The electronic device (100) can store the first inertial data in a memory (120).

[0210] While acquiring the third and fourth shooting frames at the second time point, the electronic device (100) can acquire second inertial data through an inertial sensor. The electronic device (100) can store the second inertial data in memory (120).

[0211] When a target frame set is obtained, the electronic device (100) can obtain a first inertia vector based on first inertia data. When a target frame set is obtained, the electronic device (100) can obtain a second inertia vector based on second inertia data.

[0212] For example, inertial data may represent sensing data obtained through an inertial sensor. An inertial vector may represent data that quantifies the direction of movement of an electronic device (100).

[0213] The electronic device (100) can obtain a first difference vector between a first inertia vector and a second inertia vector (S1283).

[0214] If there is no movement of the electronic device (100), the first difference vector may be 0. This is because the inertial data at the first time point and the inertial data at the second time point are the same.

[0215] If the electronic device (100) has movement, the first difference vector may have a non-zero value. This is because the first inertial data at the first time point and the second inertial data at the second time point are different.

[0216] The electronic device (100) can identify whether the magnitude of the first difference vector is greater than or equal to a threshold size (S1284). The threshold size can be changed according to the user's settings.

[0217] If the magnitude of the first difference vector is less than the threshold size (S1284-N), the electronic device (100) can generate a first long exposure frame by merging the first target frame and the second target frame included in the target frame set (S1285).

[0218] If the magnitude of the first difference vector is greater than or equal to a threshold size (S1284-Y), the electronic device (100) can identify a target object included in the first target frame and the second target frame (S1286). The target object may represent an object that is commonly included in both the first target frame and the second target frame. The first target frame may be a frame acquired at a first time point, and the second target frame may be a frame acquired at a second time point. Since the time points are different, not all objects may be included in the same way. The electronic device (100) can identify an object that is included in both the first target frame and the second target frame as a target object.

[0219] For example, there can be one target object.

[0220] For example, there can be multiple target objects.

[0221] If the target object is not identified (S1286-N), the electronic device (100) can generate a first long exposure frame by merging the first target frame and the second target frame included in the target frame set (S1285).

[0222] When the target object is identified (S1286-Y), the electronic device (100) can identify the first location of the target object in the first target frame and the second location of the target object in the second target frame (S1287).

[0223] The electronic device (100) can obtain a first motion vector of a target object based on a first position and a second position (S1288). The first motion vector may be data that quantifies the change in position of the target object from a first point in time to a second point in time. The first motion vector may represent the change in position of the target object at a second point in time relative to the first point in time.

[0224] The electronic device (100) can analyze the correlation between the first difference vector and the first motion vector (S1289). The second inertia vector can represent the position change of the electronic device (100) at the second time point. The first motion vector can represent the position change of the target object at the second time point relative to the first time point. The electronic device (100) can analyze the correlation between the position change of the electronic device (100) and the position change of the target object at the second time point.

[0225] FIG. 13 is a drawing for explaining motion analysis operations according to one embodiment.

[0226] Referring to FIG. 13, the electronic device (100) can obtain a correlation value between a first difference vector and a first motion vector (S1381). The correlation value can indicate the relationship between the first difference vector and the first motion vector. The correlation value can indicate whether the position change of the first difference vector and the first motion vector matches.

[0227] The electronic device (100) can compare the first difference vector and the first motion vector. Since the sources of the two vectors are different, the electronic device (100) can use some conversion function. The electronic device (100) can convert the first inertia vector based on a preset conversion function and compare the converted vector with the first motion vector. The preset conversion function can be stored in memory (120) in advance.

[0228] For example, the electronic device (100) can obtain the cosine similarity of the first difference vector and the first motion vector as a correlation value. An explanation regarding the cosine similarity is described in FIG. 22. Before calculating the cosine similarity, the first difference vector or the first motion vector may be converted based on a pre-set function for comparison.

[0229] The electronic device (100) can identify whether the correlation value is below a second threshold value (S1382). The second threshold value can be changed according to the user's settings.

[0230] For example, cosine similarity can range from -1 to 1. -1 indicates that the two vectors being compared are in opposite directions. 1 indicates that the two vectors being compared are in the same direction.

[0231] The first difference vector may be data obtained by the movement of the electronic device (100).

[0232] The first motion vector may represent a change in the position of a target object acquired based on captured frames. The first motion vector may be data acquired by the movement of the electronic device (100) and the movement of the target object itself.

[0233] If the target object is fixed, the first difference vector and the first motion vector may include only the movement of the electronic device (100). If there is only the movement of the electronic device (100) and the target object is fixed, the directions of the first difference vector and the first motion vector may be opposite.

[0234] For example, in a situation where a target object is being photographed, if the electronic device (100) is moved downward, the target object may move upward in the photographed frame.

[0235] If there is only movement of the electronic device (100) and the target object is fixed, the correlation value between the first difference vector and the first motion vector may be a value indicating the opposite direction.

[0236] For example, when the correlation value is obtained using cosine similarity, the correlation value may be -1. An explanation related to this is described in Fig. 22.

[0237] The electronic device (100) can determine the type of target object of the electronic device (100) by comparing the correlation value with the second threshold value. The type of target object can indicate whether the target object is a fixed object or a non-fixed object.

[0238] If the correlation value is less than or equal to the second threshold value (S1382-Y), the electronic device (100) can determine the target object as a fixed object (S1383).

[0239] If the correlation value exceeds the second threshold (S1382-N), the electronic device (100) can determine the target object as a movable object (S1384).

[0240] Fixed objects may be described as Type 1 objects. Non-fixed objects may be described as Type 2 objects.

[0241] When the type of target object is determined, the electronic device (100) can acquire a first target frame and a second target frame included in a target frame set (S1385).

[0242] The electronic device (100) can generate a second long-exposure frame by merging a first target frame and a second target frame based on a fixed object (S1386).

[0243] The first long exposure frame in operation S1285 of FIG. 12 can be generated without considering the type of target object. The second long exposure frame in operation S1386 can be generated considering the type of target object.

[0244] FIG. 14 is a drawing for explaining exposure values ​​according to one embodiment.

[0245] Referring to FIG. 14, the electronic device (100) can capture a shooting frame through a plurality of image sensors at the same time.

[0246] Referring to the embodiment (1410) of FIG. 14, the electronic device (100) may include a first image sensor having a first aperture value (F / 2.2). The electronic device (100) may acquire a first exposure value (EV 0) of the first image sensor. Based on the first exposure value (EV 0), the electronic device (100) may acquire a first shooting frame (1411) through the first image sensor.

[0247] Referring to the embodiment (1420) of FIG. 14, the electronic device (100) may include a second image sensor having a second aperture value (F / 1.8). The electronic device (100) may acquire a second exposure value (EV +1.3) of the second image sensor. Based on the second exposure value (EV +1.3), the electronic device (100) may acquire a second shooting frame (1412) through the second image sensor.

[0248] The electronic device (100) can acquire a first shooting frame (1411) and a second shooting frame (1412) at the same time.

[0249] The electronic device (100) may determine (or select) the image sensor having the largest aperture value among a plurality of image sensors as the primary image sensor. The electronic device (100) may determine the first image sensor having a first aperture value (F / 2.2) as the primary image sensor. The electronic device (100) may determine the first exposure value (EV 0) of the first image sensor as the primary exposure value. An explanation related to this is described in the S720 and S730 operations of FIG. 7.

[0250] FIG. 15 is a drawing for explaining exposure values ​​according to one embodiment.

[0251] Referring to the mathematical formula (1510) of FIG. 15, the electronic device (100) can calculate an exposure value. The exposure value can be calculated based on an aperture value (N: aperture, f-number) and a shutter speed (t). The shutter speed (t) can be described as a shutter speed, exposure time, etc.

[0252] For example, the electronic device (100) can perform shooting according to the long exposure function for a preset time. The preset time may represent the total shooting time for acquiring multiple frames. The electronic device (100) can acquire an exposure value according to the shutter speed corresponding to the image sensor.

[0253] For example, an image sensor with a shutter speed of 1 second can acquire 10 shooting frames during a preset time (10 seconds).

[0254] The shutter speed of an image sensor is determined by the characteristics of the image sensor itself or can be changed by user settings. Multiple image sensors may exist.

[0255] For example, the shutter speeds of multiple image sensors can all be the same.

[0256] For example, the shutter speeds of some image sensors among multiple image sensors may differ.

[0257] Referring to the mathematical formula (1520) of FIG. 15, the electronic device (100) can calculate the aperture value. The aperture value can be calculated based on the focal length (f) and the aperture aperture.

[0258] For example, the aperture diameter can be fixed.

[0259] For example, the aperture size can be changed in real time.

[0260] For example, the shutter speed can be the same for multiple image sensors.

[0261] For example, shutter speeds can vary depending on multiple image sensors.

[0262] According to one embodiment, in the operation described in FIGS. 6 to 13, images captured by each image sensor are acquired at the same point in time. In FIGS. 6 to 13, the shutter speed may be the same.

[0263] According to another embodiment, even if the shutter speeds are different, the electronic device (100) can acquire matching data by matching shooting frames corresponding to the same point in time for each sensor. The electronic device (100) can acquire matching data by matching shooting frames with the same absolute point in time.

[0264] FIG. 16 is a drawing for explaining a shooting frame acquired from a plurality of image sensors according to one embodiment.

[0265] The embodiment (1610) of FIG. 16 may show a plurality of frames (F1, F3, F5, F7) acquired through the first image sensor.

[0266] The electronic device (100) can acquire a first shooting frame (F1) at a first time point. The electronic device (100) can acquire a third shooting frame (F3) at a second time point. The electronic device (100) can acquire a fifth shooting frame (F5) at a third time point. The electronic device (100) can acquire a seventh shooting frame (F7) at a fourth time point.

[0267] The embodiment (1620) of FIG. 16 may show a plurality of frames (F2, F4, F6, F8) acquired through a second image sensor. The electronic device (100) may acquire a second captured frame (F2) at a first time point. The electronic device (100) may acquire a fourth captured frame (F4) at a second time point. The electronic device (100) may acquire a sixth captured frame (F6) at a third time point. The electronic device (100) may acquire an eighth captured frame (F8) at a fourth time point.

[0268] When an event for starting the long exposure function is identified, the electronic device (100) can acquire a shooting frame for a preset time (e.g., 4 seconds).

[0269] For example, the electronic device (100) can obtain a target frame set using only frames (F1, F3, F5, F7) obtained through the first image sensor.

[0270] For example, an electronic device (100) can obtain a target frame set by combining frames (F1, F4, F6, F7) obtained through a first image sensor and a second image sensor to compensate for changes in light. The combination method is described in FIG. 17.

[0271] FIG. 17 is a drawing for explaining the operation of determining a target frame according to one embodiment.

[0272] Referring to the table (1710) in FIG. 17, the electronic device (100) can match the captured frames obtained using the exposure value and the exposure value from the first time point (t1) to the fourth time point (t4).

[0273] The electronic device (100) can acquire a first shooting frame (F1) with a first exposure value (EV1) at a first time point (t1) through a first image sensor.

[0274] The electronic device (100) can acquire a third shooting frame (F3) with a third exposure value (EV3) at a second time point (t2) through a first image sensor.

[0275] The electronic device (100) can acquire a fifth shot frame (F5) with a fifth exposure value (EV5) at a third time point (t3) through a first image sensor.

[0276] The electronic device (100) can acquire a seventh shot frame (F7) with a seventh exposure value (EV7) at a fourth time point (t4) through a first image sensor.

[0277] The electronic device (100) can acquire a second shooting frame (F2) with a second exposure value (EV2) at a first time point (t1) through a second image sensor.

[0278] The electronic device (100) can acquire a seventh shooting frame (F4) with a fourth exposure value (EV4) at a second time point (t2) through a second image sensor.

[0279] The electronic device (100) can acquire a sixth shot frame (F6) with a sixth exposure value (EV6) at a third time point (t3) through a second image sensor.

[0280] The electronic device (100) can acquire an 8th shot frame (F8) with an 8th exposure value (EV8) at a 4th time point (t4) through a 2nd image sensor.

[0281] Referring to the table (1720) in FIG. 17, the electronic device (100) can determine a shooting frame having an exposure value with a small difference from the exposure values ​​as a target frame. The electronic device (100) can determine a first exposure value (0) as a target exposure value. The electronic device (100) can determine a frame having an exposure value close to the first exposure value (0), which is a reference exposure value, as a target frame at the second to fourth time points.

[0282] At the first point in time, the electronic device (100) can determine a first shooting frame (F1) having a target exposure value (0) as the first target frame.

[0283] At the second point in time, the electronic device (100) can obtain a first difference value (5), which is the absolute value of the value obtained by subtracting the third exposure value (5) from the target exposure value (0). If the first difference value (5) is greater than or equal to the threshold value (3), the electronic device (100) can obtain a second difference value (0.1), which is the absolute value of the value obtained by subtracting the fourth exposure value (-0.1) from the target exposure value (0). If the first difference value (5) is greater than the second difference value (0.1), the electronic device (100) can determine the fourth frame (F4) corresponding to the fourth exposure value (-0.1) as the second target frame.

[0284] At the third point in time, the electronic device (100) can obtain a third difference value (7), which is the absolute value of the value obtained by subtracting the fifth exposure value (7) from the target exposure value (0). If the third difference value (7) is greater than or equal to the threshold value (3), the electronic device (100) can obtain a fourth difference value (0.1), which is the absolute value of the value obtained by subtracting the sixth exposure value (0.1) from the target exposure value (0). If the third difference value (7) is greater than the fourth difference value (0.1), the electronic device (100) can determine the sixth frame (F6) corresponding to the fourth exposure value (0.1) as the third target frame.

[0285] At the fourth point in time, the electronic device (100) can obtain a fifth difference value (0), which is the absolute value of the value obtained by subtracting the seventh exposure value (0) from the target exposure value (0). If the fifth difference value (0) is not greater than or equal to the threshold value (3), the electronic device (100) can determine the seventh shooting frame (F7) having the seventh exposure value (0) as the fourth target frame.

[0286] The electronic device (100) can acquire a target frame set including a first target frame at a first time point, a second target frame at a second time point, a third target frame at a third time point, and a fourth target frame at a fourth time point. The electronic device (100) can store the target frame set in memory (120).

[0287] FIG. 18 is a diagram illustrating the operation of generating a merged image according to one embodiment.

[0288] Referring to FIG. 18, the electronic device (100) can acquire a target frame set (1810) including shooting frames (F1, F3, F5, F7) acquired using only the first image sensor. An unintended strong light may be detected at a second time point. Depending on the strong light detected at the second time point, the third shooting frame (F3) and the fifth shooting frame (F5) may be captured relatively brightly.

[0289] The electronic device (100) can acquire a long exposure frame (1880) by merging the captured frames (F1, F3, F5, F7) acquired using only the first image sensor. The long exposure frame (1880) may be a frame in which unintended strong light is reflected.

[0290] Depending on the type of image sensor, it may not react sensitively to strong light. It is assumed that the second image sensor is a sensor that does not react sensitively to strong light. The electronic device (100) can obtain a target frame set by selecting the captured frames (F1, F3, F5, F7) obtained from the first image sensor and the captured frames (F2, F4, F6, F8) obtained from the second image sensor based on the viewpoint and exposure value.

[0291] FIG. 19 is a diagram illustrating the operation of generating a merged image according to one embodiment.

[0292] Referring to FIG. 19, the electronic device (100) can compare the exposure values ​​of the first image sensor and the second image sensor at different points in time. The electronic device (100) can obtain a target frame set (1910) including shooting frames (F1, F4, F6, F7) that are close to the target exposure value.

[0293] The electronic device (100) can obtain a long exposure frame (1980) by merging the shooting frames (F2, F4, F6, F8) included in the target frame set (1910). Compared to the long exposure frame (1880) of FIG. 18, the long exposure frame (1980) may reflect strong light relatively less.

[0294] FIG. 20 is a drawing for explaining motion analysis according to one embodiment.

[0295] Referring to FIG. 20, the electronic device (100) can analyze the motion of a target object in a target frame at a specific time.

[0296] The electronic device (100) can acquire a first target frame (2010) at a first time point. The electronic device (100) can acquire first inertial data at a first time point. The electronic device (100) can acquire a first inertial vector (IMU1) based on the first inertial data. The electronic device (100) can acquire first pixel data based on the first target frame (2010) at a first time point.

[0297] The electronic device (100) can acquire a second target frame (2020) at a second time point. The electronic device (100) can acquire second inertial data at a second time point. The electronic device (100) can acquire a second inertial vector (IMU2) based on the second inertial data. The electronic device (100) can acquire second pixel data based on the second target frame (2020) at a second time point.

[0298] The electronic device (100) can acquire a third target frame (2030) at a third time point. The electronic device (100) can acquire third inertial data at a third time point. The electronic device (100) can acquire a third inertial vector (IMU3) based on the third inertial data. The electronic device (100) can acquire third pixel data based on the third target frame (2030) at a third time point.

[0299] The electronic device (100) can acquire a fourth target frame (2040) at a fourth time point. The electronic device (100) can acquire fourth inertial data at a fourth time point. The electronic device (100) can acquire a fourth inertial vector (IMU4) based on the fourth inertial data. The electronic device (100) can acquire fourth pixel data based on the fourth target frame (2040) at a fourth time point.

[0300] The electronic device (100) can identify whether there is movement of the electronic device (100) by comparing inertia vectors (IMU1, IMU2, IMU3, IMU4).

[0301] For example, the magnitude of the first difference vector (DV1) between the first inertia vector (IMU1) and the second inertia vector (IMU2) may be less than or equal to a threshold size. The electronic device (100) may determine that there is no movement of the electronic device (100) at the second time point based on the first time point.

[0302] For example, the magnitude of the second difference vector (DV2) between the second inertia vector (IMU2) and the third inertia vector (IMU3) may be greater than a threshold size. The electronic device (100) may determine that there is movement of the electronic device (100) at the third point in time based on the second point in time.

[0303] For example, the magnitude of the third difference vector (DV3) between the third inertia vector (IMU3) and the fourth inertia vector (IMU4) may be greater than a threshold size. The electronic device (100) may determine that there is movement of the electronic device (100) at the fourth point in time based on the third point in time.

[0304] For example, the electronic device (100) can identify a target object (21, 22) based on at least one of the first pixel data, the second pixel data, the third pixel data, and the fourth pixel data. The electronic device (100) can identify the type of object based on the position change of the first object (21) and the second object (22). The electronic device (100) can identify whether the object is fixed or moving.

[0305] The electronic device (100) can identify a first motion vector (MV1) by comparing a first position of a first object (21) in first pixel data and a second position of a first object (21) in second pixel data.

[0306] The electronic device (100) can identify a second motion vector (MV2) by comparing a third position of the second object (22) in the first pixel data and a fourth position of the second object (22) in the second pixel data.

[0307] The electronic device (100) can analyze the correlation by comparing the first difference vector (DV1) and the first motion vector (MV1). Based on the correlation, the electronic device (100) can determine the first object (21) as a fixed type.

[0308] The electronic device (100) can analyze the correlation by comparing the first difference vector (DV1) and the second motion vector (MV2). Based on the correlation, the electronic device (100) can determine the second object (22) as a non-fixed type.

[0309] The electronic device (100) can identify a third motion vector (MV3) by comparing the second position of the first object (21) in the second pixel data and the fifth position of the first object (21) in the third pixel data.

[0310] The electronic device (100) can identify a fourth motion vector (MV4) by comparing the fourth position of the second object (22) in the second pixel data with the seventh position of the second object (22) in the third pixel data.

[0311] The electronic device (100) can analyze the correlation by comparing the second difference vector (DV2) and the third motion vector (MV3). Based on the correlation, the electronic device (100) can determine the first object (21) as a fixed type.

[0312] The electronic device (100) can analyze the correlation by comparing the second difference vector (DV2) and the fourth motion vector (MV4). Based on the correlation, the electronic device (100) can determine the second object (22) as a non-fixed type.

[0313] The electronic device (100) can identify a fifth motion vector (MV5) by comparing the fifth position of the first object (21) in the third pixel data and the sixth position of the first object (21) in the fourth pixel data.

[0314] The electronic device (100) can identify a sixth motion vector (MV6) by comparing the seventh position of the second object (22) in the third pixel data and the eighth position of the second object (22) in the fourth pixel data.

[0315] The electronic device (100) can analyze the correlation by comparing the third difference vector (DV3) and the fifth motion vector (MV5). Based on the correlation, the electronic device (100) can determine the first object (21) as a fixed type.

[0316] The electronic device (100) can analyze the correlation by comparing the third difference vector (DV3) and the sixth motion vector (MV6). Based on the correlation, the electronic device (100) can determine the second object (22) as a non-fixed type.

[0317] The electronic device (100) can generate a long exposure frame by merging a plurality of target frames (2010, 2020, 2030, 2040) considering the types of the first object (21) and the second object (22).

[0318] FIG. 21 is a drawing for explaining the operation of generating a long exposure frame through motion analysis according to one embodiment.

[0319] Referring to FIG. 21, the electronic device (100) can generate a long exposure frame based on a fixed object. The electronic device (100) can generate a long exposure frame by merging a plurality of target frames (2110, 2120, 2130, 2140) based on the position of the fixed object.

[0320] The electronic device (100) can identify the first object (21) as a fixed object. The electronic device (100) can obtain a first position of the first object (21) in a first target frame (2110). The electronic device (100) can obtain a second position of the first object (21) in a second target frame (2120). The electronic device (100) can obtain a fifth position of the first object (21) in a third target frame (2130). The electronic device (100) can obtain a sixth position of the first object (21) in a fourth target frame (2140).

[0321] The electronic device (100) can generate a long exposure frame by merging a plurality of target frames (2110, 2120, 2130, 2140) by considering the first position, second position, fifth position, and sixth position of the first object (21).

[0322] There can be various ways to generate long-exposure frames.

[0323] For example, an electronic device (100) can generate a long exposure frame (2180-1) by cropping overlapping areas from a plurality of target frames (2110, 2120, 2130, 2140). The long exposure frame (2180-1) may have uniform quality across all areas.

[0324] For example, the electronic device (100) can generate a long exposure frame (2180-2) by considering all regions in a plurality of target frames (2110, 2120, 2130, 2140). The electronic device (100) can generate a long exposure frame (2180-2) that includes both non-overlapping regions and overlapping regions. The quality of the long exposure frame (2180-2) may not be uniform due to the non-overlapping regions.

[0325] The first resolution of the long exposure frame (2180-1) and the second resolution of the long exposure frame (2180-2) may be different. The second resolution may be larger than the first resolution.

[0326] FIG. 22 is a diagram illustrating the operation of analyzing correlations according to one embodiment.

[0327] The mathematical formula (2210) of FIG. 22 can represent the process of calculating cosine similarity. The electronic device (100) can calculate the correlation value of two vectors using the mathematical formula (2210).

[0328] The table (2220) of FIG. 22 may represent analysis results corresponding to cosine similarity. For example, the cosine similarity may range from -1 to 1. -1 may indicate that the two vectors of the comparison subjects are in opposite directions. 1 may indicate that the two vectors of the comparison subjects are in the same direction.

[0329] When a cosine similarity of less than or equal to a threshold (th) is obtained, the electronic device (100) can identify that the two vectors are in opposite directions. The electronic device (100) can determine that the target object is a fixed object.

[0330] If a cosine similarity exceeding a threshold (th) is obtained, the electronic device (100) may determine that there is no correlation between the two vectors. The electronic device (100) may determine that the target object is a non-fixed object.

[0331] FIG. 23 is a drawing for explaining a screen showing a long exposure function according to one embodiment.

[0332] When a camera application is executed, the electronic device (100) can display a first screen (2310). The first screen (2310) may include a UI (2311) for selecting various functions related to the camera.

[0333] When user input selecting a UI (2311) is received, the electronic device (100) may display a second screen (2320). The second screen (2320) may include a UI (2321) for performing a long exposure function.

[0334] When user input selecting UI (2321) is received, the electronic device (100) can execute a long exposure mode to acquire a long exposure frame.

[0335] FIG. 24 is a drawing for explaining the operation of generating a long exposure frame according to one embodiment.

[0336] Referring to the embodiment (2400) of FIG. 24, the electronic device (100) may display a screen (2410) for performing a long exposure mode to acquire a long exposure frame. The screen (2410) may include a UI (2411) for starting shooting.

[0337] When user input selecting UI (2411) is received, the electronic device (100) can acquire a shooting frame to acquire a long exposure frame.

[0338] When the electronic device (100) begins to acquire a shooting frame, it may display a screen (2420) at a first point in time. The screen (2420) may include a UI (2421) indicating the progress of the long exposure mode. The UI (2421) may include an image or text (e.g., the number 4) indicating the remaining time of the long exposure mode.

[0339] The electronic device (100) can display a screen (2430) at a second point in time. The screen (2430) may include a UI (2431) indicating the progress of the long exposure mode. The UI (2431) may include an image or text (e.g., the number 3) indicating the remaining time of the long exposure mode.

[0340] The electronic device (100) can display a screen (2440) at a third point in time. The screen (2440) may include a UI (2441) indicating the progress of the long exposure mode. The UI (2441) may include an image or text (e.g., the number 2) indicating the remaining time of the long exposure mode.

[0341] The electronic device (100) can display a screen (2450) at a fourth point in time. The screen (2450) may include a UI (2451) indicating the progress of the long exposure mode. The UI (2451) may include an image or text (e.g., the number 1) indicating the remaining time of the long exposure mode.

[0342] When an event for terminating the long exposure function is identified, the electronic device (100) can generate a long exposure frame (2480) based on a plurality of acquired shooting frames.

[0343] FIG. 25 is a drawing for explaining a method of controlling an electronic device (100) according to one embodiment.

[0344] Referring to FIG. 25, a control method for an electronic device (100) including a plurality of image sensors including a first image sensor and a second image sensor comprises the steps of: identifying a first image sensor having a large aperture value among the plurality of image sensors (S2510); determining an exposure value of the plurality of image sensors (S2520); acquiring a plurality of shooting frames through the first image sensor and the second image sensor for a preset time (S2530); identifying a plurality of target frames among the plurality of shooting frames based on the exposure value (S2540); and generating a long exposure frame based on the plurality of target frames (S2550).

[0345] The control method includes a step of determining an exposure value (S2520) which determines a first exposure value corresponding to a first image sensor and a second exposure value corresponding to a second image sensor, and a step of acquiring a plurality of shooting frames (S2530) which acquires a first shooting frame with a first exposure value at a first time point through the first image sensor and acquires a second shooting frame with a second exposure value at a first time point through the second image sensor.

[0346] The control method includes a step of determining an exposure value (S2520), which determines a third exposure value corresponding to a first image sensor when a first shooting frame is acquired, and a step of acquiring a plurality of shooting frames (S2530), which acquires a third shooting frame with a third exposure value at a second time point through the first image sensor, and acquires a fourth shooting frame with a fourth exposure value at a second time point through the second image sensor, and a step of identifying a plurality of target frames (S2540), which determines the first shooting frame as the first target frame and determines one of the third shooting frame and the fourth shooting frame as the second target frame, and a step of generating a long exposure frame (S2550), which generates a long exposure frame by merging the first target frame and the second target frame.

[0347] The step of identifying multiple target frames (S2540) may determine a first exposure value as a target exposure value, obtain a first difference value which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value, and if the first difference value is less than a first threshold value, determine the third shooting frame as a second target frame.

[0348] The step of identifying multiple target frames (S2540) may obtain a second difference value, which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value, if the first difference value is greater than or equal to the first threshold value, and if the first difference value exceeds the second difference value, determine the fourth shooting frame as the second target frame.

[0349] The step of generating a long exposure frame (S2550) may generate a long exposure frame by acquiring a first inertia vector at a first time point through an inertia sensor, acquiring a second inertia vector at a second time point through an inertia sensor, acquiring a first difference vector between the first inertia vector and the second inertia vector, and if the first difference vector is less than a threshold size, merging the first target frame and the second target frame.

[0350] The first difference vector may be a vector representing the movement of the electronic device (100).

[0351] The step of generating a long exposure frame (S2550) can identify a target object included in a first target frame and a second target frame if the first difference vector is greater than or equal to a threshold size, identify a first position of the target object in the first target frame at a first time point, identify a second position of the target object in the second target frame at a second time point, obtain a first motion vector of the target object based on the first position and the second position, and generate a long exposure frame based on the correlation value of the first difference vector and the first motion vector.

[0352] A control method in which the first motion vector represents the movement of a target object.

[0353] The step of generating a long exposure frame (S2550) may determine the target object as a fixed object if the correlation value is less than or equal to the second threshold value, determine the target object as a non-fixed object if the correlation value exceeds the second threshold value, and generate a long exposure frame by merging the first target frame and the second target frame based on the position of the fixed object.

[0354] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0355] The methods according to the various embodiments of the present disclosure described above can be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0356] The various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through an external server among at least one of the electronic device and the display device.

[0357] According to a specific example of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0358] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0359] Each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be additionally included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the functions performed by each of the respective components prior to integration in the same or similar manner. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0360] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the scope of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit of the present disclosure.

Claims

1. In an electronic device, A plurality of image sensors including a first image sensor and a second image sensor having different aperture values; Memory for storing instructions; and at least one processor including processing circuitry; and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Receive user input for long exposure shooting, and A first set of shooting frames is acquired based on a first exposure value through the first image sensor during a preset time, and During the above-mentioned preset time, a second set of shooting frames is acquired based on a second exposure value through the second image sensor, and A first brightness change amount is obtained from the above first set of shooting frames, and A second brightness change amount is obtained from the above second shooting frame set, and If at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, a plurality of target frames are identified in the first shooting frame set and the second shooting frame set, and An electronic device that generates a long exposure frame by merging the above-mentioned plurality of target frames.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identifying the first image sensor having a large aperture value among the plurality of image sensors, and Determining the exposure values ​​of the above plurality of image sensors, and Acquiring a plurality of captured frames through the first image sensor and the second image sensor during the above-mentioned preset time, and An electronic device that identifies a plurality of target frames among a plurality of shooting frames based on the above exposure value.

3. In Paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, A first exposure value corresponding to the first image sensor is determined, and Determining a second exposure value corresponding to the second image sensor, and Through the first image sensor, a first shooting frame is acquired at a first time point with the first exposure value, and An electronic device that acquires a second shooting frame with the second exposure value at the first time point through the second image sensor.

4. In Paragraph 3, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, When a first shooting frame is acquired, a third exposure value corresponding to the first image sensor is determined, and When a second shooting frame is acquired, a fourth exposure value corresponding to the second image sensor is determined, and Through the first image sensor, a third shooting frame is acquired at the second time point with the third exposure value, and Through the second image sensor, a fourth shooting frame is acquired at the second time point with the fourth exposure value, and The above first shooting frame is determined as the first target frame, and One of the above third shooting frame and the above fourth shooting frame is determined as the above second target frame, and An electronic device that generates the long exposure frame by merging the first target frame and the second target frame.

5. In Paragraph 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, The above first exposure value is determined as the target exposure value, and Obtaining a first difference value which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value, and An electronic device that determines the third shooting frame as the second target frame when the first difference value is less than the first threshold value.

6. In Paragraph 5, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the first difference value is greater than or equal to the first threshold value, a second difference value is obtained, which is the absolute value of the value obtained by subtracting the target exposure value from the fourth exposure value, and An electronic device that determines the fourth shooting frame as the second target frame when the first difference value exceeds the second difference value.

7. In Paragraph 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identifying a fixed object based on the above plurality of captured frames, and An electronic device that generates the long exposure frame by merging the first target frame and the second target frame based on the fixed object.

8. In Paragraph 7, The above electronic device is, Includes an inertial sensor, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Through the inertial sensor above, a first inertial vector is obtained at the first time point, and Through the inertial sensor above, a second inertial vector is obtained at the second time point, and Obtain the first difference vector of the first inertia vector and the second inertia vector, and If the first difference vector is less than a threshold size, the first target frame and the second target frame are merged to generate the long exposure frame, and The above first difference vector is, An electronic device, which is a vector representing the movement of the above electronic device.

9. In Paragraph 8, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the first difference vector is greater than or equal to a threshold size, the target object included in the first target frame and the second target frame is identified, and Identifying the first position of the target object in the first target frame at the first time point, and Identifying the second position of the target object in the second target frame at the second time point, and Based on the first position and the second position, a first motion vector of the target object is obtained, and The long exposure frame is generated based on the correlation value of the first difference vector and the first motion vector, and The above first motion vector is, An electronic device that indicates the movement of the above target object.

10. In Paragraph 9, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the above correlation value is less than or equal to the second threshold, the target object is determined to be a fixed object, and If the above correlation value exceeds the above second threshold, the target object is determined as a non-fixed object, and An electronic device that generates the long exposure frame by merging the first target frame and the second target frame based on the position of the fixed object.

11. A method for controlling an electronic device comprising a plurality of image sensors, including a first image sensor and a second image sensor having different aperture values, wherein A step of receiving user input for long exposure shooting; A step of acquiring a first set of shooting frames based on a first exposure value through the first image sensor for a preset time; A step of acquiring a second set of shooting frames based on a second exposure value through the second image sensor during the above-mentioned preset time; A step of obtaining a first brightness change amount in the first set of shooting frames; A step of obtaining a second brightness change amount in the second set of shooting frames; If at least one of the first brightness change amount or the second brightness change amount is greater than or equal to a threshold value, a step of identifying a plurality of target frames in the first shooting frame set and the second shooting frame set; and A control method comprising the step of merging the above plurality of target frames to generate a long exposure frame.

12. In Paragraph 11, The above control method is, A step of identifying the first image sensor having a large aperture value among the plurality of image sensors; and The step of determining the exposure value of the plurality of image sensors; is included, The step of acquiring the first shooting frame set and the second shooting frame set is Acquiring a plurality of captured frames through the first image sensor and the second image sensor during the above-mentioned preset time, and The step of identifying the plurality of target frames above is, A control method for identifying a plurality of target frames among a plurality of shooting frames based on the above exposure value.

13. In Paragraph 12, The step of acquiring the first shooting frame set and the second shooting frame set is A first exposure value corresponding to the first image sensor is determined, and Determining a second exposure value corresponding to the second image sensor, and Through the first image sensor, a first shooting frame is acquired at a first time point with the first exposure value, and A control method for acquiring a second shooting frame with the second exposure value at the first time point through the second image sensor.

14. In Paragraph 13, The step of acquiring the first shooting frame set and the second shooting frame set is When a first shooting frame is acquired, a third exposure value corresponding to the first image sensor is determined, and When a second shooting frame is acquired, a fourth exposure value corresponding to the second image sensor is determined, and Through the first image sensor, a third shooting frame is acquired at the second time point with the third exposure value, and Through the second image sensor, a fourth shooting frame is acquired at the second time point with the fourth exposure value, and The step of identifying the plurality of target frames above is, The above first shooting frame is determined as the first target frame, and One of the above third shooting frame and the above fourth shooting frame is determined as the above second target frame, and The step of generating the above long exposure frame is, A control method for generating the long exposure frame by merging the first target frame and the second target frame.

15. In Paragraph 14, The step of identifying the plurality of target frames above is, The above first exposure value is determined as the target exposure value, and Obtaining a first difference value which is the absolute value of the value obtained by subtracting the target exposure value from the third exposure value, and A control method for determining the third shooting frame as the second target frame when the first difference value is less than the first threshold value.

Images