Multi-stage synthesis method of multiple frames of equal exposure images

Abstract

A multi-stage synthesis method for multiple frames of equal exposure images includes: capturing a first set of frames under an underexposure setting; selecting a first key frame from the first set of frames; aligning the first set of frames with the first key frame; merging the first set of frames into a first frame; capturing a second set of frames under an underexposure setting; selecting a second key frame from the second set of frames; aligning the second set of frames with the second key frame; merging the second set of frames into a second frame; selecting a main set of frames based on the first key frame and the second key frame; aligning the first key frame with the second key frame based on the main set of frames; and merging the first key frame and the second key frame based on the main set of frames.

Description

Technical Field

[0001] The present disclosure relates to burst frames, and is particularly designed to provide a two-stage merging method for burst image frames. Background Art

[0002] If noise is absent, the raw data captured by a conventional camera is roughly linear with lighting conditions, aperture size, shutter speed, and ISO. Considering a static scene with a fixed camera, the raw data captured is almost linear with integration time (shutter speed) and / or ISO. This linearity makes it possible to process multiple raw frames to preserve color appearance and dynamic range. Summary of the Invention

[0003] An example method for multi-stage synthesis of multiple frames of equal exposure images includes: capturing a first group of frames under an underexposure setting; selecting a first key frame from the first group of frames; aligning the first group of frames with the first key frame; merging the first group of frames into a first frame based on the first key frame; capturing a second group of frames under an underexposure setting; selecting a second key frame from the second group of frames; aligning the second group of frames with the second key frame; merging the second group of frames into a second frame based on the second key frame; selecting a key frame in a main group of frames consisting of the first frame and the second frame based on the first frame and the second frame; aligning the key frame in the main group of frames with the main group of frames based on the key frame in the main group of frames; and merging the key frame in the main group of frames with the main group of frames based on the key frame in the main group of frames.

[0004] In one embodiment, the multi-stage synthesis method of multiple frames of equal exposure images further includes: capturing a next first group of frames under the underexposure setting; indexing the first group of frames to the next first group of frames to generate an indexed first group of frames; selecting a next first key frame from the next first group of frames; aligning the next first group of frames with the next first key frame; merging the next first group of frames into a next first frame based on the next first key frame; capturing a next second group of frames under the underexposure setting; indexing the second group of frames to the next second group of frames to generate an indexed second group of frames; and selecting a next second key frame from the next second group of frames. ; aligning the next second group of frames with the next second key frame; merging the next second group of frames into a next second frame based on the next second key frame; reselecting a next key frame in a next main group of frames consisting of a next first frame and a next second frame based on the next first frame and the next second frame; aligning the next key frame in the next main group of frames with the reselected next main group frame based on the next key frame in the reselected next main group frame; and merging the next key frame in the reselected next main group frame with the reselected next main group frame based on the next key frame in the reselected next main group frame.

[0005] In one embodiment, the multi-stage synthesis method of multiple frames of equal exposure images also includes: setting the capture index of another first group of frames under the underexposure setting to N; and setting the capture index of another second group of frames under the underexposure setting to N.

[0006] In one embodiment, the multi-stage synthesis method of multiple frames of equal exposure images further includes: iteratively indexing the capture of another first group of frames under the underexposure setting as N; and iteratively indexing the capture of another second group of frames under the underexposure setting as N.

[0007] In one embodiment, the first set of frames and the second set of frames are original data.

[0008] In one embodiment, the first set of frames and the second set of frames are low dynamic range.

[0009] In one embodiment, the first key frame is based on the best frame in the first set of frames.

[0010] In one embodiment, the second key frame is based on the best frame in the second set of frames.

[0011] In one embodiment, the first set of frames and the second set of frames are captured using approximately equal exposure values.

[0012] In one embodiment, the first set of frames is cached.

[0013] In one embodiment, the second set of frames is buffered. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the attached figure:

[0015] Figure 1 is a first example system diagram according to one embodiment of the present disclosure;

[0016] Figure 2 is a second example system diagram according to one embodiment of the present disclosure;

[0017] Figure 3 is an example two-stage burst processing flow according to one embodiment of the present disclosure;

[0018] Figure 4 is an example method according to one embodiment of the present disclosure; and

[0019] Figure 5 is another example method according to one example of the present disclosure. DETAILED DESCRIPTION

[0020] The following examples are only intended to illustrate the application of the present device and method and are not intended to limit the scope of the present invention. Equivalent modifications to the present device and method should fall within the scope of the claims.

[0021] Throughout the following description and claims, certain terms are used to refer to specific system components. As will be appreciated by those skilled in the art, different groups may refer to components and / or methods by different names. This document is not intended to distinguish between components and / or methods that have different names but the same function.

[0022] In the following discussion and claims, the terms "including" and "comprising" are used in an open-ended fashion, and thus can be interpreted to mean "including, but not limited to... ." Furthermore, the terms "coupled" or "coupled" are intended to represent either an indirect or direct connection. Thus, if a first device connects to a second device, that connection may be a direct connection or an indirect connection via other devices and connections.

[0023] Figure 1An example hybrid computing system 100 is depicted that may be used to implement a neural network associated with the operations of one or more portions or steps of process 600. In this example, processors associated with the hybrid system include a field programmable gate array (FPGA) 122, a graphics processor unit (GPU) 120, and a central processing unit (CPU) 118.

[0024] The CPU 118, GPU 120, and FPGA 122 have the ability to provide a neural network. The CPU is a general-purpose processor that can perform many different functions. The versatility of the CPU results in the ability to perform a variety of different tasks. However, the CPU is limited in its ability to process multiple data streams, and its functionality relative to neural networks is limited. The GPU is a graphics processing unit that has many small processing cores that can process parallel tasks sequentially. The FPGA is a field programmable device that has the ability to be reconfigured and perform any function that can be programmed into the CPU or GPU in the form of hard-wired circuits. Because FPGAs are programmed in the form of circuits, they are many times faster than CPUs and significantly faster than GPUs.

[0025] The system may also include other types of processors, such as accelerated processing units (APUs), which include CPUs with on-chip GPU elements, and digital signal processors (DSPs), which are designed to perform high-speed digital data processing. Application-specific integrated circuits (ASICs) can also perform the hard-wired functions of FPGAs. However, the lead time for designing and producing ASICs is on the order of several quarters of a year, as opposed to the quick turnaround implementation time available in programming FPGAs.

[0026] A graphics processor unit 120, a central processing unit 118, and a field programmable gate array 122 are connected, and the graphics processor unit 120, the central processing unit 118, and the field programmable gate array 122 are connected to the memory interface and controller 112. The FPGA is connected to the memory interface via a programmable logic circuit to memory interconnect 130. This additional device is used because FPGAs operate at very high bandwidths and to minimize the circuitry used by the FPGA to perform memory tasks. The memory interface and controller 112 is additionally connected to the persistent storage disk 110, the system memory 114, and the read-only memory (ROM) 116.

[0027] Figure 1 The system can be used to program and train the FPGA. The GPU works well with unstructured data and can be used for training. Once the data is trained, a deterministic inference model can be found, and the CPU can program the FPGA with the model data determined by the GPU.

[0028] The memory interfaces and controllers are connected to a central interconnect 124, which is further connected to GPU 120, CPU 118, and FPGA 122. Central interconnect 124 is further connected to input and output interfaces 128 and network interface 126.

[0029] Figure 2 A second example hybrid computing system 200 is depicted that may be used to implement a neural network associated with the operations of one or more portions or steps of process 1000. In this example, processors associated with the hybrid system include a field programmable gate array (FPGA) 210 and a central processing unit (CPU) 220.

[0030] The FPGA is electrically connected to an FPGA controller 212, which is connected to a direct memory access (DMA) 218. The DMA is connected to an input buffer 214 and an output buffer 216, which are coupled to the FPGA to buffer data into and out of the FPGA, respectively. DMA 218 includes two first-in-first-out (FIFO) buffers, one for the host CPU and the other for the FPGA. The DMA allows data to be written to and read from the appropriate buffers.

[0031] On the CPU side of the DMA is a host switch 228, which transfers data and commands to and from the DMA. The DMA is also connected to an SDRAM controller 224, which allows data to be transferred from the CPU 220 to the FPGA and vice versa. The SDRAM controller is also connected to external SDRAM 226 and the CPU 220. The host switch 228 is connected to a peripheral device interface 230. A flash memory controller 222 controls persistent storage and is connected to the CPU 220.

[0032] Noise in raw data is roughly proportional to the signal level. Higher sensitivity (ISO) can result in higher noise, which can be compensated for by sensor characteristics. This can lead to a method for distinguishing between noise and moving objects. Longer integration times can cause motion blur.

[0033] Raw data undergoes little to no digital processing after being captured by the sensor or camera, thus allowing the original image information to be preserved, which facilitates possible post-processing and image adjustments.

[0034] The raw data may be captured by a digital single-lens reflex (DSLR) camera or a complementary metal oxide semiconductor (CMOS) image sensor built into most mobile phones.

[0035] Using a predetermined short exposure value (EV), F frames of short-exposure raw data can be captured. Image processing can include image alignment and merging in the raw domain. After alignment and merging, the output of these key steps can be preserved in its original format. Conventional image signal processors can be used for denoising. For high dynamic range (HDR) images, tone mapping or exposure fusion methods can be used.

[0036] The globally aligned frame can be represented as Im i (x), i = 1, 2, ..., F. The merging operation can be expressed as a weighted local change, as shown in formula (1):

[0037]

[0038] Among them, s i represents the local variation alignment parameter of the i-th frame. Weighting can increase the robustness in correcting global alignment errors caused by computational errors, local moving objects, or occlusions between frames.

[0039] The linearity of the original data and the noise distinguishability allow for accumulation, weighted summation, and the like. The accumulated image can be normalized to the local variation weight. Another global normalization factor K facilitates flexible use between dynamic range expansion and noise reduction. The merged image can be expressed as Equation (2):

[0040]

[0041] Since the burst images are captured with the same EV, the signals in each frame can be considered comparable. In most regions, the weights between frames are also comparable, so we can calculate the weights by taking W for all 1≤i≤F. i = 1 to simplify the analysis of K. Therefore, the merged image becomes formula (3):

[0042]

[0043] When the normalization factor K is 1, the merged image has the same signal level as each input image, which is essentially a denoised version of the input image; the equivalent EV value of the output is the same as the input burst image.

[0044] When the normalization factor K is less than 1, the maximum value of the merged image is approximately 1 / K times the maximum value of the input individual images. In one example, when K = 0.25, the dynamic range of the merged image is approximately 4 times that of the individual images. The product of K and F is typically greater than 1, otherwise a simple merged image may become a summation of images with a digital gain of 1 / (KF), which will increase the signal level without any practical benefit.

[0045] When the normalization factor K is equal to 1 / F, the simple merged image is the sum of the input images, which is similar to a longer exposure (F times the single short exposure), but the highlight details are preserved, that is, the dynamic range is extended by a factor of F. Compared with the single input image, the equivalent EV value is increased by about log2(F).

[0046] In burst raw image processing, a key frame may be first selected as a primary frame. This selection is typically based on the best frame in the burst of frames. In other examples, the selection may be based on the highest signal-to-noise ratio, maximum contrast, etc.

[0047] Burst processing of raw frames utilizes data linearity that allows signal levels to be scalable, simulating a longer exposure to extend dynamic range. Measurable noise levels improve the accuracy of the merging operation when distinguishing between noise and motion artifacts.

[0048] If there are F frames in the burst sequence, the first F-1 frames can be put into the buffer. When the F-th frame data is received, the main frame selection, alignment and merging can be realized.

[0049] F can be factorized as F = M × N, where M and N > 1. This allows the image sequence to be evenly grouped into M groups, each with N frames. For Group 1, a common burst processing unit can be used to generate denoised or high dynamic range (HDR) raw images and send them to the Stage 2 buffer. The buffer size in Group 1 is N-1 frames. After processing is completed for Group 1, the resulting raw data can be sent to the Stage 2 buffer, and the Stage 1 buffer can be released and made ready for use by Group 2. In this way, the Stage 1 buffer can be refreshed for each group.

[0050] When the denoised or high dynamic range (HDR) raw data in group M is stored, the M-1 frame is opened in the stage 2 buffer for use. The data can be input to the second stage processing unit, where the second processing flow is almost the same as the first stage. The input frame of the second stage is the output frame from the first stage, rather than the direct raw data captured by the camera, but has the same data structure. The main frame selection operation in the first stage can be based on the direct raw data captured by the camera, while the main frame selection operation in the second stage can be based on the output of the first stage. If the first stage processes burst frames exceeding the denoising range, thereby extending the dynamic range of the stage 1 output, the bit width of the second stage can be higher than the bit width of the first stage. The main group or frame selection, alignment and merging blocks can also operate with a higher bit width.

[0051] The total frame buffer for the two-stage framework is approximately M+N-2 frames. Figure 3 The burst raw frame structure is suitable for processing linear data with a measurable noise level.

[0052] The burst processing unit outputs data in the linear domain, which can be expressed by equation (2). The processing units share the same structure and control parameters, so the dynamic range of the data obtained in different groups in stage 1 can also be the same.

[0053] The noise level of the output in the first stage can be predicted. If the noise level or variance of the raw data of the original sensor is σ 2 , and the information from the N frames in the provided group is comparable, then the noise variance can be Nσ 2 If it is high dynamic range (HDR) processing, then KF in equation (3) = 1. If it is denoising processing, then K = 1 in equation (3), and the variance of the noise becomes If the method simultaneously denoises the data and outputs high dynamic range (HDR) data, then in equation (3) 1>K>1 / F, and the variance drops to approximately

[0054] If the total number of frames, F, is factored into two factors, M and N, where both M and N are greater than 1, the frame buffer can be reduced from F-1 = MN-1 frames to M+N-2 frames. Therefore, for integers M>1 and N>1, M+N-2 is less than MN-1. To make the most efficient use of the buffer, for a given value of F, the closer the values ​​of M and N are, the smaller the buffer.

[0055] The process can be extended to include additional stages. For example, if F is factored so that the factor values ​​are greater than 1, then F = N1N2...N k , where N i >1. The k-th stage process unit can process N k frames, and the frame buffer size will be Therefore, the buffer size can be further reduced compared to the two-stage processing.

[0056] Figure 3 Two stages are depicted, stage 1, 310, and stage 2, 312. In stage 1, the frames of group 1 are divided into frame 1, 314, frame 2, 316, frame N-1, 318, and frame N, 320. Frames 1, 2, and N-1 of group 1 are routed to a stage 1 buffer 346 containing N-1 frames. A primary frame selection module 354 receives frames from stage 1 buffer 346 and frame N, 320 from group 1. Primary frame selection module 354 selects a frame based on certain criteria, in this example, the best frame, and routes the data to an alignment and merging module 362, which outputs a denoised original or HDR frame. The frame selection module may use other selection criteria, such as highest signal-to-noise ratio, maximum contrast, etc.

[0057] Group 2 consists of frame 1 322, frame 2 324, frame N-1 326, and frame N 328. Frames 1, 2, and N-1 of group 2 are routed to stage 1 buffer 348, which has N-1 frames. A primary frame selection module 356 receives frames from stage 1 buffer 348 and frame N 328 from group 2. Primary frame selection module 356 selects the best frame and routes the data to alignment and merging module 364, which outputs a denoised or HDR original frame.

[0058] Group M-1 is divided into frame 1, 330, frame 2, 332, frame N-1, 334, and frame N, 336. Frames 1, 2, and N-1 of group 2 are routed to stage 1 buffer 350, which has N-1 frames. A primary frame selection module 358 receives frames from stage 1 buffer 350 and frame N, 336 from group M-1. Primary frame selection module 358 selects the best frame and routes the data to alignment and merging module 366, which outputs a denoised or HDR original frame.

[0059] Group M is divided into frame 1, 338, frame 2, 340, frame N-1, 342, and frame N, 344. Frames 1, 2, and N-1 of group M are routed to a stage 1 buffer 352 having N-1 frames. A primary frame selection module 360 ​​receives frames from the stage 1 buffer 352 and frame N, 344 from group M. The primary frame selection module 360 ​​selects the best frame and routes the data to an alignment and merging module 368, which outputs a denoised or HDR original frame.

[0060] A stage 2 buffer 370, consisting of M-1 frames, receives data from the alignment and merging module 362, the alignment and merging module 364, and the alignment and merging module 366. A primary group selection module 372 receives output from the stage 2 buffer 370 and the alignment and merging module 368. The primary group selection module 372 outputs to the stage 2 alignment and merging module 374, which outputs the final denoised or HDR raw data.

[0061] As the number of stages increases and the number of stages applied to HDR processing, the frame buffers in the later stages will have a higher bit width than the frame buffers in the beginning stages.

[0062] Figure 4An exemplary method for multi-stage composition of multiple frames of equally exposed images is depicted, the method comprising: capturing 410 a first set of frames at an underexposed setting; selecting 412 a first keyframe from the first set of frames; aligning 414 the first set of frames with the first keyframe; and merging 416 the first set of frames into a first frame based on the first keyframe. The exemplary method further comprises: capturing 418 a second set of frames at an underexposed setting; selecting 420 a second keyframe from the second set of frames; aligning 422 the second set of frames with a second keyframe; and merging 424 the second set of frames into a second frame based on the second keyframe. The exemplary method further comprises: selecting 426 a keyframe in a main set of frames consisting of the first frame and the second frame based on the first frame and the second frame; aligning 428 the keyframe in the main set of frames with the main set of frames based on the keyframe in the main set of frames; and merging 430 the keyframe in the main set of frames with the main set of frames based on the keyframe in the main set of frames.

[0063] Figure 5 Depicts Figure 4 The example method may further include: capturing 510 a next first set of frames at an underexposed setting; indexing 512 the first set of frames to the next first set of frames, thereby generating an indexed first set of frames; selecting 514 a next first key frame from the next first set of frames; aligning 516 the next first set of frames with the next first key frame; and merging 518 the next first set of frames into a next first frame based on the next first key frame. The example method may further include: capturing 520 a next second set of frames at an underexposed setting; indexing 522 the second set of frames to the next second set of frames, thereby generating an indexed second set of frames; selecting 524 a next second key frame from the next second set of frames; aligning 526 the next second set of frames with the next second key frame; and merging 528 the next second set of frames into a next second frame based on the next second key frame. The example method may also include reselecting 530 a next key frame in a next main group frame consisting of a next first frame and a next second frame based on the next first frame and the next second frame; aligning 532 the next key frame in the next main group frame with the reselected next main group frame based on the next key frame in the reselected next main group frame; and merging 534 the next key frame in the reselected next main group frame with the reselected next main group frame based on the next key frame in the reselected next main group frame.

[0064] The example method may also include indexing the capture of another first set of frames at the underexposed setting as N; and indexing the capture of another second set of frames at the underexposed setting as N.

[0065] The example method may also include iteratively indexing the capture of another first set of frames under the underexposure setting as N; and iteratively indexing the capture of another second set of frames under the underexposure setting as N. The first set of frames and the second set of frames may be composed of raw data and / or low dynamic range data. The first key frame may be based on a best frame within the first set of frames, and the second key frame may be based on a best frame within the second set of frames. The first set of frames and the second set of frames may be captured at approximately equal exposure values. The first set of frames and / or the second set of frames may be buffered.

[0066] It will be appreciated by those skilled in the art that the various illustrative blocks, modules, elements, parts, methods and algorithms described herein can be implemented as electronic hardware, computer software or a combination of the two. In order to illustrate this interchangeability of hardware and software, various illustrative blocks, modules, elements, components, methods and algorithms have been generally described above according to their functions. Whether such functions are implemented as hardware or software depends on specific application and the design constraints imposed on the system. Technicians can implement the described functions in different ways for each specific application. Without departing from the scope of this subject technology, various parts and blocks can be arranged differently (for example, arranged in different orders or divided in different ways).

[0067] It should be understood that the specific order or hierarchy of steps in the disclosed processes is illustrative of example methods. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the processes may be rearranged. Some steps may be performed simultaneously. The accompanying method claims present elements of the various steps in a sample order and are not meant to be limited to the specific order or hierarchy presented.

[0068] The previous description is provided so that any person skilled in the art can practice the various aspects described herein. The previous description provides various examples of the subject technology, and the subject technology is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, it is not intended that the claims be limited to the various aspects shown herein, but rather that the full scope be consistent with the language of the claims, wherein elements mentioned in the singular are not intended to mean "one and only one", unless clearly stated otherwise, but rather mean "one or more". Unless otherwise clearly stated, the term "some" refers to one or more. Masculine pronouns (e.g., his) include female and neutral genders (e.g., she and it), and vice versa. Titles and subtitles, if any, are only for convenience and do not limit the present invention. The predicates "configured to", "operable to", and "programmed to" do not mean that any specific tangible or intangible modification is performed on an object, but are intended to be used interchangeably. For example, a processor configured to monitor and control an operation or component may also mean that the processor is programmed to monitor and control the operation or the processor can be operated to monitor and control the operation. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or can be operated to execute code.

[0069] Phrases such as “aspects” do not imply that such aspects are essential to the subject technology or that such aspects apply to configurations of the subject technology. Disclosure relating to an aspect may apply to a configuration, or one or more configurations. An aspect may provide one or more examples. Phrases such as “aspects” may refer to one or more aspects, and vice versa. Phrases such as “embodiments” do not imply that such embodiments are essential to the subject technology or that such embodiments apply to configurations of the subject technology. Disclosure relating to an embodiment may apply to an embodiment, or one or more embodiments. An embodiment may provide one or more examples. Phrases such as “embodiments” may refer to one or more embodiments, and vice versa. Phrases such as “configurations” do not imply that such configurations are essential to the subject technology or that such configurations apply to configurations of the subject technology. Disclosure relating to a configuration may apply to a configuration, or one or more configurations. A configuration may provide one or more examples. Phrases such as “configurations” may refer to one or more configurations, and vice versa.

[0070] The word “exemplary” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0071] It is known or will be known to those of ordinary skill in the art that structural and functional equivalents of the elements of the various aspects described throughout this disclosure are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, regardless of whether the disclosure described herein is explicitly stated in the claims, it is not intended that such disclosure be donated to the public. Unless the phrase "means for..." is used to expressly state the element, or in the case of a method claim, the phrase "step for..." is used to state the element, the elements of any claim shall not be interpreted according to the provisions of 35 U.S.C., §112, paragraph 6 of the United States Patent Act. In addition, to the extent that the terms "include", "have", etc. are used in the specification or claims, such terms are intended to be included in the manner of the term "comprising", similar to the interpretation of "comprising" when "comprising" is used as a conjunction in the claims.

[0072] References to "one embodiment," "an embodiment," "some embodiments," "various embodiments," etc. indicate that specific elements or features are included in at least one embodiment of the present invention. Although these phrases may appear in various places, they do not necessarily refer to the same embodiment. With this disclosure, those skilled in the art will be able to design and incorporate any of a variety of mechanisms suitable for implementing the aforementioned functions.

[0073] It should be understood that this disclosure teaches only one example of an illustrative embodiment and that those skilled in the art may readily devise many variations of the invention after reading this disclosure and that the scope of the invention will be determined by the following claims.

Claims

1. A multi-stage synthesis method for multiple frames of equal-exposure burst images, the method comprising: capturing a first set of frames of the burst image at an underexposed setting; selecting a first key frame from the first set of frames; aligning the first set of frames with the first keyframe; Based on the first key frame, merging the first group of frames into a first frame; capturing a second set of frames of the burst image at the underexposed setting; selecting a second key frame from the second set of frames; aligning the second set of frames with the second key frame; Based on the second key frame, merging the second group of frames into a second frame; selecting, based on the first frame and the second frame, a key frame in a main group of frames consisting of the first frame and the second frame; Based on the key frames in the main group of frames, aligning the key frames in the main group of frames with the main group of frames; and Based on the key frames in the main group of frames, the key frames in the main group of frames and the main group of frames are merged.

2. The method according to claim 1, further comprising: capturing a next first set of frames at the underexposed setting; indexing the first set of frames into the next first set of frames, thereby generating an indexed first set of frames; selecting a next first key frame from the next first set of frames; aligning the next first set of frames with the next first key frame; Based on the next first key frame, merging the next first group of frames into a next first frame; capturing a second set of frames at the underexposed setting; indexing the second set of frames to the next second set of frames, thereby generating an indexed second set of frames; selecting a next second key frame from the next second set of frames; aligning the next second set of frames with the next second key frame; Based on the next second key frame, merging the next second group of frames into a next second frame; reselecting, based on the next first frame and the next second frame, a next key frame in a next main group of frames consisting of the next first frame and the next second frame; Based on the reselected next key frame in the next main group frame, aligning the next key frame in the next main group frame with the reselected next main group frame; and Based on the reselected next key frame in the next main group frame, the next key frame in the reselected next main group frame is merged with the reselected next main group frame.

3. The method according to claim 1, further comprising: indexing the capture of another first set of frames at the underexposure setting as N; as well as The capture of another second set of frames at the underexposed setting is indexed as N.

4. The method according to claim 1, further comprising: iteratively indexing the capture of another first set of frames at the underexposed setting as N; as well as The capture of another second set of frames at the underexposed setting is iteratively indexed as N.

5. The method according to claim 1, wherein The first set of frames and the second set of frames are original data.

6. The method according to claim 1, wherein The first set of frames and the second set of frames are low dynamic range.

7. The method according to claim 1, wherein The first key frame is based on a best frame in the first set of frames.

8. The method according to claim 1, wherein The second key frame is based on a best frame in the second set of frames.

9. The method according to claim 1, wherein The first set of frames and the second set of frames are captured using equal exposure values.

10. The method according to claim 1, wherein The first set of frames is cached.

11. The method according to claim 1, wherein The second set of frames is buffered.

Images