High dynamic range technique selection for image processing.

JP2024502938A5Pending Publication Date: 2024-12-09QUALCOMM INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023536161
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-01-14
Filing Date
2021-12-16
Publication Date
2024-12-09

AI Technical Summary

Technical Problem

Existing image capture devices face challenges in capturing high dynamic range (HDR) images due to issues such as ghosting effects and varying signal-to-noise ratios (SNR), particularly when dealing with scenes that have both very dark and very bright areas, as traditional methods may not optimize for both high SNR and minimal ghosting.

Method used

The solution involves analyzing a preview image's histogram to determine if it is an HDR scene and using appropriate image capture techniques based on motion detection within the scene, such as single-frame or multi-frame HDR techniques, to minimize ghosting and maximize SNR, including methods like in-sensor Quad-Bayer Coding (QBC) HDR and multi-frame HDR (MFHDR).

Benefits of technology

This approach allows for the capture of HDR images with high SNR and minimal ghosting artifacts by dynamically selecting the most suitable image capture technique based on scene analysis, ensuring optimal image quality in diverse lighting conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The device for camera processing is configured to receive a preview image and determine whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image. The device may further determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene, and may determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected within the preview image. The device may then generate an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This application claims priority to U.S. patent application Ser. No. 17 / 149,371, filed Jan. 14, 2021, the entire contents of which are incorporated herein by reference.

[0002] FIELD OF THE DISCLOSURE This disclosure relates to image capture and processing. [Background technology]

[0003] Capture devices are incorporated into a variety of devices. In this disclosure, a capture device refers to any device capable of capturing one or more digital images, including devices capable of capturing still images and devices capable of capturing a series of images to record video. By way of example, a capture device may comprise a standalone digital camera or digital video camcorder, a wireless communication device handset with a camera, such as a mobile phone, cellular or satellite radio telephone, a personal digital assistant (PDA) with a camera, a panel or tablet, a gaming device, a computing device including a camera, such as a so-called "webcam," or any device with digital imaging or video capabilities.

[0004] Images captured by digital cameras are most commonly low dynamic range (LDR) images (sometimes called standard dynamic range or SDR). Images captured in LDR / SDR may contain loss of detail in bright or dark areas of the picture in some lighting situations, depending on the exposure settings of the camera. High dynamic range (HDR) images generally contain more possible values ​​of brightness (e.g., luminance) and therefore can more accurately represent dark areas (e.g., shadows) and well-lit areas (e.g., sunlight). In some examples, HDR images may be generated by combining multiple LDR images with different exposure settings.

[0005] The capture device may include multiple image sensors and / or multiple lenses that may be used to support various imaging techniques, such as high dynamic range (HDR) imaging, multi-frame HDR (MFHDR) imaging, etc. The image sensor may transfer image data to a camera processor. Exemplary lens types include a wide-angle lens, an ultra-wide-angle lens, a telephoto lens, a telescope lens, a periscope-style zoom lens, a fisheye lens, a macro lens, a prime lens, or various combinations thereof. As an example, a capture device having a dual-camera configuration may include a wide-angle lens and a telephoto lens. Similarly, a triple-camera configuration may include an ultra-wide-angle lens in addition to a wide-angle lens and a telephoto lens. By using multiple lenses and / or image sensors, the camera processor of the capture device may capture images with various fields of view (FOV), adjust zoom levels, apply focusing effects, and / or create one or more composite frames of image data (e.g., an image generated from a combination of multiple images). Summary of the Invention [Means for solving the problem]

[0006] Generally, this disclosure describes camera processing techniques related to high dynamic range (HDR) imaging. A capture device may be configured to capture one or more images using one of a number of image capture techniques, including HDR image capture techniques. The various image capture techniques available to a capture device may be more or less susceptible to ghosting effects. Ghosting effects generally occur in multi-frame image capture techniques in which multiple images are captured and fused together. Because the multiple images are captured over a period of time, some objects in the scene may not be in the same location in different captured images if there are objects in the scene that are moving quickly relative to the time delay in capturing the multiple images. This situation may cause motion blur or "ghosting" in an image formed by fusing the captured multiple images.

[0007] In addition, various image capture techniques, including HDR image capture techniques, available to a capture device may exhibit higher or lower signal-to-noise ratios (SNRs) based on the characteristics of the scene being captured. For example, a binning sensor, such as a Quad-Bayer color filter array (QCFA) sensor, may be configured to average the values ​​of four adjacent color sensors to generate a final output pixel. In binning mode, the QCFA sensor generally has a higher SNR when compared to a non-binning sensor, especially in low light environments. However, in some HDR image capture modes, the QCFA sensor does not use the full binning amount available and may therefore generate images with a lower SNR for some scenes.

[0008] In view of these issues, the present disclosure describes a technique for determining an image capture technique from among a plurality of image capture techniques. In one example, a capture device may analyze a histogram of luminance values ​​of pixels of a preview image to determine whether a scene being captured is or is not an HDR scene. Generally, an HDR scene will have a relatively high percentage of luminance values ​​in both unsaturated (e.g., very dark) and oversaturated (e.g., very bright) regions of the histogram. If the capture device determines that the preview image is not an HDR scene, the capture device may be configured to capture a subsequent image using a single-frame image capture technique (e.g., single binned frame). Because only a single image is captured at a time (e.g., image fusion is not performed), the use of a single-frame image capture technique greatly reduces and / or eliminates the possibility of ghosting in an image. In addition, the use of a single-frame image capture technique (e.g., binned frame) generally results in a high SNR in the resulting image.

[0009] If the capture device determines that the preview image is an HDR scene, the capture device may then determine to use one of a number of HDR capture techniques based on the motion detected in the preview image and, if motion is present, the location of the motion in the image. For example, if the capture device determines that the scene is an HDR scene (e.g., based on the histogram described above) and also determines that there is no motion in the scene, the capture device may determine to use an HDR image capture technique that captures and fuses together multiple frames (e.g., MFHDR). Although MFHDR is susceptible to ghosting, MFHDR generally produces images with a high SNR. However, because the capture device has determined that there is no motion detected in the preview image, MFHDR may be used due to the low probability of ghosting.

[0010] If the capture device determines (e.g., based on the histogram described above) that the scene is an HDR scene and also determines that motion is present in the scene, the capture device may further determine whether the motion is in a dark area of ​​the image. If the motion is not in a dark area (e.g., the motion is in a bright area of ​​the image), the capture device may choose to use a single-frame HDR technique. A single-frame HDR technique may be less susceptible to ghosting but generally result in a lower SNR. If the motion is in a dark area, the capture device may choose to fuse an image from the single-frame HDR technique with another image captured using a binning mode. Fusing a single-frame HDR image with a binning frame may result in a high SNR while being less susceptible to ghosting in bright areas. Because the motion was in a dark area of ​​the image, ghosting effects are less likely to be present in the image. In general, the techniques of the present disclosure may result in a more optimal selection of image capture techniques such that HDR scenes are captured with a high SNR and with minimal to no ghosting.

[0011] In one example, techniques of the present disclosure are directed to a method that includes receiving a preview image; determining whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image; determining to use a single frame image capture technique based on a determination that the preview image is not an HDR scene; determining to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image; and generating an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0012] In another example, techniques of the present disclosure are directed to a device configured for camera processing, the device having a memory configured to receive a preview image, determine whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image, determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene, determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image, and generate an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0013] The present disclosure also describes means for performing any of the techniques described herein. In one example, the techniques of the present disclosure are directed to an apparatus configured to perform camera processing, the apparatus comprising: means for receiving a preview image; means for determining whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image; means for determining to use a single frame image capture technique based on a determination that the preview image is not an HDR scene; means for determining to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image; and means for generating an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0014] In another example, the disclosure describes a non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to receive a preview image; determine whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image; determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene; determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image; and generate an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0015] This summary is intended to provide an overview of the subject matter described in this disclosure. This summary is not intended to provide an exclusive or exhaustive description of the systems, devices, and methods described in detail in the attached drawings and description herein. Further details of one or more examples of the disclosed technology are described in the attached drawings and the following description. Other features, objects, and advantages of the disclosed technology will be apparent from the description, drawings, and claims. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram of a device configured to perform one or more of the example techniques described in this disclosure. [Diagram 2] FIG. 2 is a conceptual diagram illustrating an example image sensor and image capture modes. [Diagram 3] 2 is a block diagram illustrating example components of the computing device of FIG. 1 in greater detail. [Figure 4] 4 is a block diagram illustrating example components of the computing device of FIG. 3 in greater detail. [Diagram 5]1 is a flowchart illustrating an exemplary method of the present disclosure. [Figure 6] FIG. 2 is a conceptual diagram illustrating an example motion map. [Figure 7] FIG. 1 is a conceptual diagram illustrating an exemplary luminance histogram. [Figure 8] 2 is a block diagram illustrating example components of the computing device of FIG. 1 in greater detail for HDR and binning frame image capture techniques. [Figure 9] 9 is a conceptual diagram illustrating example fusion weights for use with the technique of FIG. 8. [Figure 10] 5 is a flowchart illustrating an example operation of camera processing in accordance with an example technique of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Like reference numbers refer to like elements throughout the present specification and figures.

[0018] High dynamic range (HDR) imaging includes image capture techniques used to reproduce images with a wider dynamic range of brightness values ​​within an image. HDR imaging can be particularly useful for scenes that contain both very dark and very bright areas. Traditional single frame image capture modes may not be able to preserve detail in both very bright and very dark areas.

[0019] In most imaging devices, the exposure of light applied to the image sensor can be changed in two ways: either by increasing / decreasing the size of the aperture, or by increasing / decreasing the time of each exposure (e.g., by adjusting the shutter speed). Due to the physical characteristics of the camera sensor, it may not be possible to capture all of the luminance values ​​present in a scene having a very wide range of luminance values ​​(e.g., a scene exhibiting a high dynamic range). That is, the camera sensor may not be able to record all of the luminance values ​​in the scene that are perceptible by the human eye.

[0020] For example, to preserve detail in the darker areas of a scene, the camera may reduce the shutter speed and / or increase the aperture size. However, this will cause the brighter areas to be overexposed (e.g., the image sensor output will output a maximum luminance value for many different actual luminance values), and detail in the brighter areas will be lost. To preserve detail in the brighter areas, the camera may increase the shutter speed and / or decrease the aperture size. However, this will cause the darker areas to be underexposed (e.g., the image sensor output will output a minimum luminance value for many different actual luminance values), and detail in these darker areas will be lost.

[0021] In some examples, the camera device may be configured to generate an HDR image by capturing multiple images with different exposure settings and then combining the multiple images to form a final image that more accurately represents all of the luminance values ​​present in the scene. This image capture technique may be referred to as multi-frame HDR (MFHDR). In some examples, exposure changes for HDR generation are only made by changing the exposure time, not the aperture size. This is because changing the aperture size may also affect the depth of field of each of the output images, which may be undesirable in some applications. In other applications, the camera may adjust both the aperture and the shutter speed to output multiple images that are used to form the HDR image. The camera may also be configured to capture the HDR image using a single-frame intra-sensor HDR technique to generate a wider dynamic range image and preserve highlight details in the bright and dark areas of the scene. Generally, MFHDR may generate images with a higher signal-to-noise ratio (SNR), but may be more susceptible to ghosting (e.g., motion blur) by capturing multiple consecutive images over a period of time. In-sensor single-frame HDR techniques are less susceptible to ghosting, but may produce images with a lower SNR compared to MFHDR techniques.

[0022] As can be seen from the above examples, using a single HDR image capture technique may not be appropriate for all HDR scenes. Some HDR image capture techniques may be susceptible to ghosting, while other HDR image capture techniques may be free of ghosting but produce images with a lower SNR than desired. In light of these shortcomings, the present disclosure describes techniques related to analyzing a scene from a preview image and determining one of a plurality of HDR image capture techniques to use to capture and generate an HDR image based on the analysis.

[0023] The camera device may be configured to determine an HDR image capture technique to use based on one or more of a histogram of brightness values ​​in the preview image (e.g., to identify an HDR scene), motion detected in the preview image, and / or a particular area of ​​the preview image (e.g., a dark area or a light area) in which image motion is detected. In this manner, the techniques of this disclosure may enable the camera device to select an HDR image capture technique that will produce an image with a high SNR with a low ghosting artifact probability.

[0024] 1 is a block diagram of a computing device 10 configured to execute one or more of the exemplary techniques described in this disclosure for determining HDR image capture techniques. Examples of computing devices 10 include computers (e.g., personal computers, desktop computers, or laptop computers), mobile devices such as tablet computers, wireless communication devices (e.g., mobile phones, cellular phones, satellite phones, and / or mobile phone handsets, etc.), Internet phones, handheld devices such as digital cameras, digital video recorders, portable video game devices or personal digital assistants (PDAs), drone devices, or any device that may include one or more cameras. In some examples, computing device 10 may include one or more camera processors 14, a central processing unit (CPU) 16, a video encoder / decoder 17, a graphics processing unit (GPU) 18, a local memory 20 of GPU 18, a user interface 22, a memory controller 24 providing access to system memory 30, and a display interface 26 that outputs signals that cause graphical data to be displayed on a display 28.

[0025] 1, computing device 10 includes one or more image sensors 12A-N. Image sensors 12A-N may be referred to in some instances herein simply as "sensor 12," but in other instances may be referred to as a plurality of "sensors 12" where appropriate. Sensor 12 may be any type of image sensor, including a sensor including a Bayer filter, or an HDR interlaced sensor, such as a Quad-Bayer sensor or a Quad-Bayer color filter array (QCFA) sensor.

[0026] Computing device 10 further includes one or more lenses 13A-N. Similarly, lenses 13A-N may be referred to simply as "lenses 13" in some instances herein, but may be referred to as multiple "lenses 13" in other instances, where appropriate. In some examples, sensor 12 represents one or more image sensors 12, each of which may include processing circuitry, an array of pixel sensors (e.g., pixels) for capturing representations of light, memory such as buffer memory or on-chip sensor memory, etc. In some examples, each of image sensors 12 may be coupled to a different type of lens 13, with each lens and image sensor combination having a different aperture and / or field of view. Exemplary lenses may include telephoto lenses, wide-angle lenses, ultra-wide-angle lenses, or other lens types.

[0027] As shown in FIG. 1, the computing device 10 includes multiple cameras 15. The term "camera" as used herein refers to a particular image sensor 12 of the computing device 10, or multiple image sensors 12 of the computing device 10, where the image sensor 12 is disposed in combination with one or more lenses 13 of the computing device 10. That is, a first camera 15 of the computing device 10 refers to a first collective device including one or more image sensors 12 and one or more lenses 13, and a second camera 15 separate from the first camera 15 refers to a second collective device including one or more image sensors 12 and one or more lenses 13. In addition, image data may be received from the image sensor 12 of a particular camera 15 by the camera processor 14 or CPU 16. That is, the camera processor 14 or CPU 16 may, in some examples, receive a first set of frames of image data from the first image sensor 12 of the first camera 15 and receive a second set of frames of image data from the second image sensor 12 of the second camera 15.

[0028] In one example, the term "camera" as used herein refers to a combined image sensor 12 and lens 13 coupled together configured to capture at least one frame of image data and transfer the at least one frame of image data to the camera processor 14 and / or CPU 16. In one illustrative example, a first camera 15 is configured to transfer a first frame of image data to the camera processor 14 and a second camera 15 is configured to transfer a second frame of image data to the camera processor 14, where the two frames are captured by different cameras, as may be evidenced, for example, by a difference in FOV and / or zoom level of the first and second frames. The difference in FOV and / or zoom level may correspond to a difference in focal length between the first and second cameras 15 and 15.

[0029] Computing device 10 may include a dual lens device, a triple lens device, a 360 degree camera lens device, etc. As such, each lens 13 and image sensor 12 combination may provide various zoom levels, angles of view (AOV), focal lengths, fields of view (FOV), etc. In some examples, a particular image sensor 12 may be assigned to each lens 13, or vice versa. For example, multiple image sensors 12 may each be assigned to a different lens type (e.g., a wide angle lens, an ultra wide angle lens, a telephoto lens, and / or a periscope lens, etc.).

[0030] The camera processor 14 may be configured to control the operation of the camera 15 and perform processing on images received from the camera 15. In some examples, the camera processor 14 may include an image signal processor (ISP) 23. For example, the camera processor 14 may include circuitry for processing image data. The camera processor 14, including the ISP 23, may be configured to perform various operations, including auto white balance, color correction, or other post-processing operations, on image data captured by the image sensor 12. FIG. 1 shows a single ISP 23 configured to operate on the output of the camera 15. In other examples, the camera processor 14 may include an ISP 23 for each of the cameras 15 to increase processing speed and / or improve synchronization for simultaneous image capture from multiple cameras of the camera 15.

[0031] In some examples, camera processor 14 may execute "3A" algorithms. Such algorithms may include autofocus (AF), auto exposure control (AEC), and auto white balance (AWB) techniques. In such examples, 3A may represent the functionality of a statistical algorithm processing engine, and one of camera processors 14 may implement and operate such a processing engine.

[0032] In some examples, camera processor 14 is configured to receive image frames (e.g., pixel data) from image sensor 12 and process the image frames to generate image and / or video content. For example, image sensor 12 may be configured to capture individual frames, HDR frames, frame bursts, a sequence of frames for generating video content, still photos captured while recording a video, preview frames, or motion photos before and / or after capture of a still photo. CPU 16, GPU 18, camera processor 14, or some other circuitry may be configured to process the images and / or video content captured by sensor 12 into images or videos for display on display 28. An image frame may generally refer to a frame of data for a still image or frame of video data, or a combination thereof with a motion photo, etc. Camera processor 14 may receive the pixel data of the image frame from sensor 12 in any format. For example, the pixel data may include different color formats, such as RGB, YCbCr, YUV, etc. In either case, camera processor 14 may receive multiple frames of image data from image sensor 12.

[0033] In examples including multiple camera processors 14, the camera processors 14 may share the sensors 12, in which case the camera processors 14 may each interface with each of the sensors 12. In either case, the camera processors 14 may initiate capture of a video or image of a scene using the multiple pixel sensors of the sensors 12. In some examples, a video may include a sequence of individual frames. Thus, the camera processors 14 cause the sensors 12 to capture images using the multiple pixel sensors.

[0034] The sensor 12 may then output pixel information (e.g., pixel values, luma values, color values, charge values, analog digital unit (ADU) values, etc.) to the camera processor 14, where the pixel information represents a captured image or a sequence of captured images. In some examples, the camera processor 14 may process the monochrome and / or color images to obtain an extended color image of the scene. In some examples, the camera processor 14 may determine a universal blending weight factor for different types of pixel blending, or may determine different blending weight factors for blending different types of pixels that make up a frame of pixels (e.g., a first blending weight factor for blending pixels acquired via a monochrome sensor of the first camera 15 with pixels acquired via a monochrome sensor of the second camera 15, a second blending weight factor for blending pixels acquired via a Bayer sensor of the first camera 15 with pixels acquired via a Bayer sensor of the second camera 15, etc.).

[0035] In certain examples of the present disclosure, camera processor 14 may cause a particular image sensor of image sensor 12 to capture one or more images in HDR mode. In some examples, camera processor 14 and / or camera 15 may be configured to capture and generate HDR images according to one of a number of HDR capture techniques.

[0036] In one example, camera processor 14 may cause image sensor 12 to capture multiple (e.g., three) images with different exposure settings (e.g., long exposure, medium exposure, and short exposure). Such techniques may be referred to as multi-frame HDR (MFHDR). In one example, camera processor 14 may be configured to cause image sensor 12 to generate multiple images with different shutter speeds but the same aperture size. Image sensor 12 may output multiple images with a particular field of view (FOV) and at a particular resolution based on, for example, a zoom setting and the type of lens selected (e.g., a telephoto lens, a wide-angle lens, an ultra-wide-angle lens, etc.). ISP 23 may receive the multiple images and, in addition to any 3A or other processing, may perform MFHDR processing techniques to combine the multiple images into an HDR image.

[0037] In another example, camera processor 14 may generate an HDR image using a single frame output from one of cameras 15. Single frame HDR may include the use of single image histogram equalization post processing, a technique sometimes referred to as local tone mapping (LTM). In another example, as described below, camera processor 14 may instruct one of cameras 15 to capture a single frame HDR image using an in-sensor Quad-Bayer Coding (QBC) HDR technique. The in-sensor QBC HDR technique may include capturing a single image, where a first portion of pixel sensors of image sensor 12 captures light with a long exposure, a second portion of pixel sensors of image sensor 12 captures light with a medium time exposure, and a third portion of pixel sensors of image sensor 12 captures light with a short time exposure.

[0038] In another example, camera processor 14 may use binning frames in conjunction with adaptive dynamic range coding (ADRC) to generate an HDR image from the output of one of cameras 15. When performing ADRC, camera processor 14 may preserve high brightness areas of the image by capturing the image at a low sensor exposure. Camera processor 14 may apply digital gain to the output image to compensate for overall brightness.

[0039] Another exemplary technique for generating an HDR image is a staggered multi-frame HDR image capture technique. In the staggered multi-frame HDR image capture technique, the camera processor 14 may be configured to cause the camera 15 to capture multiple images using multiple rolling shutters. A rolling shutter does not generate an image from the entire scene at one time instance, but rather generates an image by scanning either a row or a column of the image sensor 12. For example, the camera 15 may combine a long-term rolling shutter capture with a short-term rolling shutter capture. The camera 15 may row-interleave the output of two rolling shutters.

[0040] Various image capture techniques available for computing device 10 may be somewhat susceptible to ghosting effects. Ghosting effects commonly occur in HDR image capture techniques where multiple images are captured and fused together (e.g., MFHDR). Because the multiple images are captured over a period of time, some objects in the scene may not be in the same location in the different captured images if there are objects in the scene that are moving quickly relative to the time delay in capturing the multiple images. This situation may cause motion blur or "ghosting" in the image formed by fusing the captured images.

[0041] In addition, various image capture techniques available to computing device 10, including HDR image capture techniques, may exhibit higher or lower signal-to-noise ratios (SNRs) based on the characteristics of the scene being captured. For example, one of image sensors 12 may be a binning sensor, such as a Quad-Bayer Color Filter Array (QCFA) sensor. The QCFA sensor may be configured to average values ​​of four adjacent color sensors to generate a final output pixel. In binning modes, the QCFA sensor generally has a high SNR when compared to non-binning sensors, especially in low light environments. However, in some HDR image capture modes, the QCFA sensor does not use the full binning amount available and therefore may generate images with a lower SNR for some scenes.

[0042] In view of these issues, this disclosure describes a technique for determining an image capture technique from among a plurality of image capture techniques. In one example, HDR selector 19 may analyze a histogram of luminance values ​​of pixels of a preview image captured by one of image sensors 12 to determine whether the scene being captured is an HDR scene or not. Generally, an HDR scene will have a relatively high percentage of luminance values ​​in both unsaturated (e.g., very dark) and oversaturated (e.g., very bright) regions of the histogram. When HDR selector 19 determines that the preview image is not an HDR scene, camera processor 14 may be configured to capture a subsequent image using a single-frame image capture technique (e.g., single binned frame). Because only a single image is captured at a time (e.g., image fusion is not performed), the use of a single-frame image capture technique greatly reduces and / or eliminates the possibility of ghosting in an image. In addition, the use of a single-frame image capture technique (e.g., binned frame) generally results in a high SNR in the resulting image.

[0043] If the HDR selector 19 determines that the preview image is an HDR scene, the HDR selector 19 may determine to use one of a plurality of HDR capture techniques based on motion detected in the preview image, and if there is motion detected, the HDR selector 19 may further determine the HDR capture technique based on the location of the motion in the preview image. For example, if the HDR selector 19 determines that the scene is an HDR scene (e.g., based on the histogram described above) and also determines that there is no motion in the scene, the HDR selector 19 may determine to use an HDR image capture technique that captures and fuses multiple frames together (e.g., MFHDR). Although MFHDR is susceptible to ghosting, MFHDR generally produces images with a high SNR. However, because the HDR selector 19 has determined that there is no motion detected in the preview image, MFHDR may be used due to the low probability of ghosting.

[0044] If the HDR selector 19 determines (e.g., based on the histogram described above) that the scene is an HDR scene and also determines that motion is present in the scene, the HDR selector 19 may further determine whether the motion is present in a dark area of ​​the image. If the motion is not present in a dark area (e.g., the motion is in a light area of ​​the image), the HDR selector 19 may select to use a single-frame HDR technique. A single-frame HDR technique may be less susceptible to ghosting, but generally result in a lower SNR. If the motion is in a dark area, the HDR selector 19 may select to blend an image from the single-frame HDR technique with another image captured using a binning mode. Blending a single-frame HDR image with a binning frame may result in a high SNR, while at the same time being less susceptible to ghosting in light areas. Because the motion was in a dark area of ​​the image, ghosting effects are less likely to be present in the image. In general, the techniques of the present disclosure may result in a more optimal selection of image capture techniques such that HDR scenes are captured with a high SNR and with minimal to no ghosting.

[0045] In one example, camera processor 14 may divide the preview frame into multiple regions (e.g., 64×48 pixel regions). HDR selector 19 may obtain luma information (e.g., average luma information) for each of the regions from the AEC performed by ISP 23. HDR selector 19 may determine whether the region is dark based on the luma information of the region compared to a threshold.

[0046] According to one example of the present disclosure, camera processor 14 may be configured to receive a preview image, determine whether the preview image is an HDR scene based on luminance values ​​of pixels of the preview image, determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene, determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image, and generate an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0047] Additional details regarding the operation of HDR selector 19 are described in more detail below with reference to Figures 2-10. As described in more detail below, camera processor 14 may include and / or be configured to execute HDR selector 19. That is, HDR selector 19 may be software executed by camera processor 14, firmware executed by camera processor 14, or dedicated hardware within camera processor 14. Although Figure 1 illustrates HDR selector 19 as being separate from ISP 23, in some examples, HDR selector 19 may be part of ISP 23.

[0048] Although the various structures of the computing device 10 are shown as separate in FIG. 1, the techniques of this disclosure are not so limited, and in some examples, the structures may be combined to form a system on a chip (SoC). As an example, the camera processor 14, the CPU 16, the GPU 18, and the display interface 26 may be formed on a common integrated circuit (IC) chip. In some examples, one or more of the camera processor 14, the CPU 16, the GPU 18, and the display interface 26 may be formed on separate IC chips. Additionally, in some examples, the HDR selector 19 of the camera processor 14 may be part of the ISP 23. In other examples, the HDR selector 19 may be implemented in software and executed by the CPU 16. Various other permutations and combinations are possible, and the techniques of this disclosure should not be considered limited to the example shown in FIG. 1. In an example, the CPU 16 may include the camera processor 14, such that one or more of the camera processors 14 are part of the CPU 16. In such examples, CPU 16 may, as the case may be, be configured to perform one or more of the various techniques attributed to camera processor 14 herein. For purposes of this disclosure, camera processor 14 is described herein as separate and distinct from CPU 16, although this will not always be the case.

[0049] The various structures shown in Figure 1 may be configured to communicate with each other using a bus 32. The bus 32 may be any of a variety of bus structures, such as a third generation bus (e.g., a HyperTransport bus or an InfiniBand bus), a second generation bus (e.g., an Advanced Graphics Port bus, a Peripheral Component Interconnect (PCI) Express bus, or an Advanced eXtensible Interface (AXI) bus), or another type of bus or device interconnect. It should be noted that the particular configuration of buses and communication interfaces between the different structures shown in Figure 1 is merely an example, and other configurations of computing devices and / or image processing systems with the same or different structures may be used to implement the techniques of this disclosure.

[0050] 1 (whether formed on one device or on different devices), including sensor 12 and camera processor 14, may be formed as at least one of fixed-function or programmable circuitry, or a combination of both, such as in one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other equivalent integrated or discrete logic circuitry. Additionally, examples of local memory 20 include one or more volatile or non-volatile memory or storage devices, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media, or optical storage media.

[0051] In some examples, memory controller 24 may facilitate transfer of data to and from system memory 30. For example, memory controller 24 may receive memory read and write commands and service such commands to memory 30 in order to provide memory services to various components of computing device 10. In such examples, memory controller 24 may be communicatively coupled to system memory 30. Although memory controller 24 is illustrated in the example computing device 10 of FIG. 1 as being a processing circuit separate from both CPU 16 and system memory 30, in some examples, some or all of the functionality of memory controller 24 may be implemented in one or more of CPU 16, system memory 30, camera processor 14, video encoder / decoder 17, and / or GPU 18.

[0052] The system memory 30 may store program modules and / or instructions and / or data accessible by the camera processor 14, the CPU 16, and / or the GPU 18. For example, the system memory 30 may store user applications (e.g., instructions for a camera application), obtained images from the camera processor 14, and the like. The system memory 30 may additionally store information for use by and / or generated by other components of the computing device 10. For example, the system memory 30 may act as device memory for the camera processor 14. The system memory 30 may include one or more volatile or non-volatile memory or storage devices, such as, for example, RAM, SRAM, DRAM, ROM, EPROM, EEPROM, flash memory, magnetic data media, or optical storage media. In addition, the system memory 30 may store image data (e.g., frames of video data, encoded video data, sensor mode settings, zoom settings, 3A parameters, etc.). In some examples, the system memory 30 or the local memory 20 may store image data in an on-chip memory, for example in a memory buffer of the system memory 30 or the local memory 20. In another example, system memory 30 or local memory 20 may output image data to be stored outside of the memory of the chip or a buffer, such as on a Secure Digital (SD™) card of the camera device, or in some cases on another internal storage of the camera device. In an illustrative example, system memory 30 or local memory 20 may be embodied as a buffer memory on the chip of camera processor 14, on the chip of GPU 18, or on both, where a single chip includes both processing circuits.

[0053] In some examples, system memory 30 may include instructions that cause camera processor 14, CPU 16, GPU 18, and / or display interface 26 to perform the functions attributed to these components in this disclosure. Thus, system memory 30 may be a computer-readable storage medium that stores instructions that, when executed, cause one or more processors (e.g., camera processor 14, CPU 16, GPU 18, and display interface 26) to perform various techniques of this disclosure.

[0054] In some examples, the system memory 30 is a non-transitory storage medium. The term "non-transitory" indicates that the storage medium is not embodied in a carrier wave or propagated signal. However, the term "non-transitory" should not be interpreted to mean that the system memory 30 is non-movable or that its contents are static. As an example, the system memory 30 may be removed from the computing device 10 and moved to another device. As another example, a memory substantially similar to the system memory 30 may be inserted into the computing device 10. In some examples, the non-transitory storage medium may store data (e.g., in RAM) that can change over time.

[0055] Additionally, camera processor 14, CPU 16, and GPU 18 may store image data, user interface data, etc. in respective buffers allocated in system memory 30. Display interface 26 may retrieve data from system memory 30 and configure display 28 to display images represented by the image data, such as via a screen of user interface 22. In some examples, display interface 26 may include a digital-to-analog converter (DAC) configured to convert digital values ​​retrieved from system memory 30 into analog signals consumable by display 28. In other examples, display interface 26 may communicate digital values ​​directly to display 28 for processing.

[0056] Computing device 10 may include a video encoder and / or a video decoder 17, either of which may be integrated as part of a combined video encoder / decoder (CODEC) (e.g., a video coder). Video encoder / decoder 17 may include a video coder that encodes video captured by one or more cameras 15, or a decoder that can decode compressed or encoded video data. In some instances, CPU 16 and / or camera processor 14 may be configured to encode and / or decode video data, in which case CPU 16 and / or camera processor 14 may include video encoder / decoder 17.

[0057] CPU 16 may comprise a general-purpose or special-purpose processor that controls the operation of computing device 10. A user may provide input to computing device 10 to cause CPU 16 to execute one or more software applications. The software applications executing on CPU 16 may include, for example, a camera application, a graphics editing application, a media player application, a video game application, a graphical user interface application, or another program. For example, a camera application may allow a user to control various settings of camera 15. A user may provide input to computing device 10 via one or more input devices (not shown), such as a keyboard, a mouse, a microphone, a touchpad, or another input device coupled to computing device 10 via user interface 22.

[0058] One exemplary software application is a camera application. CPU 16 executes the camera application, and in response, the camera application causes CPU 16 to generate content for display 28 to output. For example, display 28 may output information such as light intensity, whether a flash is enabled, and other such information. The camera application may also cause CPU 16 to instruct camera processor 14 to process images output by sensor 12 in a manner defined by the user. A user of computing device 10 may interface with display 28 (e.g., via user interface 22) to configure the manner in which images are generated (e.g., with zoom settings applied, with or without flash, focus settings, exposure settings, video or still images, and other parameters). For example, CPU 16 may receive instructions via user interface 22 to capture an image in HDR mode (e.g., capture and generate an HDR image). As described above, in response to a user's indication of use of HDR mode, HDR selector 19 may be configured to determine to use one of a number of HDR image capture techniques based on an analysis of a preview image. In some examples, the HDR selector 19 may be configured to determine to use one of a plurality of HDR image capture techniques regardless of whether a user selects an HDR mode, i.e., the HDR selector 19 may be configured to autonomously determine one of a plurality of HDR modes.

[0059] Display 28 may include a monitor, television, projection device, HDR display, liquid crystal display (LCD), plasma display panel, light emitting diode (LED) array, organic LED (OLED), electronic paper, surface conduction electron emission display (SED), laser television display, nanocrystal display, or another type of display unit. Display 28 may be integrated within computing device 10. For example, display 28 may be the screen of a mobile telephone handset, tablet computer, or laptop. Alternatively, display 28 may be a standalone device coupled to computing device 10 via a wired or wireless communication link. For example, display 28 may be a computer monitor or flat panel display connected to a personal computer via a cable or wireless link. Display 28 may provide a preview frame that a user can view to see what would be stored or what the photo might look like if camera 15 actually took a picture or started recording a video.

[0060] In some examples, camera processor 14 may output the flow of frames to memory controller 24 to have the output frames stored as a video file. In some examples, CPU 16, video encoder / decoder 17, and / or camera processor 14 may output HDR images (e.g., with or without bokeh effects) to be stored as a video file. In some examples, memory controller 24 may generate and / or store the output frames in any suitable video file format. In some examples, video encoder / decoder 17 may encode the output frames before CPU 16, video encoder / decoder 17, and / or camera processor 14 cause the output frames to be stored as encoded video. Encoder / decoder 17 may encode frames of image data using various encoding techniques, including those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), Versatile Video Coding (VCC), etc., and extensions thereof. In a non-limiting example, CPU 16, video encoder / decoder 17, and / or camera processor 14 may cause output frames to be stored using the Moving Picture Experts Group (MPEG) video file format.

[0061] FIG. 2 is a conceptual diagram illustrating an example image sensor and image capture modes. FIG. 2 illustrates a portion of an image sensor 12 capable of operating in both binning mode 200 and in-sensor QBC HDR mode 210. In this example, the image sensor 12 is a QCFA sensor that can be configured to output standard Bayer raw images in either binning mode 200 or in-sensor QBC HDR mode 210, among others. One example of an image sensor 12 is an IMX686 sensor manufactured by Sony Corporation. In one example, the pixel sensor size of the image sensor 12 is 0.8um.

[0062] In some examples, the sensor 12 may be capable of outputting an image at a first higher resolution (e.g., 48 megapixels (MP) or other pixels) in the re-mosaic mode and outputting an image at a second lower resolution (e.g., 12 MP or another reduction from the maximum resolution) in the binning mode 200. FIG. 2 shows one example of 2×2 binning in which four adjacent pixel sensors are averaged to form a single value, as shown in the sensor portion 202. In other examples, other levels of binning may be used. Generally, in the binning mode, the image sensor 12 may be configured to average adjacent pixels (e.g., two or four adjacent pixels) having the same color filter to provide a higher SNR for low-light image capture. Although the resolution is lower compared to the re-mosaic mode (e.g., no binning), the binning mode 200 captures more light per output pixel compared to the re-mosaic mode, thereby increasing the SNR of the image output from the binning mode. This is because the binning mode averages four pixel sensors per output pixel, thereby increasing the effective size of the pixel sensor (e.g., 0.8x to 3.2um in this example).

[0063] In Figure 2, the image sensor 12 has a Quad-Bayer mosaic pattern. In one example, each 16x16 portion of a QCFA sensor may contain two sets of four adjacent red filters, four adjacent blue filters, and four adjacent green filters. Combined together, such a pattern can represent a wide range of colors. More green filters are used because the human eye is more sensitive to green.

[0064] In some examples, camera processor 14 may cause image sensor 12 to use binning mode 200 for MFHDR image capture. In MFHDR image capture, image sensor 12 may be configured to capture three or more consecutive images with different exposure times (e.g., short, medium, long exposure). For example, camera processor 14 may execute an auto exposure algorithm that determines three or more different exposure settings for image sensor 12. For the three image example, camera processor 14 may determine a short exposure time that preserves detail in high luminance areas of the scene, may determine a long exposure time that boosts detail in dark areas of the scene, and may determine a medium long exposure time that maintains more accurate luminance values ​​of mid-tones in the scene. Camera processor 14 may fuse three or more images together to generate an HDR image.

[0065] Because MFHDR image capture involves fusing multiple images captured at different times, the resulting HDR image may exhibit ghosting artifacts and / or other motion blur distortions. Thus, while the use of binning mode 200 for MFHDR generally results in images with a high SNR, images generated from MFHDR modes may be susceptible to ghosting when motion is present in the captured scene. Generally, MFHDR using binning mode 200 results in images with a wider dynamic range (e.g., compared to a single binned frame), images with improved SNR in dark areas (e.g., compared to a non-binning mode or a mode with less binning), and images that may exhibit ghosting artifacts around fast moving objects in the captured scene.

[0066] In another example, the image sensor 12 may be configured to operate according to the QBC HDR mode 210 (e.g., in an in-sensor QBC HDR mode). When operating according to the QBC HDR mode 210, the image sensor 12 may be configured to capture a single image. However, the image capture process and pixel fusion process are different from the binning mode 200. As shown in FIG. 2, for the QBC HDR mode 210, each 2x2 grid of color filters includes one pixel sensor configured to capture a long exposure, two pixel sensors configured to capture a medium time exposure, and one pixel sensor configured to capture a short time exposure. Of course, other combinations of exposure times may be used.

[0067] The QBC HDR mode 210 allows for the capture of light at different exposure times at the same time instance to produce images with a wider dynamic range. As shown in the sensor portion 212, four pixels in each 2×2 color filter grid are fused into one pixel for each color channel. Compared to the binning mode 200, images produced from the QBC HDR mode 210 have a lower SNR because the pixels from the three exposure times fused together are from one or two pixel sensors instead of four. However, because the three exposure times are captured nearly simultaneously with no time lag, images captured using the QBC HDR mode 210 exhibit little or no ghosting. In general, the QBC HDR mode 210 produces images with a wider dynamic range compared to the single frame binning mode, produces images with little or no ghosting compared to the MFHDR mode, and produces images with a lower SNR in dark areas of the scene compared to the MFHDR mode.

[0068] Figure 3 is a block diagram illustrating in more detail example components of the computing device of Figure 1. The technique of Figure 3 will be described with respect to HDR selector 19. As explained above, it should be understood that HDR selector 19 may be implemented in hardware, firmware, software, or any combination thereof, and may be part of camera processor 14, ISP 23, CPU 16, and / or any other processing unit of computing device 10.

[0069] The HDR selector 19 may be configured to receive a preview image 300 from the image sensor 12. In this context, the preview image 300 may be any image captured by the image sensor 12 that is displayed to a user (e.g., on the display 28) while using a camera application of the computing device 10 and / or that is used by the camera processor 14 to determine camera settings. The preview image 300 is generally not a "snapshot" that will be saved on the computing device 10, but rather a "snapshot" that is used by the user to position the scene for capture and by the ISP 23 to set camera functions (e.g., 3A settings). In accordance with the techniques of this disclosure, the HDR selector 19 may be configured to analyze the preview image 300 to determine one of a plurality of image capture techniques (e.g., HDR image capture techniques) to use to capture and generate an HDR image (e.g., output image 314). In some examples, the HDR selector 19 may be configured to continuously analyze the preview image (e.g., at set time intervals) to continuously update the decision of one of the multiple image capture techniques, as lighting conditions and scenes may change over time. The computing device 10 may be configured to capture the preview image 300 using any image capture mode. In some examples, the computing device 10 may capture the preview image 300 using an in-sensor QBC HDR mode.

[0070] HDR selector 19 may be configured to first analyze histogram information 304 associated with preview image 300. Histogram information 304 may include a histogram of brightness values ​​(e.g., luminance values) for each of the pixels of preview image 300. In some examples, HDR selector 19 or another functional unit of camera processor 14 and / or CPU 16 may be configured to determine a histogram of brightness values ​​of the pixels of preview image 300. More information regarding histogram information is described below with reference to FIG.

[0071] In one example of the present disclosure, the HDR selector 19 may be configured to determine whether the preview image 300 is an HDR scene based on the luminance values ​​of the pixels of the preview image. Generally, an HDR scene will have a relatively high percentage of luminance values ​​in both the unsaturated (e.g., very dark) and oversaturated (e.g., very bright) regions of the histogram. If the HDR selector 19 determines that the preview image is not an HDR scene, the HDR selector 19 may be configured to capture a subsequent image using a single frame image capture technique. That is, if the actual scene being captured does not have HDR characteristics (e.g., there is no wide range of luminance values ​​in the scene), the use of the HDR image capture technique may not provide any benefit and may actually reduce the SNR of the output image 314.

[0072] Instead, the HDR selector 19 may determine to use a high SNR image capture technique. Examples of high SNR image capture techniques may include adaptive dynamic range coding (ADRC), single binning frame (e.g., 2×2 binning mode 200 of FIG. 2), and / or single binning frame capture techniques for single frame remosaic mode. The use of a high SNR single frame image capture technique greatly reduces and / or eliminates the possibility of ghosting in an image, since only a single image is captured at a time (e.g., image fusion is not performed). In addition, the use of a single frame image capture technique (e.g., binning frame) generally results in a high SNR in the resulting image.

[0073] If the HDR selector 19 determines that the preview image 300 is an HDR scene based on the histogram information 304, the HDR selector may determine to use one of a plurality of HDR image capture techniques further based on motion detected within the preview image 300. As described above, the plurality of HDR image capture techniques may include MFHDR techniques, staggered HDR techniques, in-sensor QBC HDR techniques, as well as other HDR techniques. Additionally, according to techniques of the present disclosure, the HDR selector 19 may be further configured to select an HDR capture technique that includes the use of a QBC HDR frame fused with a single long exposure binning when there is detected motion but the motion is not within a dark area of ​​the preview image 300.

[0074] The motion detection information 302 may include both an indication of whether there is motion detected within the preview image 300 and an indication of where within the preview image 300 the motion was detected. The location of the motion within the preview image 300 may further include information as to whether the location is a relatively dark or light area of ​​the preview image 300. A further description of the motion detection information is provided below with reference to FIG.

[0075] In one example of the present disclosure, based on determining that the preview image is an HDR scene, the HDR selector 19 may be further configured to detect whether motion is present in the preview image. The motion detection may be performed by the HDR selector 19, or another functional unit of the camera processor 14 and / or the CPU 16 may generate motion detection information 302 based on a comparison of the preview image 300 with a previously captured preview image. The HDR selector 19 may determine to use a first HDR image capture technique based on detecting that motion is not present in the preview image 300, the first HDR image capture technique including capturing multiple images. In one example, the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique. While MFHDR and other multi-frame HDR capture techniques are susceptible to ghosting, MFHDR generally produces images with a high SNR. However, because the HDR selector 19 has determined that there is no motion detected in the preview image 300, MFHDR or other multi-frame HDR capture may be used due to the low probability of ghosting.

[0076] In another example, based on the determination that the preview image 300 is an HDR scene, the HDR selector 19 may be further configured to detect whether motion is present in the preview image 300, and based on detecting the presence of motion in the preview image 300, determine whether the motion is within a dark area of ​​the preview image 300. Based on the determination that the motion is within a dark area of ​​the preview image, the HDR selector 19 may determine to use a single-frame HDR image capture technique (e.g., in-sensor QBC HDR). Single-frame HDR techniques such as QBC HDR may be less susceptible to ghosting, but generally provide a lower SNR. However, increasing the SNR for dark areas may result in ghosting effects if motion is detected within the dark areas. Using QBC HDR for HDR scenes with motion within the dark areas of the scene may provide the best tradeoff between limiting ghosting artifacts and maintaining a high SNR.

[0077] The HDR selector 19 may be configured to determine to use a single frame HDR image capture technique (e.g., QBC HDR) in conjunction with a single frame image capture technique based on determining that the motion is not within a dark area of ​​the preview image 300. Fusing a single frame HDR image with an image from a single frame image capture (e.g., a long exposure binned frame) provides a high SNR, especially in dark areas of the scene, while at the same time being less susceptible to ghosting in bright areas.

[0078] The HDR selector 19 may further be configured to determine a fusion weight where more long exposure binned frames are fused (e.g., blended) together with the QBC HDR image in areas where the scene is dark, but where no motion is present. For example, the HDR selector 19 may be configured to determine the blending weight as a function of pixel luminance values. For brighter areas of the scene, a smaller amount (e.g., down to none) of long exposure binned frames are fused with the QBC HDR image. This is because the HDR selector 19 has detected motion in the bright areas. Blending long exposure binned frames in bright areas of the scene with motion may result in ghosting. However, because bright areas of the scene generally have a higher SNR than dark areas of the scene, a high overall SNR may still be obtained even when pixels from the QBC HDR image make up a large portion of the output image 314.

[0079] Based on the image capture technique selected by the HDR selector 19, the HDR selector 19 or another processing unit of the computing device 10 may cause the image sensor 12 to capture one or more images 308. As explained above, in some circumstances, the HDR selector 19 may select an HDR image capture technique in which only a single image is captured by the image sensor 12, or may select an HDR image capture technique in which multiple images 308 are captured by the image sensor 12. If the image 308 includes multiple images, the multiple images may be processed by the fusion control unit 310. In general, the fusion control unit 310 may apply a processing technique that fuses and / or blends the multiple images into a single image. The fused image is then passed to the tone mapping unit 312. If the image 308 includes only a single image, the image 308 may be passed directly to the tone mapping unit 312. The tone mapping unit 312 may apply a tone mapping process to the received image. In general, the tone mapping process maps one set of colors (e.g., from an input image) to another set of colors to approximate and / or improve the appearance of the HDR image for a medium that may have a more limited dynamic range (e.g., display). The tone mapping unit 312 may then send an output image 314 (e.g., an HDR image) for storage and / or display.

[0080] Figure 4 is a block diagram illustrating in more detail exemplary components of the computing device of Figure 3. Figure 4 illustrates an example in which the HDR selector 19 is configured to select one of four image capture techniques based on the histogram information 304 and the motion detection information 302. As described above, the HDR selector 19 may be configured to select one of the four image capture techniques that achieves high SNR, wide dynamic range, and minimal to no ghosting for various situations.

[0081] The HDR selector 19 may be configured to select the in-sensor QBC HDR image capture technique plus long exposure binned frame (image 308A) in situations where the histogram information 304 indicates that the scene in the preview image 300 is an HDR scene and the motion detection information 302 indicates that motion is present but not within the dark areas of the preview image 300. Because there are multiple images for this image capture technique, the image 308A may then be passed to a fusion control unit 310 for blending.

[0082] As described below with reference to Figures 8-9, the HDR selector 19 may further determine blending weights based on brightness (e.g., luminance values) in the in-sensor QBC HDR frame. The combination of the in-sensor QBC HDR frame and the long exposure binned frame results in an output image 314 that has a wide dynamic range (e.g., from the QBC HDR frame), a high SNR (e.g., from the fusion of the long exposure binned frame in the dark areas of the QBC HDR frame), and little or no ghosting. The fusion of the images 308 results in little ghosting because the long exposure binned frame is more highly blended in the dark areas of the frame when no motion is detected.

[0083] The HDR selector 19 may be configured to select the MFHDR image capture technique (e.g., image 308B) in situations where the histogram information 304 indicates that the scene of the preview image 300 is an HDR scene and the motion detection information 302 indicates that there is no motion in the preview image 300. The image 308B may include three binned frames captured at different times with different length exposures. Since there are multiple images for this image capture technique, the image 308B may then be passed to the fusion control unit 310 for blending. In general, the MFHDR may produce an output image 314 with a wide dynamic range and high SNR. Although MFHDR is subject to ghosting, since no motion was detected in the preview image 300, no ghosting would be expected in the output image 314.

[0084] The HDR selector 19 may be configured to select the in-sensor QBC HDR image capture technique (image 308C) in situations where the histogram information 304 indicates that the scene of the preview image 300 is an HDR scene and the motion detection information 302 indicates that motion is present and that the motion is present in the dark areas of the preview image 300. Since there are no multiple images for this image capture technique, the image 308C may then be passed directly to the tone mapping unit 312. Although the in-sensor QBC HDR capture technique generally has a lower SNR than the MFHDR or QBC HDR plus long exposure binned frames, the single frame in-sensor QBC HDR frame produces an output image 314 with minimal to no ghosting in all situations and a wide dynamic range. Because the motion is in the dark areas due to the selection of the in-sensor QBC HDR capture technique, any attempt to improve the dark areas of the QBC HDR frame may result in unwanted ghosting.

[0085] The HDR selector 19 may be configured to select a single binned frame (e.g., binned frame for ADRC) image capture technique (image 308D) in situations where the histogram information 304 indicates that the scene of the preview image 300 is not an HDR scene. Since there are no multiple images for this image capture technique, the image 308D may then be passed directly to the tone mapping unit 312. The single binned frame for ADRC generally has a high SNR and is free of ghosting for all situations, but exhibits a narrower dynamic range in the output image 314 compared to the other three techniques. However, since the histogram information 304 indicated that the scene was not an HDR scene, the user is unlikely to focus on capturing an image with a narrower dynamic range.

[0086] 5 is a flowchart illustrating an exemplary method of the present disclosure. The HDR selector 19 may first access the histogram information 304 and determine whether the scene of the preview image is an HDR scene (400). If the scene is not an HDR scene (no at 400), the HDR selector 19 selects HDR selection 4 (e.g., single binned frame for ADRC) to generate the image 308D. Generally, HDR selection 4 is a high SNR, narrow dynamic range image capture technique.

[0087] If the scene is an HDR scene (yes at 400), the HDR selector may proceed to examine the motion detection information 302. First, the HDR selector 19 may determine whether there is any motion detected in the preview image (402). If there is no motion detected in the preview image (no at 402), the HDR selector 19 selects HDR selection 2 (e.g., MFHDR) to generate the image 308B. Generally, HDR selection 2 is a multi-frame HDR image capture technique.

[0088] If there is detected motion in the preview image (yes at 402), the HDR selector 19 then determines whether motion is present in a dark area of ​​the preview image (404). If motion is not present in a dark area (no at 404), the HDR selector 19 selects HDR selection 1 (e.g., in-sensor QBC HDR+long exposure binned frame) to generate image 308. If motion is present in a dark area (yes at 404), the HDR selector 19 selects HDR selection 3 (e.g., in-sensor QBC HDR frame) to generate image 308C.

[0089] Fig. 6 is a conceptual diagram illustrating an exemplary motion map. Fig. 6 illustrates a motion map 600 and a motion map 610. The motion maps 600 and 610 are examples of the motion detection information 302. As shown in the motion map 600, the areas 602 marked in black indicate areas of motion. Similarly, in the motion map 610, the areas 604 marked in black indicate areas of motion. The HDR selector 19 may use the motion map to determine whether a scene in a preview image has a moving object.

[0090] The HDR selector 19 or another processing unit of the computing device 10 may use any technique to calculate the motion in the preview image. In one example, the HDR selector 19 may determine the sum of absolute differences (SAD) between pixels of the current preview image compared to pixels of a previously captured preview image. If the SAD of a pixel location in the current frame and the previous frame is greater than some threshold, motion is detected for that pixel. Otherwise, no motion is detected. If SAD(current frame - previous frame) > threshold, Movement detected else No movement detected In one example, the threshold is between 9 and 15.

[0091] The HDR selector 19 may further be configured to determine whether any motion is in a dark area or in a light area according to grid stats. As shown in FIG. 6, the motion maps 600 and 610 may be divided into grids, where each grid represents an M×N region of pixels. If the luminance of the grid where the motion is detected (e.g., the average luminance of the pixels in the grid) is greater than a predetermined threshold, the HDR selector 19 may determine that the motion is in a light area of ​​the preview image. Otherwise, the HDR selector 19 determines that the motion is in a dark area of ​​the preview image. If luminance (grid statistics of moving objects) > threshold Movement in bright areas else Movement in dark areas In one example, the threshold is approximately 100 in the linear domain (eg, before gamma correction).

[0092] FIG. 7 is a conceptual diagram illustrating an example luminance histogram. FIG. 7 illustrates example histograms 700, 710, and 714 that may be generated by the HDR selector 19 and / or other processing circuitry of the computing device 10 and included in the histogram information 304. As shown in histogram 700, the histogram includes information indicating the number of pixels in the preview image having each of the possible luminance values ​​that may be detected by the image sensor 12. In this example, the digital values ​​of the luminance values ​​(e.g., luminance values) may range from 0 to 255. The computing device 10 may further define dark regions 702 and saturated regions 704 (e.g., bright regions) for each histogram. In the example of histogram 700, the dark regions are values ​​0 to 15 (e.g., bins 0 to 15), and the saturated regions are values ​​250 to 255 (e.g., bins 250 to 255). Of course, dark and saturated regions of other sizes may be used.

[0093] The HDR selector 19 may use the histogram distribution information to determine whether the preview image is an HDR scene. The HDR selector 19 or another processing unit of the computing device 10 may calculate the percentage of saturated areas by dividing the number of saturated pixels (i.e., bright pixels) by the total number of pixels (e.g., all saturated pixels / all pixels). The HDR selector 19 may further calculate the percentage of dark areas by dividing the number of dark pixels by the total number of pixels (e.g., all dark pixels / all pixels). If the percentage of saturated areas added to the percentage of dark areas is greater than a predefined threshold, the HDR selector 19 may determine that the scene of the preview image is an HDR scene. Otherwise, the HDR selector 19 may determine that the scene of the preview image is not an HDR scene. In one example, the saturated areas (e.g., bright pixels) are defined as the highest 95%-100% of the luminance values ​​available in the histogram. The dark areas are defined as the lowest 0-25% of the luminance values ​​available in the histogram. Of course, the definitions of saturated and dark areas may be adjusted for other applications.

[0094] Histogram 710 shows an example histogram for an HDR scene, where the percentage of dark areas is 5% and the percentage of saturated areas is 15%. Histogram 714 shows an example histogram for a non-HDR scene, where the percentage of dark areas is 3% and the percentage of saturated areas is 0.7%.

[0095] 8 is a block diagram illustrating in more detail example components of the computing device of FIG. 3 for the HDR and binning frame image capture technique. Specifically, FIG. 8 illustrates an example in which the HDR selector 19 selects HDR selection 1 (e.g., in-sensor QBC HDR frame+long exposure binned frame) for the generated output image 314. In this example, the fusion control unit 310 may be configured to fuse the QBC HDR frame with the long exposure binned frame according to the QBC-HDR weights 316.

[0096] The HDR selector 19 or another processing unit of the computing device 10 may be configured to determine the QBC-HDR weights 316 for each pixel in the output image 314. For example, the HDR selector 19 may be configured to determine the QBC-HDR weights 316 as a function of pixel intensity (e.g., brightness or luminance) in the preview image.

[0097] In one example, the HDR selector 19 may calculate the QBC-HDR weights 316 as follows:

number

[0098] In general, the fusion control unit 310 may be configured to apply higher weights to the QBC HDR frames and lower weights to the long exposure binned frames for brighter pixels. The fusion control unit 310 may create the fused image using the following formula: QBC-HDR weight(pixel intensity)*QBC HDR frame+(1-QBC-HDR weight(pixel intensity))*binned frame In the above equation, the term QBC-HDR(pixel intensity) indicates that the QBC-HDR weight at any particular pixel is a function of the intensity (e.g., luminance) of that pixel. As can be seen from the plot 900, the higher the pixel intensity (e.g., the higher the luminance), the higher the value of the QBC-HDR weight. Thus, for HDR selection 1, the fusion control unit 310 will further use pixels from the long exposure binned frame for very dark pixel intensities (e.g., below th1), will use pixels from the in-sensor QBC HDR frame for very bright pixel intensities (e.g., above th2), and will blend pixels from the long exposure binned frame and the QBC HDR frame for pixel intensities between th1 and th2. In this way, the SNR is improved in dark areas of the output image 314 while maintaining an absence of ghosting in bright areas.

[0099] 10 is a flowchart illustrating an example operation of camera processing according to an example technique of this disclosure. The technique of FIG. 10 may be performed by one or more structural units of computing device 10, including camera processor 14 and CPU 16.

[0100] In one example of the present disclosure, camera processor 14 and / or CPU 16 may be configured to receive a preview image (1000). For example, camera processor 14 and / or CPU 16 may be configured to cause one of cameras 15 to capture an image. Camera processor 14 and / or CPU 16 may be further configured to determine whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image (1010). Camera processor 14 and / or CPU 16 may be configured to determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene (1020).

[0101] For example, camera processor 14 and / or CPU 16 may be configured to determine a histogram of luminance values ​​of pixels of the preview image. To determine whether the preview image is an HDR scene based on the luminance values ​​of the pixels of the preview image, camera processor 14 and / or CPU 16 may be configured to determine whether the preview image is an HDR scene based on whether a ratio of luminance values ​​in the oversaturated region of the histogram and the unsaturated region of the histogram is greater than a threshold (e.g., as described above). In one example, the single frame image capture technique is a single binning frame capture technique for adaptive dynamic range coding (ADRC).

[0102] The camera processor 14 and / or CPU 16 may be further configured to determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image (1030). In one example, to determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image, the camera processor 14 and / or CPU 16 may be configured to detect whether motion is present in the preview image and determine to use a first HDR image capture technique based on detecting an absence of motion in the preview image, the first HDR image capture technique including capturing a plurality of images. In one example, the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique.

[0103] In another example, to determine to use one of a plurality of HDR image capture techniques based on determining that the preview image is an HDR scene and further based on motion detected in the preview image, camera processor 14 and / or CPU 16 may be configured to detect whether motion is present in the preview image, determine whether the motion is within a dark area of ​​the preview image based on detecting the presence of motion in the preview image, determine to use a single frame HDR image capture technique based on determining that the motion is within a dark area of ​​the preview image, and determine to use a single frame HDR image capture technique together with the single frame image capture technique based on determining that the motion is not within a dark area of ​​the preview image. In one example, the single frame HDR image capture technique is an in-sensor Quad-Bayer coding (QBC) HDR image capture technique and the single frame image capture technique is a long exposure binning frame image capture technique.

[0104] When using the single frame HDR image capture technique together with the single frame image capture technique, camera processor 14 and / or CPU 16 may be configured to capture a first image using the in-sensor QBC HDR image capture technique, capture a second image using the long exposure binned frame image capture technique, and blend the first image and the second image into an output image based on the blending weights. In this example, camera processor 14 and / or CPU 16 may be further configured to determine the blending weights as a function of pixel luminance values.

[0105] In the above example, camera processor 14 and / or CPU 16 may be configured to detect whether motion is present in the preview image based on a function of the sum of absolute differences in pixel values ​​between the preview image and a previously captured image. Camera processor 14 and / or CPU 16 may be configured to generate an output image using one or more images captured using a single frame image capture technique or one of a plurality of HDR image capture techniques (1140). For example, camera processor 14 and / or CPU 16 may be configured to cause one or more of cameras 15 to capture one or more images using a single frame image capture technique or one of a plurality of HDR image capture techniques.

[0106] Other illustrative examples of the present disclosure are described below in the following sections.

[0107] Clause 1. A device configured for camera processing, comprising: a memory configured to receive images; and one or more processors in communication with the memory, wherein the one or more processors are configured to receive a preview image; determine whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image; determine to use a single frame image capture technique based on a determination that the preview image is not an HDR scene; determine to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image; and generate an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0108] Clause 2. The device of clause 1, wherein the one or more processors are further configured to: detect whether motion is present in the preview image; and determine to use one of a plurality of HDR image capture techniques based on determining that the preview image is an HDR scene and further based on motion detected in the preview image; and determine to use a first HDR image capture technique based on detecting an absence of motion in the preview image, the first HDR image capture technique including capturing a plurality of images.

[0109] Clause 3. The device of clause 2, wherein the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique.

[0110] Clause 4. The device of any combination of clauses 1 to 3, wherein to determine to use one of a plurality of HDR image capture techniques based on determining that the preview image is an HDR scene and further based on motion detected in the preview image, the one or more processors are further configured to: detect whether motion is present in the preview image; determine whether the motion is within a dark area of ​​the preview image based on detecting the presence of motion in the preview image; determine to use a single frame HDR image capture technique based on determining that the motion is within a dark area of ​​the preview image; and determine to use a single frame HDR image capture technique together with the single frame image capture technique based on determining that the motion is not within a dark area of ​​the preview image.

[0111] Clause 5. The device of clause 4, wherein the single frame HDR image capture technique is an in-sensor Quad-Bayer coding (QBC) HDR image capture technique, and the single frame image capture technique is a long exposure binning frame image capture technique.

[0112] Clause 6. The device of clause 5, wherein the one or more processors determine to use a single frame HDR image capture technique together with the single frame capture technique, and the one or more processors are further configured to capture a first image using the in-sensor QBC HDR image capture technique and capture a second image using the long exposure binned frame image capture technique, and fuse the first image and the second image into an output image based on blending weights.

[0113] Clause 7. The device of clause 6, wherein the one or more processors are further configured to determine blending weights as a function of pixel luminance values.

[0114] Clause 8. The device of any combination of clauses 1 to 7, wherein the one or more processors are further configured to detect whether motion is present in the preview image based on a function of a sum of absolute differences in pixel values ​​between the preview image and a previously captured image.

[0115] Clause 9. The device of any combination of clauses 1 to 8, wherein the one or more processors are further configured to determine a histogram of luminance values ​​of pixels of the preview image.

[0116] Clause 10. The device of clause 9, wherein to determine whether the preview image is an HDR scene based on luminance values ​​of pixels of the preview image, the one or more processors are configured to determine whether the preview image is an HDR scene based on whether a proportion of luminance values ​​within an oversaturated region of a histogram and an unsaturated region of the histogram is greater than a threshold.

[0117] Clause 11. The device of clause 10, wherein the single frame image capture technique is a single binning frame capture technique for adaptive dynamic range coding (ADRC).

[0118] Clause 12. The device of any combination of clauses 1 to 11, further comprising one or more cameras configured to capture one or more images using one of a single frame image capture technique or a plurality of HDR image capture techniques.

[0119] Clause 13. A method of camera processing, comprising: receiving a preview image; determining whether the preview image is a high dynamic range (HDR) scene based on luminance values ​​of pixels of the preview image; determining to use a single frame image capture technique based on a determination that the preview image is not an HDR scene; determining to use one of a plurality of HDR image capture techniques based on a determination that the preview image is an HDR scene and further based on motion detected in the preview image; and generating an output image using one or more images captured using the single frame image capture technique or one of the plurality of HDR image capture techniques.

[0120] Clause 14. The method of clause 13, wherein determining to use one of a plurality of HDR image capture techniques based on determining that the preview image is an HDR scene and further based on motion detected in the preview image includes detecting whether motion is present in the preview image, and determining to use a first HDR image capture technique based on detecting an absence of motion in the preview image, the first HDR image capture technique including capturing a plurality of images.

[0121] Clause 15. The method of clause 14, wherein the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique.

[0122] Clause 16. The method of any combination of clauses 13 to 15, wherein the step of determining to use one of a plurality of HDR image capture techniques based on determining that the preview image is an HDR scene and further based on motion detected in the preview image includes the steps of detecting whether motion is present in the preview image, determining whether the motion is within a dark area of ​​the preview image based on detecting the presence of motion in the preview image, determining to use a single frame HDR image capture technique based on determining that the motion is within a dark area of ​​the preview image, and determining to use a single frame HDR image capture technique together with the single frame image capture technique based on determining that the motion is not within a dark area of ​​the preview image.

[0123] Clause 17. The method of clause 16, wherein the single frame HDR image capture technique is an in-sensor Quad-Bayer coding (QBC) HDR image capture technique, and the single frame image capture technique is a long exposure binning frame image capture technique.

[0124] Clause 18. The method of clause 17, further comprising: determining to use a single frame HDR image capture technique in conjunction with the single frame image capture technique; capturing a first image using an in-sensor QBC HDR image capture technique; capturing a second image using a long exposure binned frame image capture technique; and fusing the first image and the second image into an output image based on blending weights.

[0125] Clause 19. The method of clause 18, further comprising determining blending weights as a function of pixel luminance values.

[0126] Clause 20. The method of any combination of clauses 13 to 19, further comprising detecting whether motion is present in the preview image based on a function of a sum of absolute differences in pixel values ​​between the preview image and a previously captured image.

[0127] Clause 21. The method of any combination of clauses 13 to 20, further comprising determining a histogram of luminance values ​​of pixels of the preview image.

[0128] Clause 22. The method of clause 21, wherein determining whether the preview image is an HDR scene based on luminance values ​​of pixels of the preview image includes determining whether the preview image is an HDR scene based on whether a proportion of luminance values ​​within an oversaturated region of a histogram and an unsaturated region of a histogram is greater than a threshold.

[0129] Clause 23. The method of clause 22, wherein the single frame image capture technique is a single binning frame capture technique for adaptive dynamic range coding (ADRC).

[0130] Clause 24. The method of any combination of clauses 13 to 23, further comprising capturing one or more images using one of a single frame image capture technique or a plurality of HDR image capture techniques.

[0131] Clause 25. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a device to perform any combination of the techniques of clauses 13 to 24.

[0132] Clause 26. A device for camera processing, comprising means for performing any combination of the techniques of clauses 13 to 24.

[0133] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium that corresponds to a tangible medium, such as a data storage medium. In this manner, a computer-readable medium may generally correspond to a tangible computer-readable storage medium that is non-transitory. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0134] By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, cache memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and accessed by a computer. It should be understood that computer-readable storage media and data storage media do not include carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0135] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the above structures, or any other structure suitable for implementing the techniques described herein. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0136] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but they do not necessarily require realization by different hardware units.

[0137] Various examples have been described. These and other examples are within the scope of the following claims. [Explanation of symbols]

[0138] 10. Computing Devices 12 Image sensor, sensor 12A~N Image Sensor 13 Lens 13A~N Lens 14 Camera Processor 15 Camera, 1st Camera, 2nd Camera 16 Central Processing Unit (CPU) 17 Video Encoders / Decoders 18 Graphics Processing Unit (GPU) 19 HDR Selector 20 Local Memory 22 User Interface 23 Image Signal Processor (ISP) 24 Memory Controller 26 Display Interface 28 Display 30 System Memory, Memory 32 Bus 200 Binning Mode 202 Sensor part 210 In-sensor QBC HDR mode, QBC HDR mode 300 Preview Images 302 Motion Detection Information 304 Histogram Information 308 images 308A Images 308B Images 308C Images 308D Images 310 Fusion Control Unit 312 Tone Mapping Unit 314 Output Image 316 QBC-HDR Weights 600 Movement Map 610 Movement Map 700 Histogram 710 Histogram 714 Histogram 900 Plots

Claims

1. A device configured for image processing, comprising: a memory configured to receive the image; One or more cameras; and one or more processors in communication with the memory, the one or more processors comprising: receiving a preview image from the one or more cameras; comparing luminance values ​​of pixels in the preview image to a first threshold to determine dark areas of the preview image, the dark areas being less than all of the pixels of the preview image; detecting whether motion is present in the dark area of ​​the preview image; determining to use one of a plurality of high dynamic range (HDR) image capture techniques based on whether motion is present in the dark areas of the preview image; generating an output image using one or more images captured using one of the plurality of HDR image capture techniques, wherein the one or more cameras are configured to capture the one or more images using the one of the plurality of HDR image capture techniques; A device configured to:

2. The one or more processors, determining that the preview image represents an HDR scene based on the luminance values ​​of pixels of the preview image; determining to use one of the plurality of HDR image capture techniques based on the determination that the preview image represents an HDR scene and further based on whether motion is present in the dark areas of the preview image; [0023] 20. The method according to claim 1, further comprising: the one or more processors: determining, based on detecting an absence of motion in the preview image, to use a first HDR image capture technique, the first HDR image capture technique including capturing a plurality of images. The device of claim 1 , further configured to:

3. The device of claim 2 , wherein the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique.

4. determining to use one of the plurality of HDR image capture techniques based on whether motion is present in the dark areas of the preview image, the one or more processors; determining to use a single frame HDR image capture technique based on determining that the motion is within the dark area of ​​the preview image; and determining to use the single frame HDR image capture technique in conjunction with a single frame image capture technique based on determining that the motion is not within the dark area of ​​the preview image; The device of claim 1 , further configured to:

5. 5. The device of claim 4, wherein the single-frame HDR image capture technique is an in-sensor Quad-Bayer coding (QBC) HDR image capture technique, and the single-frame image capture technique is a long exposure binning frame image capture technique.

6. the one or more processors: determining to use the single frame HDR image capture technique together with a single frame image capture technique, the one or more processors: capturing a first image using the in-sensor QBC HDR image capture technique; capturing a second image using said long exposure binned frame image capture technique; Blending the first image and the second image into the output image based on blending weights. The device of claim 5 , further configured to:

7. the one or more processors: Determining the blending weights as a function of pixel luminance values The device of claim 6, further configured to:

8. A method of image processing, comprising the steps of: receiving a preview image; comparing luminance values ​​of pixels in the preview image to a first threshold to determine dark areas of the preview image; detecting whether motion is present in the dark area of ​​the preview image; determining to use one of a plurality of high dynamic range (HDR) image capture techniques based on whether motion is present in the dark areas of the preview image; generating an output image using one or more images captured using one of the plurality of HDR image capture techniques; A method comprising:

9. The method according to claim 8, further comprising: determining that the preview image represents an HDR scene based on the luminance values ​​of pixels of the preview image; determining to use one of the plurality of HDR image capture techniques, determining to use one of the plurality of HDR image capture techniques based on the determination that the preview image represents an HDR scene and further based on whether motion is present in the dark areas of the preview image; 10. The method of claim 8, wherein the method further comprises: determining to use a first HDR image capture technique based on detecting an absence of motion in the preview image, the first HDR image capture technique capturing multiple images.

10. The method of claim 9 , wherein the first HDR image capture technique is one of a multi-frame HDR (MFHDR) image capture technique or a staggered multi-frame HDR image capture technique.

11. determining to use one of the plurality of HDR image capture techniques based on whether motion is present in the dark area of ​​the preview image; determining to use a single frame HDR image capture technique based on determining that the motion is within the dark area of ​​the preview image; determining to use the single frame HDR image capture technique in conjunction with a single frame image capture technique based on determining that the motion is not within the dark areas of the preview image; The method of claim 8 , comprising:

12. The method of claim 11 , wherein the single-frame HDR image capture technique is an in-sensor Quad-Bayer coding (QBC) HDR image capture technique, and the single-frame image capture technique is a long exposure binning frame image capture technique.

13. determining to use the single frame HDR image capture technique together with a single frame image capture technique; capturing a first image using the in-sensor QBC HDR image capture technique; capturing a second image using the long exposure binned frame image capture technique; blending the first image and the second image into the output image based on blending weights; 13. The method of claim 12, further comprising:

14. determining said blending weights as a function of pixel luminance values; 14. The method of claim 13, further comprising:

15. A computer program which, when executed, causes one or more processors of a device to perform a method according to any one of claims 8 to 14.