Systems and methods for controlling exposure settings based on motion characteristics associated with an image sensor

Abstract

The calculation of exposure settings may incur processing delays.SOLUTION: A system and method for processing one or more images includes determining one or more current exposure settings for a current image of a current scene at a current time. One or more motion characteristics associated with the image sensor are determined. A location of a portion of the current image to use for determining one or more future exposure settings for the image sensor is determined based on the one or more motion characteristics, the one or more future exposure settings for capturing a future image of a future scene at a future time, the future time being subsequent to the current time. The one or more future exposure settings are determined based on the predicted portion of the current image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Claiming priority under 35 U.S.C. § 119

[0001] This patent application claims priority to non-provisional application Ser. No. 16 / 438,359, filed on June 11, 2019, entitled "SYSTEMS AND METHODS FOR CONTROLLING EXPOSURE SETTINGS BASED ON MOTION CHARACTERISTICS ASSOCIATED WITH AN IMAGE SENSOR," which is assigned to the assignee of the present application and expressly incorporated by reference herein.

[0002]

[0002] The present disclosure relates generally to techniques and systems for controlling auto exposure of an image. More specifically, exemplary aspects are directed to controlling the exposure of an image based on motion characteristics associated with an image sensor. [Background technology]

[0003]

[0003] In photography, the exposure of an image captured by a camera refers to the amount of light per unit area that reaches the photographic film, or in modern cameras, the electronic image sensor. Exposure is based on camera settings such as shutter speed and lens aperture, as well as the luminance of the scene being photographed. Many cameras are equipped with an automatic exposure or "auto exposure" mode, in which the exposure settings (e.g., shutter speed, lens aperture, etc.) can be automatically adjusted to match as closely as possible the luminance of the scene or subject being photographed.

[0004]

[0004] Calculating exposure settings can incur processing delays. For example, processing delays can occur from the time calculations for exposure settings for an image begin to the time the exposure settings are applied to a camera to capture the image. This processing delay can be unacceptably high in some situations. Summary of the Invention

[0005] In some examples, techniques and systems for processing one or more images are described. Some examples include determining one or more current exposure settings for a current image of a current scene to be captured by a camera or image sensor at a current time. One or more motion characteristics associated with the image sensor may be determined. The one or more motion characteristics may include one or more of a speed or a direction of travel of the image sensor. In some examples, the one or more motion characteristics of the image sensor may be determined based at least in part on one or more of an inertial sensor or an input to a steering system in communication with the image sensor. In some examples, determining the one or more motion characteristics may be based at least in part on one or more regions of interest in the current image. Based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor may be predicted. One or more future exposure settings that may be used to capture a future image of a future scene at a future time may be determined based on a predicted portion of the current image. In some examples, the future time follows the current time. In some examples, the one or more future exposure settings may be determined before a field of view of the image sensor reaches the future scene. In this way, once the field of view of the image sensor reaches or includes the future scene, the future exposure settings may be ready and available, thus allowing processing of the future image to be expedited.

[0006] According to at least one example, a method for processing one or more images is provided. The method includes determining one or more current exposure settings for a current image of a current scene at a current time. The method further includes determining one or more motion characteristics associated with the image sensor. The method further includes predicting, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor, one or more future exposure settings for capturing a future image of a future scene at a future time, and a future time subsequent to the current time. The method further includes determining the one or more future exposure settings based on the predicted portion of the current image.

[0007] In another example, an apparatus for processing one or more images is provided, the apparatus including: a memory configured to store one or more images; and a processor coupled to the memory. The processor is implemented in circuitry and configured to determine one or more current exposure settings for a current image of a current scene at a current time. The processor is further configured to determine one or more motion characteristics associated with the image sensor. The processor is further configured to predict, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor, the one or more future exposure settings being for capturing a future image of a future scene at a future time, the future time being subsequent to the current time. The processor is further configured to determine, based on the predicted portion of the current image, the one or more future exposure settings.

[0008]

[0008] In another example, a non-transitory computer-readable medium is provided that stores instructions that, when executed by one or more processors, cause the one or more processors to: determine one or more current exposure settings for a current image of a current scene at a current time; determine one or more motion characteristics associated with an image sensor; predict, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor; the one or more future exposure settings are for capturing a future image of a future scene at a future time, the future time being subsequent to the current time; and determine the one or more future exposure settings based on the predicted portion of the current image.

[0009] In another example, an apparatus for processing one or more images is provided. The apparatus includes means for determining one or more current exposure settings for a current image of a current scene at a current time. The apparatus further includes means for determining one or more motion characteristics associated with the image sensor. The apparatus further includes means for predicting, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor, the one or more future exposure settings being for capturing a future image of a future scene at a future time, the future time being subsequent to the future time. The apparatus further includes means for determining the one or more future exposure settings based on the predicted portion of the current image.

[0010]

[0010] In some embodiments, the above-described methods, apparatus, and computer-readable media further include determining one or more future exposure settings before the field of view of the image sensor reaches the future scene.

[0011]

[0011] In some aspects of the above-described methods, devices, and computer-readable media, predicting the location of a portion of the current image to be used to determine one or more future exposure settings includes excluding a region of the current image from being used to determine one or more future exposure settings.

[0012] In some aspects of the above-described methods, apparatus, and computer-readable media, the one or more motion characteristics include one or more of a velocity or direction of travel of the image sensor.

[0013]

[0013] In some aspects, the above-mentioned methods, apparatus, and computer-readable media further include determining one or more movement characteristics based at least in part on one or more of the inertial sensors or inputs to the steering system in communication with the image sensor.

[0014]

[0014] In some aspects, the above-mentioned methods, apparatus, and computer-readable media further include determining one or more motion characteristics based at least in part on one or more regions of interest of the current image.

[0015]

[0015] In some aspects of the above-described methods, devices, and computer-readable media, determining one or more future exposure settings based on a predicted portion of the current image includes predicting one or more regions of interest of the future image within the predicted portion, and adjusting exposure settings for the one or more regions of interest.

[0016]

[0016] In some embodiments, the above-described methods, apparatus, and computer-readable media further include determining a grid comprising one or more grid elements associated with one or more regions of a future image within the predicted portion, and determining weights for the one or more grid elements, wherein one or more future exposure settings are based on the weights.

[0017] In some aspects of the above-described methods, apparatus, and computer-readable media, the one or more future exposure settings include a variation in one or more current exposure settings of the current image.

[0018] In some aspects, the above-described methods, apparatus, and computer-readable media further include capturing future images with the image sensor based on the one or more future exposure settings.

[0019] This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used separately to determine the scope of the claimed subject matter, which subject matter should be understood by reference to the entire patent specification, any or all drawings, and appropriate portions of each claim.

[0020]

[0020] The foregoing, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings.

[0021]

[0021] Exemplary embodiments of the present application are described in detail below with reference to the following figures: [Brief explanation of the drawings]

[0022] [Figure 1]

[0022] FIG. 1 is a block diagram illustrating an exemplary image processing system according to the present disclosure. [Figure 2A]

[0023] 1 is a schematic diagram illustrating an image captured by a camera according to the present disclosure. [Figure 2B]

[0024] 1 is a schematic diagram of a grid for processing an image according to the present disclosure; [Figure 3A]

[0025] 1 is a schematic diagram of a grid for processing an image based on one or more motion characteristics associated with a camera, according to the present disclosure. [Figure 3B] 1 is a schematic diagram of a grid for processing an image based on one or more motion characteristics associated with a camera, according to the present disclosure. [Figure 4A]

[0026] 1 is a schematic diagram of a grid for processing an image based on one or more motion characteristics and one or more regions of interest associated with a camera, according to the present disclosure. [Figure 4B] 1 is a schematic diagram of a grid for processing an image based on one or more motion characteristics and one or more regions of interest associated with a camera, according to the present disclosure. [Figure 5]

[0027] 1 is a schematic diagram of a grid for processing an image based on one or more motion characteristics associated with a camera and a steering direction of a steering wheel, according to the present disclosure. [Figure 6A]

[0028] 1 is a schematic diagram of variation in camera field of view based on camera movement, according to the present disclosure; [Figure 6B] 1 is a schematic diagram of the variation in camera field of view based on camera movement, according to the present disclosure; [Figure 6C] 1 is a schematic diagram of the variation in camera field of view based on camera movement, according to the present disclosure; [Figure 6D] 1 is a schematic diagram of the variation in camera field of view based on camera movement, according to the present disclosure; [Figure 7]

[0029] 1 is a flowchart illustrating an example process for processing one or more images according to the present disclosure. [Figure 8]

[0030] FIG. 1 is a block diagram illustrating an example computing device architecture of an example computing device capable of implementing various techniques described herein. DETAILED DESCRIPTION OF THE INVENTION

[0023]

[0031] Certain aspects and embodiments of the present disclosure are provided below. As will be apparent to one skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be limiting.

[0024]

[0032] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the present application, as set forth in the appended claims.

[0025]

[0033] A camera may include a mechanism for calculating appropriate exposure settings for an image captured by the camera. In some examples, the exposure settings of a camera may be dynamically adjusted. For example, the exposure settings may include a setting for the aperture of the camera's lens. In some examples, the exposure settings may also include a setting for the camera's sensitivity or gain, which may be based on a standard such as the sensitivity defined by the International Organization for Standardization (ISO), also known as ISO sensitivity. The exposure settings for aperture and shutter speed, for example, may control the amount of time an image of a scene is exposed to the camera's image sensor. The exposure settings for aperture and speed may also be referred to as exposure settings for exposure time.

[0026]

[0034] The camera's shutter speed and aperture can control the amount of light that enters the camera. In some examples, electronic sensors within the camera can detect light reflected from a scene, where measurements of the reflected light can be used to adjust the exposure settings of an image of the scene captured by the camera. Measuring reflected light is also referred to as photometry.

[0027]

[0035] In some conventional metering mechanisms (also known as metering modes or systems), the exposure setting for the entire image may be calculated based on an “assumption” of the amount of reflectance in the default scene (e.g., a determination made without performing real-time calculations or analysis to make the determination). In some examples, the metering mode may assume that the scene has a predetermined percentage of reflectance (e.g., approximately 18% in some known implementations) and calculate the exposure accordingly. In some examples, an average metering mode may be utilized, where the reflected light intensity from nearly the entire field of view of the camera is added and averaged, and the average intensity is used to control the exposure setting for the camera. While these metering modes may be appropriate in some situations (e.g., a static camera used to capture images of scenes that do not include very bright or very dark backgrounds), these metering modes may not be effective in preventing disproportionately bright or dark areas of the scene from adversely affecting the exposure of the entire scene.

[0028]

[0036] For example, relying on such assumptions of a predetermined percentage reflectance or average measurement may be inappropriate for scenes with a large proportion of highly reflective surfaces and may lead to undesirable results. For example, scenes with snowy backgrounds, large bodies of water, bright light sources, etc. may lead to images of the scene that are underexposed. An example of such an underexposed image may be seen in a photograph of a snowy scene, where the snow is depicted in a cold blue color instead of a more realistic bright white appearance. Similarly, scenes with a large proportion of non-reflective surfaces or darker backgrounds (e.g., nighttime or dimly lit scenes or dark backgrounds) may lead to images of the scene that are overexposed.

[0029]

[0037] In some examples, the brightness range of light from a scene may significantly exceed the brightness levels that an image sensor can capture. For example, a digital single-lens reflex (DSLR) camera may be capable of capturing a contrast ratio of 1:30,000 of light from a scene, while the brightness levels of a high dynamic range (HDR) scene may exceed a contrast ratio of 1:1,000,000. In some cases, an HDR sensor may be utilized to increase the contrast ratio of an image captured by the camera. In some examples, an HDR sensor may be used to capture multiple exposures within a single image or frame. In some examples, an HDR sensor may be used to capture multiple exposures across multiple frames, where such multiple exposures may include short, intermediate, and long exposures. However, spatial or temporal artifacts may limit the effectiveness of using such an HDR sensor to obtain appropriate photometry for a scene. For example, some scenes may not require multiple exposures, and therefore, the use of multiple exposures may be wasteful.

[0030]

[0038] Thus, some measurement modes attempt to reduce the effect of disproportionately bright or dark areas of a scene and set appropriate exposure amounts for various regions of an image within the camera's field of view. In some examples, measurement modes may be used to control exposure settings to avoid unnecessary use of multiple exposures for scenes that do not have high contrast ratio artifacts. In some examples, such measurement modes may use weight-based techniques to weight the brightness levels of different regions of an image to be captured. For example, one or more weights may be associated with a region of an image, where the exposure setting for that region may be based on the weight or weights associated with that region. In some examples, using one or more weights to more heavily weight the brightness levels of a region may result in a first type of exposure setting for the region, while less heavily weighting the brightness levels for the region may result in a second type of exposure setting. In some examples, the first type of exposure setting may include a higher or larger exposure setting, while the second type of exposure setting may include a lower or smaller exposure setting. In some examples, the weight-based measurement mode may include a predetermined pattern, such as a grid, to determine the exposure setting for a region of an image to be captured. For example, grid elements of a grid can correspond to exposure settings for different regions of an image. A weight-based metering mode can include weights for the grid elements, where the weights can be used to control exposure settings for the corresponding regions of the image. In some cases, a weight-based metering mode can also be referred to as grid metering. Various types of grid metering can be used, such as center-weighted metering, spot metering, matrix metering, etc.

[0031]

[0039] In center-weighted metering, increased emphasis or weighting is applied to grid elements closest to the center of the image to be captured by the camera. Center-weighted metering operates under the assumption that the object of interest may be at the center of the camera's field of view, and the most accurate exposure setting may be desired for the object of interest. As can be appreciated, this assumption may not be true and / or may not be sustainable in some situations. For example, if there are more than one object of interest in a scene, and / or if one or more objects are moving while the camera is fixed (e.g., in a surveillance camera located in a fixed position), and / or if the camera is also moving, exposure settings based on center-weighted metering may not provide appropriate exposure settings for grid elements in different regions of the image.

[0032]

[0040] Spot metering is similar to center-weighted metering, except that in spot metering, a spot or area of ​​the image to be captured is emphasized, although the spot need not be the center of the image. For example, spot metering may provide weighted emphasis to off-center locations of the image by adjusting exposure settings to those locations before composing the final scene. Spot metering may also be inappropriate in various situations for reasons similar to center-weighted metering.

[0033]

[0041] Matrix metering may adaptively adjust weights for different grid elements based on different algorithms. Matrix weight metering may provide fine-tuned exposure settings in a distributed manner across grid elements in various regions of an image, rather than being limited to the center or spot, as discussed with reference to center-weighted metering and spot metering. However, matrix metering may involve significant computational effort in calculating and recalculating weights for grid elements in various regions, where these calculations may incur processing delays.

[0034]

[0042] In some examples, the scene seen by the camera (e.g., the field of view detected and captured by an image sensor of an image capture device, e.g., a camera) may change from moment to moment. In some examples, the camera may capture images of different scenes, for example, at different time instances. In some examples, the camera may capture a series of still images of different scenes or the same scene. Additionally, the camera may include a video mode for capturing video comprising a sequence of images across different scenes or a sequence of images of the same scene. In some examples, the exposure settings for the camera may also need to be changed for images of different scenes to be captured. For example, in a weight-based metering mode utilized to control exposure settings, the weights may need to be changed from a current image of a current scene at a current time to a future image of a future scene at a future time, where the scene may change from the current scene to the future scene after the passage of time from the current time to the future time, the future time following the current time. Computing and recalculating weights to keep up with scene changes (e.g., in various grid metering modes discussed above) incurs processing time. These processing times may vary based on the grid measurement mode, as described above. If the transition from the current scene to the future scene is gradual, or if the time lapse is large, the processing times may not be significant, but rapid changes between scenes with smaller time lapses may pose challenges.

[0035]

[0043] In some situations, movement of an object within a scene viewed by a camera may trigger the need to readjust exposure settings. In an illustrative example, a moving automobile may be within a scene viewed by a camera while the camera is relatively stationary or fixed. In these situations, various analytical tools may be used to recalculate weights for adjusting exposure settings as new images, such as of a moving automobile, enter the camera's field of view. For example, predictive algorithms may be used in various grid measurement modes to calculate appropriate weights for new images or to facilitate recalculation of appropriate weights for new images.

[0036]

[0044] In some situations, the scene seen by a camera may change due at least in part to movement of the camera itself. For example, a moving camera may be used to capture a still image, or there may be movement in both the camera and objects within the scene seen by the camera, causing relative movement. An illustrative example of a moving camera may involve a sweeping motion of a camera held by a user from left to right to capture a panoramic view of a natural landscape. An illustrative example of both a moving camera and object may involve a camera placed on a moving vehicle while capturing images of another moving vehicle.

[0037]

[0045] In these situations, where the camera itself may be moving, traditional analytical tools used by weight-based measurement modes to calculate and recalculate weights may be ineffective. Traditional grid measurement modes are not well-suited to take into account any movement in the camera itself when predicting weights for future images. This is because the camera's motion characteristics (e.g., direction, speed, etc.) may not be known or may not be predictable by traditional systems. Thus, traditional mechanisms may not be able to predict or accurately determine exposure settings fast enough to keep up with rapid scene changes caused by camera movement and / or movement associated with the apparatus or device with which the camera resides or communicates.

[0038]

[0046] Exemplary aspects of the present disclosure address the above-described limitations of conventional techniques. In one or more examples, a current scene of a camera or image sensor may include a current image to be captured by the image sensor at a current time. The scene may change from a current scene at the current time (e.g., a first time instance) to a future scene at a future time after the current time (e.g., a second time instance following the first time instance). The change from the current scene to the future scene may be caused at least in part by movement of the image sensor, a camera including the image sensor, and / or a device or apparatus including a camera further including an image sensor. The camera's exposure setting for a current image of a current field of view may be referred to as a current exposure setting, while the camera's exposure setting for capturing a future image of a future scene may be referred to as a future exposure setting.

[0039]

[0047] According to example aspects, systems and methods are described for determining future exposure settings for capturing future images of a future scene. In some examples, a current exposure setting for capturing a current image of a current scene is determined. One or more motion characteristics associated with the image sensor may also be determined. Based on the motion characteristics, a location of a portion of the current image to be used to determine a future exposure setting for the image sensor may be predicted. Future exposure settings (e.g., exposure settings not yet determined) to be utilized for capturing future images of the future scene, which may be used to capture future images of the future scene at a future time, may be determined based on the predicted portion of the current image.

[0040]

[0048] In various examples, calculations for determining future exposure settings may begin based on a prediction of the location of a portion of the current image before the image sensor's field of view reaches the future scene. In this way, processing time for determining future exposure settings for capturing future images once the camera's field of view reaches the future scene may be minimized or eliminated.

[0041]

[0049] In one or more examples, the motion characteristics may include one or more of a camera's direction of travel or speed. In exemplary aspects, the motion characteristics associated with the image sensor can be determined using different techniques, which may be used alone or in any combination. In some examples, the motion characteristics may be determined based at least in part on a motion measurement unit (e.g., a gyroscope, an accelerometer, and / or other inertial measurement unit) that is part of or in communication with the camera. In some examples, the motion characteristics may be determined based at least in part on input to a steering system that communicates with the image sensor. In some examples, the motion characteristics may be determined based at least in part on one or more regions of interest identified in the current image.

[0042]

[0050] 1 is a block diagram of an exemplary image processing system 100 configured according to aspects of the present disclosure. Image processing system 100 may be a component of one or more devices having one or more image capture devices, a component of one or more image capture devices, and / or a component of one or more video capture devices, such as an autonomous vehicle having one or more cameras, a digital camera, a digital video camera, a mobile phone having one or more cameras, a tablet having one or more cameras, a personal computer having one or more cameras, a virtual reality (VR) device (e.g., a head-mounted display (HMD), a head-up display (HUD), or other VR device) having one or more cameras, an augmented reality (AR) device (e.g., AR glasses or other AR device) having one or more cameras, a gaming system having one or more cameras, or other suitable device.

[0043]

[0051] Image processing system 100 may include various components, among which are illustrated image sensor 110, application platform (AP) 120, and motion measurement unit 130. While not explicitly illustrated, it will be understood that image processing system 100 may include additional components, such as one or more memory devices (e.g., RAM, ROM, cache, buffers, and / or other memory components or devices) and / or other processing devices. The components of image processing system 100 may include and / or be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable electronic circuits (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuitry), and / or may include and / or be implemented using computer software, firmware, or any combination thereof, for performing various operations described herein.

[0044]

[0052] Image sensor 110 may include one or more components not specifically illustrated. For example, image sensor 110 may include components such as a tone adjustment engine, a lens shading correction unit, and a linear statistical engine, among other image processing components. In some examples, image sensor 110 may capture an image by capturing light that makes up the pixels of the image. For example, the digital image sensing mechanism of image sensor 110 may capture an image received through one or more lenses (not separately illustrated). The captured image may be digitized and stored as image data in a local memory or storage medium at image sensor 110 (and / or in another part of image processing system 100) and / or may be stored remotely and accessed by image processing system 100 from the remote storage. Various formats for the image data are possible. In one example, the image data may represent the captured image in a red-green-blue (RGB) color space. For example, an RGB image includes a red component, a green component, and a blue component for each pixel (although the brightness of the image is expressed in RGB values). In another example, the image data may represent a captured image in a luma, chroma blue, chroma red (YCbCr) color space (where the luma and chroma components are separated). For example, a YCbCr image includes a luma component, a chroma blue color component, and a chroma red color component for each pixel. Other examples of image color spaces are known and may be utilized when performing the techniques described herein.

[0045]

[0053] In some examples, the image data may be processed in one or more components of the image sensor 110, such as the tone adjustment engine, lens shading correction unit, linear statistics engine, etc. described above. The tone adjustment engine may perform tone mapping of the image data, where a set of input colors in the image data's high dynamic range is mapped to a set of output colors in a lower dynamic range. Tone mapping may be used to address the strong contrast ratio reduction from the captured radiance of natural scenes to the displayable range, while preserving image detail and color appearance important for perceiving the originally captured content. The lens shading correction unit may perform lens shading correction or compensation on the image data to correct for or compensate for shading effects that may be introduced into the captured image. The linear statistics engine may, for example, sort the image data into different bins based on the total number of brightness levels (e.g., measured in lumas) in each bin and generate linear image statistics based on the total number. In some examples, the linear image statistics may include a histogram of linear statistics.

[0046]

[0054] The AP 120 is shown to include a grid processing engine 122 that can be configured to implement one or more grid measurement functions discussed above. The AP 120 also includes an AEC engine 124 configured to determine exposure settings for an image based on, for example, the grid measurements and possibly other information, such as linear image statistics (e.g., a histogram of linear statistics) from the image sensor 110. The AP 120 and the AEC engine 124 will be described in further detail in the following sections. In some examples, the AP 120 can receive image data from the image sensor 110. In some examples, the image data can be processed through one of the components, such as tone mapping being implemented in the image sensor 110, before the image data is received by the AP 120. In some examples, the AP 120 can also receive linear image statistics (e.g., a histogram) from the image sensor 110. The AP 120 can also receive one or more motion characteristics from the motion measurement unit 130.

[0047]

[0055] The motion measurement unit 130 may include one or more motion detection and / or measurement mechanisms to detect and measure motion of one or more components of the image processing system 100. For example, the motion measurement unit 130 may be mounted on a common platform shared by the image sensor 110 such that the motion measurement unit 130 is associated with and can measure motion of the image sensor 110. The measured motion may reveal motion characteristics associated with the image sensor 110. These motion characteristics may include one or more of heading or direction, linear velocity, linear acceleration, angular velocity, angular acceleration or rotational rate, any suitable combination thereof, and / or other motion characteristics. In some examples, the motion measurement unit 130 may include an inertial measurement unit that can detect linear acceleration using one or more accelerometers and / or rotational rate using one or more gyroscopes. In some examples, the motion measurement unit 130 may also (or alternatively) include a magnetometer to provide an orientation reference. In some examples, the motion measurement unit 130 may include one or more accelerometers, gyroscopes, and / or magnetometers for each axis of motion characteristic to be measured in each of three axes (referred to as pitch, roll, and yaw). For example, the motion measurement unit 130 may determine the rate and degree of movement of the image sensor 110 along a horizontal axis (e.g., pitch), a longitudinal axis (e.g., roll), and / or a vertical axis (e.g., yaw).

[0048]

[0056] In some examples, one or more motion characteristics measured by motion measurement unit 130 may be provided to AP 120. The one or more motion characteristics may be used to predict the location of a portion of a current image of a current scene to be used to determine future exposure settings for capturing a future image of the future scene by image sensor 110. In some examples, which will be described in further detail below, the images captured by image sensor 110 may differ based on variations in the field of view or scene seen by image sensor 110. For example, a scene change may result in variations between the current exposure settings for a current image of the current scene view and a future image of the future scene. In some examples, the process of estimating future exposure settings for a future image can be initiated, and in some cases, completed, before the field of view of image sensor 110 reaches or includes the future scene.

[0049]

[0057] FIG. 2A is a schematic diagram illustrating an image 200 that can be processed using, for example, image processing system 100. FIG. 2A illustrates one or more example regions of interest within image 200. For example, region 202 of image 200 is shown, although region 202 may be defined based on input received from a user or a client device. In one example, the input may be touch input, although a touch screen display associated with a camera may display an image previewed through the camera lens. Image processing system 100 may receive input from the touch screen display based on region 202 associated with the touch input. In another example, region 204 of image 200 may include a subject's face. A face detection mechanism may be used to identify features, such as faces, of one or more people being photographed by the camera. Regions of image 200, such as region 202 and region 204, may include regions of interest (ROIs) to be enhanced. In some examples, enhancing one or more regions of interest of an image may include optimizing exposure settings for the one or more regions of interest. In some examples, the grid processing engine 122 and / or the AEC engine 124 may be configured to adjust exposure settings for one or more regions of interest.

[0050]

[0058] In some examples, the lattice processing engine 122 may be used to determine a brightness level for the image 200, as discussed in the following sections. The lattice processing engine 122 and / or any other mechanism may be used to obtain brightness levels for various portions that make up the image 200. In some examples, an exposure setting for the image 200 may be determined by taking one or more regions of interest into account. For example, the AEC engine 124 may determine weights to be applied to regions of interest, such as regions 202, 204, when calculating exposure settings for the image 200. In one example, a weight w1 may be assigned to a region of interest, such as region 202 or region 204. A weight w2 may be assigned to the remaining frames of the image 200. A weighted average may be performed to obtain a brightness level for the image 200, as follows: Image Brightness = (ROI Brightness * w1 + Frame Brightness * w2) / (w1 + w2), where ROI brightness refers to the brightness of the ROI (e.g., region 202 or 204) and frame brightness refers to the brightness of the rest of the image 200 other than the ROI. In some examples, the weights may be calculated as a function of the size of the ROI and / or determined using other mechanisms.

[0051]

[0059] FIG. 2B illustrates a schematic representation of a grid 250 that may be used by grid processing engine 122 to determine exposure settings for an image, such as image 200 or any other image. In some examples, image 200 in FIG. 2B may be a current image of a current scene captured by image sensor 110. In FIG. 2B, grid elements of grid 250 that may be used to calculate exposure settings for image 200 are illustrated. Grid 250 may be configured according to a grid measurement mode used by image processing system 100. In some examples, grid 250 may include weights to be applied to statistics, such as brightness levels of various grid elements. In some examples, the statistics may be derived from a Bayer filter in conjunction with data acquired from the image sensor. The Bayer filter may include an arrangement of photosensors corresponding to the grid elements of grid 250. For example, one or more photosensors may be associated with each grid element. The one or more photosensors corresponding to the grid elements may use filtering techniques to detect the brightness of the region of image 200 corresponding to the grid element. In some cases, the Bayer filter may also use tone adjustment information and linear statistical histograms in determining brightness levels for the grid elements. In some examples, the dimensions or size of grid 250, the number of grid elements in grid 250, etc. may be predefined, or these aspects may be based on characteristics of the image. In some examples, the configuration of grid 250 may be based on the grid measurement mode used by grid processing engine 122.

[0052]

[0060] In some examples, the statistical values ​​of grid elements may be emphasized based on weights. In some examples, weights may be assigned to regions of interest as described with reference to FIG. 2A . In some examples, weights may be assigned to grid elements based on a grid measurement mode. For example, the brightness levels of grid elements of grid 250 obtained using a Bayer filter may be weighted according to a center-weighted measurement mode. In grid 250, statistics such as brightness levels about center 252 are emphasized. In some examples, the size and number of grid elements of grid 250 may be determined based on the capabilities of grid processing engine 122. For example, a grid composed of a large number of grid elements may provide brightness levels at a finer granularity (which may lead to more accurate statistics), but the power consumption for processing the statistics of such a grid may be adversely affected. For example, the power consumption of grid processing engine 122 for processing a grid with a large number of grid elements is higher than the power consumption of grid processing engine 122 for processing a grid with a fewer number of grid elements. Thus, the number of elements in the grid 250 may be chosen for a particular measurement mode to balance the accuracy of the statistics and the capabilities of the grid processing engine 122 .

[0053]

[0061] More specifically, grid 250 is shown in the illustrative example of FIG. 2B as having N rows and M columns, although N and M may be chosen according to the metering mode and / or capabilities of grid processing engine 122 or other factors. Grid elements of grid 250 may be associated with regions of image 200 for which exposure settings should be calculated using grid 250. Within grid 250, the portion identified as center 252 corresponds to a grid element that may be used to calculate exposure settings for a focus region or region of interest of the image. As previously mentioned, the grid elements may correspond to different regions of the image (e.g., a spot other than center 252 in spot metering may correspond to a focus region of the image, while one or more grid elements in a matrix metering mode may correspond to a region of interest, such as regions 202, 204, of image 200). Weights for grid elements corresponding to focus regions may, in some examples, be based on a grid metering technique. As such, while center 252 is illustrated as comprising a contiguous block of grid elements, it will be understood that aspects of the present disclosure are not limited to any particular location or configuration of foci in other examples.

[0054]

[0062] In some examples, the grid processing engine 122 can obtain statistics in terms of brightness (or luminance or luma) levels per grid element and apply weights to different grid elements as identified in grid 250. For example, the brightness levels can be in a luma value range of 0 to 255, with a value of 0 corresponding to the darkest possible pixel and a value of 255 corresponding to the brightest possible pixel. As previously described, the grid processing engine 122 can use a Bayer filter to obtain the statistics. The grid processing engine 122 can apply weights to the statistics based on the grid measurement mode. In some examples, grid elements for which greater emphasis or focus is desired can be assigned a higher weight. In grid 250, weights ranging from 0 to 8 are shown for the various grid elements. The weight for the grid element at the center 252 is shown to be the highest (weight value of 8). The highest weight for the grid elements at the center 252 means that the grid elements at the center 252 are assigned the highest focus, while the weight for the grid elements is gradually decreased toward the outer edges and corners of the grid 250 (e.g., from a value of 7 for the grid elements surrounding the center to values ​​of 0, 1, or 2 at the outer edges as shown).

[0055]

[0063] In some examples, normalized weighted statistics may be used by grid processing engine 122. For example, a grid element's statistical value or brightness level may be multiplied by the corresponding weight in grid 250 to obtain a weighted statistical value. A sum of the weights in grid 250 may be obtained. The normalized weighted statistical value may be calculated by dividing the weighted statistical value by the sum of the weights. The normalized weighted statistical value for a grid element may, in some examples, be used instead of the original brightness value to bring the normalized weighted statistical values ​​for various grid elements into a closer range.

[0056]

[0064] The AEC engine 124 may receive the statistics and weights (or, in some examples, normalized weighted statistics) from the grid processing engine 122 and adjust exposure settings for the image 200 based on the statistics and corresponding weights for the grid elements. For example, the AEC engine 124 may adjust exposure settings by determining a weighted brightness level for each grid element of the grid 250. In some examples, the weighted brightness level for the grid element may be determined based on the brightness level of the grid element and the weight for the grid element. Thus, the brightness level for a grid element may be emphasized by increasing the weight for the grid element. For example, if the weight for a region including one or more grid elements of the grid 250 is increased, the region is emphasized relative to other regions. Emphasizing a region may result in the brightness level for the region contributing more to the brightness level calculated based on the grid 250. For example, in the case of an image including patches of white snow, the brightness of the region of the grid 250 corresponding to the white snow will be higher. Enhancement of the area may result in the brightness of the captured image of the white snow being exposed relative to the rest of the image, while the overall brightness of the captured image may be reduced. Similarly, if the image to be captured includes a dark tunnel, enhancement of the area of ​​the grid 250 that corresponds to the dark tunnel may reduce the brightness of the dark tunnel in the captured image, while increasing the overall brightness of the captured image.

[0057]

[0065] Thus, in some examples, the AEC engine 124 may emphasize one or more regions of the grid 250 that have a greater weight, while regions that have a lesser weight may not be emphasized. In an illustrative example, the AEC engine 124 may increase the emphasis for a region of the grid 250 that corresponds to the center 252 and decrease or leave the emphasis unchanged for a region of the grid 250 that corresponds to the outer edges of the grid 250.

[0058]

[0066] 3A and 3B illustrate schematic representations of motion-based variations in a grid 250. FIG. 3A illustrates the grid 250 discussed with reference to FIG. 2B and also includes an indication of a motion direction 304. The motion direction 304 represents movement in the field of view or scene seen by the camera or image sensor 110 used to capture the image for which exposure settings are calculated based on the grid 250. The direction of movement of the image sensor 110 can be any direction, such as horizontal, vertical, depth, or any combination thereof. In the illustrative example illustrated in FIGS. 3A and 3B, the motion direction 304 may be caused by horizontal movement of the image sensor 110, causing a corresponding horizontal movement in the scene seen by the image sensor 110. One or more motion characteristics of the movement, such as the motion direction 304, the speed at which the movement occurs, etc., may be obtained from the motion measurement unit 130 in some examples. In some examples, a scene change based on direction 304 may result in a change in weights for grid elements to be used in controlling exposure settings for capturing images in the changing scene. For example, if grid 250 represents weights for grid elements of current image 200 of a current scene, a scene change associated with motion direction 304 may result in that future image being captured by image sensor 110.

[0059]

[0067] FIG. 3B illustrates a grid 300 for controlling future exposure settings for capturing future images of a future scene based on movement. Movement is defined by one or more movement characteristics, such as a direction of movement 304, a speed of movement, and / or other movement characteristics. In the future scene, there may be a new area of ​​focus for images to be captured by the image sensor 110. According to some examples, sections 306 and 308 are illustrated in FIG. 3B. Section 306 indicates a portion of the grid 300 associated with the current image that may be used to calculate future exposure settings for future images of the future scene. Section 308 indicates an area of ​​the grid 300 associated with the current image that may be excluded or no longer used to calculate future exposure settings.

[0060]

[0068] In one or more embodiments, the location of a portion of the current image for determining future exposure settings may be predicted using one or more motion characteristics obtained from the motion measurement unit 130. For example, the location of section 306 of grid 300 to be used to determine future exposure settings may be predicted based on one or more motion characteristics, including motion direction 304. In the illustrated example, grid processing engine 122 may calculate weights for future images of a future scene based on predicting the location of section 306 of grid 300 to be used to determine future exposure settings. Correspondingly, predicting the location of a portion of the current image to be used to determine one or more future exposure settings may also include excluding certain regions of the current image from being used to determine one or more future exposure settings. For example, grid processing engine 122 may calculate weights for future images based on excluding section 308 from being used to determine one or more future exposure settings.

[0061]

[0069] In some examples, the motion measurement unit 130 can determine one or more motion characteristics that can be used in predicting the location of section 306 to be used to determine one or more future exposure settings and to exclude section 308 from being used to determine one or more future exposure settings. For example, the motion measurement unit 130 can include an accelerometer to detect the speed of movement, a magnetometer to detect the direction of travel (e.g., motion direction 304), and / or a gyroscope to measure the rate of rotation in degrees per second along several axes (e.g., three axes in the x-, y-, and z-directions). For example, panning a camera to the left (i.e., motion direction 304) from the current scene at a constant speed can result in the center of a future image at a future time being located a determinable or predictable distance to the left of the center of the current image 200 shown in FIG. 2B . For example, the center 302 to be highlighted within section 306 to determine future exposure settings can be predicted to be at a location that is a determinable distance to the left of the center 252 of the grid 200 shown in FIG. 2B .

[0062]

[0070] More specifically, weights for grid elements of grid 300 may be calculated based on a prediction of the location of section 306 of grid 300 that will be used to determine future exposure settings for capturing future images. For example, the weights may be determined based on a prediction that section 306 will be located within the left portion of grid 300, corresponding to a movement of center 352 to the left from center 252. Because section 306 is predicted to be used to calculate weights for future exposure settings, grid elements within section 306 may be weighted more heavily. Conversely, section 308 is predicted to be excluded from being used to calculate weights for future exposure settings, and correspondingly, grid elements in section 308 may be weighted less heavily. In some examples, weights for center 302 can be emphasized by determining their maximum value (e.g., 8), and weights for the remaining grid elements in section 306 may be recalculated according to the selected measurement mode. The weights for the grid 300, having weights for the grid elements in sections 306 and 308 as described above, may be calculated in anticipation of a future field of view that will be reached by the image sensor 110 at a future time based on one or more motion characteristics.

[0063]

[0071] A motion sensor in motion measurement unit 130 may be used to determine one or more motion characteristics. In some examples, the motion sensor may provide information that may be used in predicting the location of section 306. For example, the motion sensor may reveal whether a moving camera is accelerating or decelerating, and AEC engine 124 may be tuned based on this. Correspondingly, during periods of acceleration, less conservative tuning may result in an estimation that center 302 is shifted to the left edge of grid 300 (with respect to center 252), while more conservative tuning may result in an estimation of a smaller shift of center 302.

[0064]

[0072] In one example, grid processing engine 122 may determine grid measurement weights, as illustrated in grid 250, for image data provided by image sensor 110 at the current time. Grid processing engine 122 may calculate weights for grid elements of grid 250, with center 252 highlighted, to determine a current exposure setting for a current image of a current scene to be captured by image sensor 110. Grid 250 may include weights based on the center-weight-based measurement mode discussed above. A camera including image sensor 110 may already be moving at the current time or may begin moving at the current time. Motion measurement unit 130 may measure the movement of image sensor 110 and provide one or more motion characteristics based on the movement to grid processing engine 122. Grid processing engine 122 may receive one or more motion characteristics (e.g., including a direction of movement 304 and a speed of travel along the direction of movement 304) from motion measurement unit 130.

[0065]

[0073] The grid processing engine 122 may determine the grid 300 for a future time later than the current time, where the location of section 306 of the grid 300 to be used to determine future exposure settings for capturing future images at the future time is predicted based on one or more motion characteristics. For example, the distance the camera travels may be determined based on speed and direction. The distance the camera travels may be used to estimate a scene change from the current scene to the future scene during the time period between the current time and the future time. The grid processing engine 122 may determine or estimate that the center 302 to be emphasized for calculating the future exposure setting field of view will be located at a distance offset from the center 252, and the grid processing engine 122 may determine weights for the grid elements of section 306 and section 308 of the grid 300 accordingly.

[0066]

[0074] In some examples, grid processing engine 122 may begin processing to determine weights for grid elements of grid 300 before the future time. In some examples, grid processing engine 122 may complete processing to determine weights for grid elements of grid 300 before the future time, such that the weights for grid elements of grid 300 are already determined based on camera motion before the camera's field of view reaches the future scene. Thus, processing delays in calculating weights for future exposure settings when the camera's field of view reaches or includes the future scene may be eliminated or minimized.

[0067]

[0075] In the above examples, it is assumed that the predictions or estimates involved in calculating the weights for the grid elements of grid 300 are accurate. To verify the predictions, image processing system 100 may include a mechanism for verifying whether the camera followed a trajectory indicated by one or more motion characteristics. For example, a future scene may be determined at a future time and compared to an estimated future scene. If the estimated future scene substantially matches the actual future scene at the future time, the weights calculated by grid processing engine 122 for the grid elements of grid 300 may be retained. If not, grid processing engine 122 may discard the estimated weights and recalculate the weights.

[0068]

[0076] 4A and 4B illustrate schematic representations of a grid metering technique using a spot metering mode and based on one or more motion characteristics. FIG. 4A illustrates a grid 400 used to calculate exposure settings for an image to be captured by a camera. The grid 400 may include N1 rows and M1 columns of grid elements. In an illustrative example, the camera may be a point of view (POV) camera or a dash camera disposed on a vehicle and configured to capture a scene in the vehicle's direction of travel. Weights (not specifically shown) for the grid elements of the grid 400 may be calculated by the grid processing engine 122. In one example, the grid processing engine 122 may utilize the spot metering mode to calculate the weights for the grid 400 based on one or more motion characteristics associated with the camera.

[0069]

[0077] The point of focus may change as the vehicle moves. Based on one or more motion characteristics, such as the vehicle's speed, direction of travel, etc., grid processing engine 122 can calculate weights for determining future exposure settings for capturing future images of a future scene in accordance with the present disclosure. For example, spot 402 comprises a group of one or more grid elements of grid 400, where spot 402 indicates an area of ​​grid 400 that is to be image-enhanced. If grid 400 represents a grid to be used to determine a current exposure setting for a current image by emphasizing spot 402, the location of a portion of grid 400, such as section 406, that will be used to determine a future exposure setting may be predicted based on one or more motion characteristics associated with the vehicle (and, correspondingly, associated with a camera mounted on the vehicle). Correspondingly, section 408 of grid 400 represents an area of ​​the current image that will be excluded from use in determining one or more future exposure settings.

[0070]

[0078] For example, the motion measurement unit 130 can receive information about motion characteristics associated with the vehicle (e.g., from one or more accelerometers, gyroscopes, etc. used by the vehicle) in addition to, or as an alternative to, one or more inertial measurement units, which may be components of the motion measurement unit 130. The motion measurement unit 130 can provide one or more motion characteristics to the lattice processing engine 122 based on any combination of the motion measurements. The lattice processing engine 122 can recalculate weights for emphasizing a new spot (not shown) in the section 406 based on an estimated point of focus in a future scene. The section 406 is predicted to be used to determine future exposure settings. The location of the section 406 can be predicted based on one or more motion characteristics. The lattice processing engine 122 can assign a higher weight to a new spot in the section 406 that corresponds to a point of focus in a future image. The AEC engine 124 can adjust the exposure settings based on the weights from the lattice processing engine 122. For example, based on one or more motion characteristics, the AEC engine 124 may increase the brightness levels for grid elements of spots in section 406 that have a greater weight to determine future exposure settings for capturing future images of future scenes.

[0071]

[0079] FIG. 4B illustrates a grid 450 used to calculate exposure settings for an image to be captured by a camera. The grid 450 may include N rows and M columns of grid elements. In an illustrative example, the camera may be a point-of-view (POV) camera or dash camera disposed on a vehicle and configured to capture a scene in the vehicle's direction of travel. In FIG. 4B, the direction of travel may include an off-center location in a future scene with respect to a current scene seen by the camera. A marker 454 is identified in the image underlying the grid 450 and includes an arrow mark. The marker 454 provides an indication of a curve in the road along which the vehicle is traveling. The marker 454 may be identified by a graphics processing unit (GPU) (not specifically shown) of the image processing system 100. The graphics processing unit may include artificial intelligence or machine learning capabilities for detecting markers such as marker 454 and predicting one or more motion characteristics accordingly. The motion characteristics predicted using the markers 454 may, in some examples, be used together with motion characteristics obtained from the motion measurement unit 130. For example, the motion characteristics predicted using the markers 454 may be provided by the graphics processing unit to the motion measurement unit 130, where the motion measurement unit 130 may combine motion characteristics from the inertial measurement sensors of the motion measurement unit 130 with motion characteristics from the graphics processing unit when estimating one or more motion characteristics for the camera. In some examples, the motion measurement unit 130 may receive detailed instructions about progress from a navigation system (e.g., based on a global positioning system) and combine the detailed instructions with input from the graphics processing unit. For example, the navigation system may have information about an upcoming right turn in the direction of travel based on previously mapped directions, and the graphics processing unit may map the direction of travel to the current scene (e.g., an upcoming left turn or intersection) as viewed by the camera.The motion measurement unit 130 can determine the location of the right turn based on the navigation system and the graphics processing unit, and the grid processing engine 122 can assign a higher weight to expose images in the right turn direction. The AEC engine 124 can correspondingly emphasize the right turn direction based on the higher weight.

[0072]

[0080] 4B , a curve in the road, as indicated by marker 454, may result in the focus in the vehicle's future field of view being located at spot 452. Weights (not specifically shown) for the grid elements of grid 450 may be calculated by grid processing engine 122. In one example, grid processing engine 122 may utilize a spot metering mode to calculate weights for grid 450 based on one or more motion characteristics obtained from motion measurement unit 130, which may include motion characteristics from the graphics processing unit as discussed above. For example, spot 452 comprises a group of one or more grid elements of grid 450, where spot 452 indicates a greater weight to be applied to the point of estimated focus of the vehicle's driver.

[0073]

[0081] Based on one or more motion characteristics, such as speed, direction of travel, etc., as obtained from the motion measurement unit 130 and the graphics processing unit, the grid processing engine 122 can predict the location of sections 456 of the grid 450 that will be used to determine future exposure settings for future images and sections 458 that will be excluded from determining future exposure settings for future images. The grid processing engine 122 can calculate weights for future exposure settings for future images by assigning greater weights to spots, such as spot 452, using sections 406 to calculate weights for future exposure settings. The AEC engine 124 can adjust exposure settings by prioritizing points of focus. For example, based on one or more motion characteristics, the AEC engine 124 can increase brightness levels for grid elements of spots that have greater weights for future images.

[0074]

[0082] FIG. 5 illustrates a schematic representation of a grid measurement technique based on one or more motion characteristics using a center-weight-based measurement mode. In FIG. 5, a grid 500 is illustrated that can be used to calculate exposure settings for an image to be captured by a camera. A steering wheel 506 is also illustrated that can be used with the motion measurement unit 130 to obtain one or more motion characteristics associated with the camera. The grid 500 can include N rows and M columns of grid elements. In an illustrative example, the camera can be a point-of-view (POV) camera or dash camera disposed on a vehicle and configured to capture a scene in the vehicle's direction of travel. The steering wheel 506 can be used to control the vehicle's direction of travel. In one example, the steering direction 504 can correspond to a change in the vehicle's turn or direction caused by turning the steering wheel 506 in the steering direction 504. In one example, the steering direction 504 can cause a corresponding change in the scene viewed by the camera. In some examples, the grid processing engine 122 may calculate weights for the grid 500 to be used to calculate future exposure settings based at least in part on the steering direction 504 .

[0075]

[0083] In some examples, sensors or actuators can detect motion characteristics, such as a change in the vehicle's orientation, caused by the rotation of the steering wheel 506. The steering direction 504 can be calculated based on the detected motion characteristics and the vehicle's traveling speed and / or other possible motion characteristics obtained from the motion measurement unit 130. In some examples, the rotation of the steering wheel 506 can be provided as an input to the motion measurement unit 130, which can determine one or more motion characteristics, including the steering direction 504. For example, the motion measurement unit 130 can combine motion characteristics based on the rotation of the steering wheel 506 with motion characteristics based on one or more inertial measurement sensors of the motion measurement unit 130 to generate motion characteristics to be used by the lattice processing engine 122.

[0076]

[0084] 5, a steering direction 504 (caused at least in part by a rotation of a steering wheel 506) may lead to a change in the scene, in which case the grid 500 may be used to determine exposure settings for images in the changing scene. For example, weights for the grid elements of the grid 500 may be calculated to emphasize the center of the image to be captured.

[0077]

[0085] In some examples, based on one or more motion characteristics, the location of section 506 to be used to determine future exposure settings for future images may be predicted based on the one or more motion characteristics. Correspondingly, section 508 that will be excluded from grid 500 for determining future exposure settings may also be predicted. For example, future exposure settings may be determined by emphasizing the center of the future image. In some examples, the weight for center 502 may be estimated to be a maximum value (e.g., 8), and weights for the remaining grid elements in section 506 may be smaller. The weight for section 508 may be determined (e.g., to be a low value) based on a prediction that section 508 will be excluded from determining future exposure settings. Weights for sections 506 and 508 of grid 500 may be calculated in anticipation of future images that will be captured by a camera at a future time based on one or more motion characteristics. The AEC engine 124 may adjust the exposure settings based on the center 502 being more heavily weighted. For example, based on one or more motion characteristics, the AEC engine 124 may increase the brightness level for the region of the future image that corresponds to the center 502 .

[0078]

[0086] 6A, 6B, 6C, and 6D illustrate schematic diagrams of variations in a camera's field of view based at least in part on motion characteristics associated with the camera. FIG. 6A shows a current scene 600 including a tunnel 602. The current scene 600 may be viewed by a vehicle camera, similar to the example discussed above. In the illustrative example of FIG. 6A, the current scene 600 is illustrated as mapped to degrees on the camera's horizontal field of view (FOV) scale. For example, FIG. 6A illustrates the current scene 600 mapped to 90 degrees on the x-axis, from −45° to +45°. The tunnel 602 is illustrated as being within a region of the current scene 600 mapped to −20°. The tunnel 602 is referred to as being located within a portion of the current scene 600 that corresponds to a peripheral location.

[0079]

[0087] FIG. 6B illustrates a 2D mapping of the height of an image of interest, such as tunnel 602 (e.g., image height [mm] shown on the horizontal axis) to the field of view (FOV) in degrees (e.g., FOV [°] shown on the vertical axis). As illustrated in FIG. 6A, a two-dimensional (2D) image may extend radially within the radial FOV in both the horizontal axis (x-axis) and the vertical axis (y-axis). The mapping of a region of an image, such as tunnel 602, to the current scene 600 (e.g., horizontal FOV) in degrees may be based on characteristics of the camera (e.g., characteristics of the lens or image sensor used to view the image of the current scene 600). In some examples, the image height may be converted from a unit of length, such as mm, to a different unit, such as the number of pixels in the vertical or height direction of tunnel 602.

[0080]

[0088] FIG. 6C illustrates an exemplary embodiment of a scene change of the camera 606 caused due to the movement of the camera 606. In FIG. 6C, the current scene 600 is illustrated from the perspective of the camera 606, with the tunnel 602 located in a peripheral location, i.e., at a −20° position in the current scene 600 as in FIG. 6A. In FIG. 6C, the movement of the camera 606 is also illustrated in the direction 604. In one illustrative example, the movement of the camera 606 may be caused due to the rotation of the steering wheel 506, as discussed with reference to FIG. 5. In various examples, the movement of the camera 606 may be based on any rotation in the direction 604, as shown in FIG. 6C. One or more motion characteristics associated with the camera 606, such as the rotation in the direction 604, may be determined by the motion measurement unit 130. In some examples, any other mechanism, such as a graphics processing unit or a steering wheel rotation detection system, may be used together with the motion measurement unit 130 to determine one or more motion characteristics associated with the camera 606.

[0081]

[0089] To determine current exposure settings for capturing a current image of the current scene 600, the grid processing engine 122 may generate a grid with weights that emphasize exposure settings for the tunnel 602. For example, if there is a significant contrast ratio between the tunnel 602 and the rest of the current image captured by the camera 606 in the current scene 600, the grid processing engine 122 may assign a higher weight to grid elements that correspond to the tunnel 602 than grid elements that correspond to other portions of the image (e.g., using a spot metering mechanism).

[0082]

[0090] 6D illustrates a future scene 650 that will be viewed by camera 606 based on movement along rotation 604. In FIG. 6D, tunnel 602 is illustrated as being centered or at a 0° position in future scene 650. In an exemplary embodiment, the location of tunnel 602 in future scene 650 may be predicted based on rotation 604. In an exemplary embodiment, a future exposure setting for future scene 650 may be determined based on predicting the location of a portion of the current image that will be used to determine the future exposure setting. Calculations for determining the future exposure setting may be performed (e.g., started and / or completed) before the field of view of camera 606 reaches or includes future scene 650. The following example illustrates a technique for determining future exposure settings.

[0083]

[0091] In an illustrative example, the camera 606 may stream video footage at a rate of 30 frames per second (fps). Thus, the field of view of the camera 606 may include a new frame every 1 / 30 seconds (or 1 / 30 seconds). Based on the camera's movement speed (angular and / or linear velocity) relative to the 30 fps rate, the field of view may change to include new frames at a rate less than one frame per 1 / 30 seconds, equal to one frame per 1 / 30 seconds, or greater than one frame per 1 / 30 seconds. In one example, determining future exposure settings may involve predicting the location of one or more regions of the current image that will be used to determine future exposure settings. For example, predicting the location of one or more regions of the current image may include predicting a portion of the image that will be at the center or 0° position when the camera's field of view reaches or includes the future scene 650. In an illustrative example in which a camera streams video images at a rate of 30 fps, the location of a portion of the current image (e.g., tunnel 602) that will be in the center of the camera's field of view in 1 / 30 of a second can be estimated or predicted based on the speed at which the camera is moving.

[0084]

[0092] In one example, a sensor such as a gyroscope in the motion measurement unit 130 can be used to determine the angular velocity ω [° / s] of the rotation 604 of the camera 606 along the yaw or x-axis, which can be calculated as a displacement in degrees: Displacement [°] = ω * 1 / 30. The displacement in degrees can be converted to a displacement in pixels, i.e., displacement [number of pixels]. The displacement in pixels can reveal an image location in the current image that will be centered in a future image 1 / 30 seconds later. In the illustrated example, the tunnel 602 can appear at the center of the camera 606 after a certain multiple of 1 / 30 seconds based on the displacement calculated above. Thus, for a future scene 650 in which the center of the camera 606 includes the tunnel 602, future exposure settings can be calculated to highlight the tunnel 602 in future images of the future scene 650.

[0085]

[0093] In some examples, motion measurement unit 130 may use similar calculations to those described above for other types of inertial measurements. For example, an accelerometer may be used to determine the speed or velocity of the camera for linear motion, similar to a gyroscope used to determine angular velocity.

[0086]

[0094] In some examples, the motion measurement unit 130 can use a navigation system to provide steering and velocity measurements. Based on the vehicle's turning radius r provided by the steering and the linear velocity v, the angular velocity ω can be calculated as "r / v". Based on the angular velocity ω, the deviation [°] can be obtained as ω*1 / 30.

[0087]

[0095] Thus, in an exemplary embodiment, future exposure settings for future images of a future scene of a camera may be calculated before the camera's field of view reaches or includes the future scene. The performance of an image processing system for processing images captured by a camera can be improved by minimizing or eliminating the processing time involved in calculating exposure settings when the camera's field of view reaches or includes the future scene. Performance improvements may be significant in instances where fast (linear / angular) camera motion characteristics may be involved.

[0088]

[0096] Accordingly, it will be appreciated that example embodiments include various methods for performing the processes, functions, and / or algorithms disclosed herein. For example, FIG. 7 illustrates a process 700 for processing one or more images according to embodiments of the present disclosure. In some examples, process 700 may be implemented by image processing system 100.

[0089]

[0097] At block 702, process 700 includes determining one or more current exposure settings for the current image of the current scene at the current time. For example, the one or more current exposure settings for the current image of the current scene may be calculated using a grid 250, with its center 252 emphasized with a greater weight. The grid 250 may be used by the grid processing engine 122 to determine current exposure settings for capturing the current image 200 of the current scene as seen by the image sensor 110 at the current time, as described with reference to FIG. 2B .

[0090]

[0098] At block 704, process 700 includes determining one or more motion characteristics associated with the image sensor. In some examples, the one or more motion characteristics from the motion measurement unit 130 may be obtained by the grid processing engine 122. In some examples, the one or more motion characteristics may include one or more of a velocity or direction of travel of the image sensor 110. In some examples, the one or more motion characteristics may be based at least in part on an inertial sensor in communication with the image sensor 110. For example, the one or more motion characteristics may be determined by the motion measurement unit 130 using one or more inertial measurement sensors, such as an accelerometer, a gyroscope, or a magnetometer.

[0091]

[0099] In some examples, the one or more motion characteristics can be based at least in part on a steering system in communication with the image sensor. For example, the steering direction 504 of the steering wheel 506 can be obtained by one or more sensors connected to the steering wheel 506 as shown in FIG. 5. The steering direction 504 can be included in the calculation of the one or more motion characteristics by the motion measurement unit 130.

[0092]

[0100] In some examples, the one or more motion characteristics can be based at least in part on one or more regions of interest in the current image. For example, the graphics processing unit, possibly in combination with a navigation system as discussed above, can determine a region of interest, such as spot 452, based on a heading determined using marker 454 and input from the navigation system, as shown in FIG. 4B. In another example, the graphics processing unit can determine that a future point of focus can include a new spot, such as spot 452, based on the heading of the vehicle, as shown in FIG. 4A. In another example, the region of interest can include tunnel 602, which has a high contrast ratio, along with the rest of the image in the camera's field of view, as shown in FIGS. 6A-6D.

[0093]

[0101] At block 706, process 700 includes predicting, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor, the one or more future exposure settings being for capturing a future image of a future scene at a future time, the future time being subsequent to the current time. For example, based on the one or more motion characteristics, a location of a portion of grid 300 corresponding to a portion of current image 200, such as section 306, to be used to determine one or more future exposure settings may be predicted. Correspondingly, a section of grid 300 corresponding to an area of ​​current image 200, such as section 308, to be excluded from being used to determine one or more future exposure settings may also be predicted.

[0094]

[0102] In some examples, the grid processing engine 122 can determine weights to be applied to grid elements based on sections 306 and 308 of the grid 300. For example, in a center-weighted metering mode, the center 302 can be weighted more heavily in section 306 to be used to determine one or more future exposure settings. In some examples, grid elements corresponding to one or more regions of interest in an image can be weighted in a manner that emphasizes the regions of interest in the image. In some examples, the emphasis provided to regions with a greater weight can include adjusting exposure settings, such as brightness levels, for the regions.

[0095]

[0103] In some examples, the AEC engine 124 can use the weights determined by the grid processing engine 122 to determine exposure settings for the grid elements based on the estimated variation and the current exposure setting for the current image of the current field of view of the image sensor. For example, the AEC engine 124 may use the brightness level for the current image of the current field of view of the grid 250 and the weights determined by the grid processing engine 122 for the grid 300 using section 306 based on the motion direction 304 as shown in Figures 3A and 3B.

[0096]

[0104] At block 708, the process 700 includes determining one or more future exposure settings based on the predicted portion of the current image. For example, the AEC engine 124 may determine one or more future exposure settings for the future image using section 306 of the grid 300, with the center 302 being weighted more heavily, as discussed above.

[0097]

[0105] In some examples, one or more future exposure settings for a future image may be determined before the image sensor's field of view reaches or includes the future field of view. For example, the process of determining future exposure settings may begin, or in some cases may be completed, before the camera's field of view reaches or includes the future scene based on one or more motion characteristics associated with the image sensor 110. Depending on the processing speed of the AEC engine 124 and the camera's movement speed, for example, the future exposure settings for a future image may be ready to be applied to the future image before the camera's field of view reaches or includes the future scene.

[0098]

[0106] 8 illustrates an example computing device architecture 800 of an example computing device that can implement various techniques described herein. For example, computing device architecture 800 can implement one or more processes described herein. The components of computing device architecture 800 are shown in electrical communication with each other using connections 805, such as a bus. The example computing device architecture 800 includes a processing unit (CPU or processor) 810 and computing device connections 805 that couple various computing device components to the processor 810, including computing device memory 815, such as read-only memory (ROM) 820 and random access memory (RAM) 825.

[0099]

[0107] The computing device architecture 800 may include a cache of high-speed memory directly connected to the processor 810, close to the processor 810, or integrated as part of the processor 810. The computing device architecture 800 may copy data from the memory 815 and / or the storage device 830 to the cache 812 for quick access by the processor 810. In this manner, the cache 812 may provide performance improvements that avoid delays to the processor 810 while waiting for data. These and other modules may control or be configured to control the processor 810 to perform various actions. Other computing device memory 815 may be available for use as well. The memory 815 may include multiple different types of memory with different performance characteristics. The processor 810 may include any general-purpose processor and hardware or software services, such as service 1 832, service 2 834, and service 3 836 stored in the storage device 830, configured to control the processor 810 as well as dedicated processors, although software instructions are incorporated into the processor design. Processor 810 may be a self-contained system containing multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors may be symmetric or asymmetric.

[0100]

[0108] To enable user interaction with computing device architecture 800, input device(s) 845 can represent any number of input mechanisms, such as a microphone for audio, a touch-sensitive screen for gesture or graphic input, a keyboard, a mouse, motion input, voice, etc. Output device(s) 835 can also be one or more of numerous output mechanisms known to those skilled in the art, such as a display, a projector, a television, a speaker device, etc. In some examples, a multimodal computing device can allow a user to provide multiple types of input to communicate with computing device architecture 800. Communications interface 840 can generally govern and manage user input and computing device output. There is no limitation to operating on any particular hardware configuration, and thus the basic features herein can be easily substituted for improved hardware or firmware configurations as they are developed.

[0101]

[0109] The storage device 830 is non-volatile memory and can be a hard disk or other type of computer-readable medium capable of storing data accessible by a computer, such as a magnetic cassette, a flash memory card, a solid-state memory device, a digital versatile disk, a cartridge, a random access memory (RAM) 825, a read-only memory (ROM) 820, and hybrids thereof. The storage device 830 can include services 832, 834, 836 for the control processor 810. Other hardware or software modules are envisioned. The storage device 830 can be connected to the computing device connection 805. In one aspect, a hardware module that performs a specific function can include software components stored on a computer-readable medium in association with the necessary hardware components, such as the processor 810, the connection 805, the output device 835, etc., to perform that function.

[0102]

[0110] As used herein, the term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instructions and / or data. Computer-readable media may also include non-transitory media on which data can be stored and which do not include carrier waves and / or transitory electronic signals propagating wirelessly or over wired connections. Examples of non-transitory media include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. A computer-readable medium may store code and / or machine-executable instructions, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0103]

[0111] In some embodiments, computer-readable storage devices, media, and memories may include cable or wireless signals containing bitstreams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0104]

[0112] Specific details are provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, it will be understood by those skilled in the art that embodiments may be practiced without these specific details. For ease of explanation, in some cases, the technology may be presented as including individual functional blocks, including functional blocks comprising steps or routines in a device, device component, method embodied in software, or a combination of hardware and software. Additional components may be used other than those illustrated in the figures and / or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments.

[0105]

[0113] Individual embodiments may be described above as a process or method that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. The order of operations may also be rearranged. A process terminates when the operation is completed, but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0106]

[0114] The processes and methods according to the above-described examples may be implemented using computer-executable instructions stored on or otherwise available from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a processing device to perform a certain function or group of functions, or otherwise configure a general-purpose computer, a special-purpose computer, or a processing device to perform a certain function or group of functions. Portions of the computer resources used may be accessible over a network. The computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions during, information used in, and / or information created by, the methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, network-attached storage devices, etc.

[0107]

[0115] Devices implementing processes and methods according to these disclosures can include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and can take any of a wide variety of form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) to perform the necessary tasks can be stored on a computer-readable or machine-readable medium. A processor can perform the necessary tasks. Typical examples of form factors include laptop computers, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, standalone devices, etc. The functionality described herein can also be embodied in peripheral devices or add-in cards. Such functionality can also be implemented on a circuit board, in different chips, or different processes running within a single device, as further examples.

[0108]

[0116] The instructions, media for carrying such instructions, computing resources for executing them, and other structures for supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0109]

[0117] In the foregoing description, aspects of the application have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the application is not limited thereto. Accordingly, while exemplary embodiments of the application have been described in detail herein, it should be understood that the disclosed concepts may be variously embodied or employed in other ways, and that the appended claims are intended to be construed to include such variations except insofar as limited by the prior art. Various features and aspects of the above-described application examples may be used alone or in combination. Furthermore, the embodiments may be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive. For purposes of illustration, methods have been described in a particular order. It should be recognized that in alternative embodiments, methods may be performed in an order different from that described.

[0110]

[0118] Those skilled in the art will recognize that the less than sign ("<") and greater than sign (">") or terms used herein may be replaced with the less than or equal to sign ("≦") and greater than or equal to sign ("≧"), respectively, without departing from the scope of this description.

[0111]

[0119] When a component is described as being "configured to" perform a certain operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuit (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0112]

[0120] The phrase "coupled to" refers to any component that is directly or indirectly physically connected to another component and / or that is in direct or indirect communication with another component (e.g., connected to the other component over a wired or wireless connection and / or other suitable communication interface).

[0113]

[0121] Claim language or other language stating "at least one of" a set indicates that one member of the set or multiple members of the set satisfies the claim. For example, claim language stating "at least one of A and B" means A, B, or A and B.

[0114]

[0122] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled engineers may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0115]

[0123] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a wide variety of devices, such as a general-purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Any features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise a memory or data storage medium, e.g., random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, etc. The techniques may additionally or alternatively be realized at least in part by a computer-readable communications medium, such as a propagated signal or wave, that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer.

[0116]

[0124] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated circuit or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Accordingly, the term “processor” as used herein may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or apparatus suitable for implementing the techniques described herein.

Claims

1. 1. A method for processing one or more images, comprising: determining one or more current exposure settings for a current image of a current scene at a current time; determining one or more motion characteristics associated with the image sensor; predicting a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor based on the one or more motion characteristics, the one or more future exposure settings for capturing a future image of a future scene at a future time, the future time being subsequent to the current time; determining the one or more future exposure settings based on the predicted portions of the current image.

2. The method of claim 1 , further comprising determining the one or more future exposure settings before the field of view of the image sensor reaches the future scene.

3. 2. The method of claim 1 , wherein predicting the location of the portion of the current image to be used to determine the one or more future exposure settings comprises excluding an area of ​​the current image from being used to determine the one or more future exposure settings.

4. The method of claim 1 , wherein the one or more motion characteristics comprise one or more of a velocity or direction of travel of the image sensor.

5. 5. The method of claim 4, further comprising determining the one or more movement characteristics based at least in part on one or more of an inertial sensor or an input to a steering system in communication with the image sensor.

6. The method of claim 4 , further comprising determining the one or more motion characteristics based at least in part on one or more regions of interest of the current image.

7. Determining the one or more future exposure settings based on the predicted portion of the current image includes: predicting one or more regions of interest of the future image within the predicted portion; and adjusting exposure settings for the one or more regions of interest.

8. determining a grid comprising one or more grid elements associated with one or more regions of the future image within the predicted portion; The method of claim 1 , further comprising determining a weight for the one or more grid elements, wherein the one or more future exposure settings are based on the weight.

9. The method of claim 1 , wherein the one or more future exposure settings include variations in one or more current exposure settings of the current image.

10. The method of claim 1 , further comprising capturing the future image with the image sensor based on the one or more future exposure settings.

11. 1. An apparatus for processing one or more images, comprising: a memory configured to store the one or more images; Implemented in a circuit configuration, determining one or more current exposure settings for a current image of a current scene at a current time; determining one or more motion characteristics associated with the image sensor; predicting a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor based on the one or more motion characteristics, the one or more future exposure settings for capturing a future image of a future scene at a future time, the future time being subsequent to the current time; determining the one or more future exposure settings based on the predicted portion of the current image; and a processor configured to:

12. The processor: The apparatus of claim 11 , further configured to determine the one or more future exposure settings before the field of view of the image sensor reaches the future scene.

13. 12. The apparatus of claim 11, wherein predicting the location of the portion of the current image to be used to determine the one or more future exposure settings comprises excluding an area of ​​the current image from being used to determine the one or more future exposure settings.

14. The apparatus of claim 11 , wherein the one or more motion characteristics comprise one or more of a speed or direction of travel of the image sensor.

15. The processor:

12. The apparatus of claim 11, further configured to determine the one or more movement characteristics based at least in part on one or more of an inertial sensor or an input to a steering system in communication with the image sensor.

16. The apparatus of claim 11 , wherein the processor is further configured to determine the one or more motion characteristics based at least in part on one or more regions of interest in the current image.

17. Determining the one or more future exposure settings based on the predicted portion of the current image includes: predicting one or more regions of interest of the future image within the predicted portion; and adjusting exposure settings for the one or more regions of interest.

18. The processor: determining a grid comprising one or more grid elements associated with one or more regions of the future image within the predicted portion; The apparatus of claim 11 , further configured to determine weights for the one or more grid elements, and wherein the one or more future exposure settings are based on the weights.

19. The apparatus of claim 11 , wherein the one or more future exposure settings include variations in one or more current exposure settings of the current image.

20. The processor: The apparatus of claim 11 , further configured to capture the future image with the image sensor based on the one or more future exposure settings.

21. The device of claim 11 , further comprising a camera for capturing one or more images.

22. The apparatus of claim 11 , wherein the apparatus comprises a mobile device having a camera for capturing one or more images.

23. The device of claim 11 , further comprising a display for displaying one or more images.

24. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: determining one or more current exposure settings for a current image of a current scene at a current time; determining one or more motion characteristics associated with the image sensor; predicting a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor based on the one or more motion characteristics, the one or more future exposure settings for capturing a future image of a future scene at a future time, the future time being subsequent to the current time; and determining the one or more future exposure settings based on the predicted portions of the current image.

25. When executed by the one or more processors, the one or more processors 25. The non-transitory computer-readable medium of claim 24, further comprising instructions that cause determining the one or more future exposure settings before a field of view of the image sensor reaches the future scene.

26. 25. The non-transitory computer-readable medium of claim 24, wherein predicting the location of the portion of the current image to be used to determine the one or more future exposure settings comprises excluding an area of ​​the current image from being used to determine the one or more future exposure settings.

27. When executed by the one or more processors, the one or more processors 25. The non-transitory computer-readable medium of claim 24, further comprising instructions to determine the one or more movement characteristics based at least in part on one or more of an inertial sensor or an input to a steering system in communication with the image sensor.

28. When executed by the one or more processors, the one or more processors 25. The non-transitory computer-readable medium of claim 24, further comprising instructions to cause determining the one or more movement characteristics based at least in part on input to a steering system in communication with the image sensor.

29. 1. An apparatus for processing one or more images, comprising: means for determining one or more current exposure settings for a current image of a current scene at a current time; means for determining one or more motion characteristics associated with the image sensor; means for predicting, based on the one or more motion characteristics, a location of a portion of the current image to be used to determine one or more future exposure settings for the image sensor, the one or more future exposure settings for capturing a future image of a future scene at a future time, the future time being subsequent to the current time; and means for determining the one or more future exposure settings based on the predicted portions of the current image.

30. 30. The apparatus of claim 29, wherein predicting the location of the portion of the current image to use to determine the one or more future exposure settings comprises excluding an area of ​​the current image from being used to determine the one or more future exposure settings.

Images