Systems and methods for detection and mitigation of a rolling band

Abstract

An example method includes receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The method also includes determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The method includes further receiving, from an ambient light sensor, ambient data associated with the two consecutive frames. The method also includes determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The method additionally includes correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

Description

SYSTEMS AND METHODS FOR DETECTION AND MITIGATION OF A ROLLING BANDBACKGROUND

[0001] Many modem computing devices, including mobile phones, personal computers, and tablets, include image capture devices, such as still and / or video cameras. The image capture devices can capture images, such as images that include people, animals, landscapes, and / or objects. The image capture devices can include an image sensor. The image sensor includes a plurality of light-sensing pixels that measure an intensity of light incident thereon and thereby collectively capture an image of an environment. The intensity of light may have a high dynamic range based on the environment.SUMMARY

[0002] This application generally relates to detecting and correcting rolling bands in live capture videos. For example, a rolling band may occur when recording a video on a smartphone. It is generally caused by a mismatch between the frame rate of the video and the frequency of the ambient lighting that is being used to capture a scene. This can result in horizontal lines or bands that appear to be moving across the video. Some existing technologies are configured to detect a rolling band based on row sums of pixel intensities, and then increase the exposure time of the camera to remove and / or reduce the banding. However this can result in a reduction of the dynamic range (e.g., in indoor bright scenes), and users are likely to see bright areas of the image as over-exposed. Also, detection of banding based on the row sums alone may be unreliable. For example, changes in row sums may be caused by a moving object. However, as disclosed herein, a frequency of the banding may be compared to a frequency of the ambient light to eliminate a false detection of banding, such as the one caused by motion. Techniques that address rolling band based on flickering are unable to remove the rolling band from all areas of the image, and may also lead to a degradation of image quality. Accordingly, there is a technical problem that involves detection, and reduction and / or removal, of rolling bands in videos.

[0003] In one aspect, a computer-implemented method is provided. The method includes receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The method also includes determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The method includes further receiving, from an ambient lightsensor, ambient data associated with the two consecutive frames. The method also includes determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The method additionally includes correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0004] In another aspect, a system is provided. The system may include one or more processors. The system may also include data storage, where the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the system to carry out operations. The operations may include receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The operations may also include determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The operations may additionally include receiving, from an ambient light sensor, ambient data associated with the two consecutive frames. The operations may also include determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The operations may further include correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0005] In another aspect, a computing device is provided. The device includes one or more processors and data storage that has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out operations. The operations may include receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The operations may also include determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The operations may additionally include receiving, from an ambient light sensor, ambient data associated with the two consecutive frames. The operations may also include determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The operations may further include correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0006] In another aspect, an article of manufacture is provided. The article of manufacture may include a non-transitory computer-readable medium having stored thereon program instructions that, upon execution by one or more processors of a computing device, cause the computing device to carry out operations. The operations may include receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The operations may also include determining apattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The operations may additionally include receiving, from an ambient light sensor, ambient data associated with the two consecutive frames. The operations may also include determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The operations may further include correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0007] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the figures and the following detailed description and the accompanying drawings.BRIEF DESCRIPTION OF THE FIGURES

[0008] Figure 1 A is an example illustration of a presence of a rolling band, in accordance with example embodiments.

[0009] Figure IB is an example illustration of an absence of a rolling band, in accordance with example embodiments.

[0010] Figure 2 is an example overview of a processing pipeline for rolling band detection and correction, in accordance with example embodiments.

[0011] Figure 3 is a block diagram of an example computing device, in accordance with example embodiments.

[0012] Figure 4 is a flowchart of a method, in accordance with example embodiments.

[0013] Figure 5 is another flowchart of a method, in accordance with example embodiments.DETAILED DESCRIPTION

[0014] Example methods, devices, and systems are described herein. It should be understood that the words “example” and “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or features. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein.

[0015] Thus, the example embodiments described herein are not meant to be limiting. Aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.

[0016] Further, unless context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should be generally viewed as component aspects of one or more overall embodiments, with the understanding that not all illustrated features are necessary for each embodiment.Overview

[0017] Some existing approaches to detecting a rolling band may involve determining row sums of pixel intensities in successive frames of a video. For example, row-wise changes in pixel intensities can be used to detect whether or not there is banding. Some other approaches to detecting a rolling band may involve using data from an ambient light sensor (ALS) of a camera. For example, data from the ALS can be used to determine a frequency of ambient light. However, detection of banding based on row sums alone, or on ALS alone, may be unreliable. For example, changes in row sums, and / or changes in ALS data may be caused by motion (e.g., motion of an object in the scene being captured, movement of the camera, etc.). Also, existing approaches to correction of the banding generally involve increasing the exposure time of the camera. However, this may be unnecessary in some situations, and / or may result in a reduction of the dynamic range in other situations.

[0018] As described herein, an effective approach to detection of a rolling band is based on a combination of an analysis of row sums and an analysis of the data from an ALS. For example, a first frequency measurement of the ambient light and a second frequency measurement based on the pixel sums may be compared to detect banding, an extent of the detected banding, and whether correction of the banding is needed to resolve a visible banding. For example, rowwise changes in pixel intensities can be used to detect whether or not there is banding. In some instances, a sinusoidal curve can indicate banding, whereas a substantially linear function can indicate an absence of banding. Also, for example, a frequency of the banding, if present, can be determined from the sinusoidal curve. A visibility score can be assigned to the banding to determine an extent to which banding is visible. Also, for example, data from the ALS can be used to determine a frequency of flickering in the ambient light, and a confidence level of such measurement. The frequency of the flickering and the frequency of the banding may be compared to determine an amount of banding, and whether correction is needed to resolve a visible banding. Factors such as the visibility score and the confidence level of the flicker measurement come into play.

[0019] Based upon a determination that a rolling band is significant and / or visible enough to be corrected, techniques for correction (e.g., removal and / or reduction) of the rolling band may be applied. For example, in bright sunlit conditions, the rolling band can be removed byincreasing the exposure time of the camera. However this may lead to a loss of the dynamic range (e.g., low brightness regions are captured whereas high brightness regions are visible as white regions without any detail). Accordingly, machine learning algorithms may be applied for object detection, segmentation, and / or masking to identify objects and / or regions that include image information that needs to be preserved. For example, a percentage of the image occupied by a bright region (e.g., a sunlit sky), a type of textured detail in an object, etc. may be determined. As an example, if a large portion of the image corresponds to the sky, then there may be a lesser need to compensate for a loss of dynamic range. However, if the brighter regions include objects of interest, textured patterns of interest, and so forth, then compensation for a potential loss of dynamic range is desired. These, and other aspects, are described in more detail herein.Example Camera System

[0020] As image capture devices, such as cameras, become more popular, they may be employed as standalone hardware devices or integrated into various other types of devices. For instance, still and video cameras are now regularly included in wireless computing devices (e.g., mobile devices, such as mobile phones), tablet computers, laptop computers, video game interfaces, home automation devices, and even automobiles and other types of vehicles.

[0021] The physical components of a camera may include one or more apertures through which light enters, one or more recording surfaces for capturing the images represented by the light, and lenses positioned in front of each aperture to focus at least part of the image on the recording surface(s). The apertures may be of a fixed size or may be adjustable. In an analog camera, the recording surface may be a photographic film. In a digital camera, the recording surface may include an electronic image sensor (e.g., a charge coupled device (CCD) or a complementary metal -oxide-semiconductor (CMOS) sensor) to transfer and / or store captured images in a data storage unit (e.g., memory).

[0022] One or more shutters may be coupled to, or positioned near, the lenses or the recording surfaces. Each shutter may either be in a closed position, in which it blocks light from reaching the recording surface, or an open position, in which light is allowed to reach the recording surface. The position of each shutter may be controlled by a shutter button. For instance, a shutter may be in the closed position by default. When the shutter button is triggered (e.g., pressed), the shutter may change from the closed position to the open position for a period of time, known as the shutter cycle. During the shutter cycle, an image may be captured on the recording surface. At the end of the shutter cycle, the shutter may change back to the closed position.

[0023] Alternatively, the shuttering process may be electronic. For example, before an electronic shutter of a CCD image sensor is “opened,” the sensor may be reset to remove any residual signal in its photodiodes. While the electronic shutter remains open, the photodiodes may accumulate charge. When or after the shutter closes, these charges may be transferred to longer-term data storage. Combinations of mechanical and electronic shuttering may also be possible.

[0024] Regardless of type, a shutter may be activated and / or controlled by something other than a shutter button. For instance, the shutter may be activated by a softkey, a timer, or some other trigger. Herein, the term “capture” may refer to any mechanical and / or electronic shuttering process that results in one or more images being recorded, regardless of how the shuttering process is triggered or controlled.

[0025] The exposure of a captured image may be determined by a combination of the size of the aperture, the brightness of the light entering the aperture, and the length of the shutter cycle (also referred to as the shutter length, the exposure length, or the exposure time). Additionally, a digital and / or analog gain (e.g., based on an ISO setting) may be applied to the image, thereby influencing the exposure. In some embodiments, the term “exposure length,” “exposure time,” or “exposure time interval” may refer to the shutter length multiplied by the gain for a particular aperture size. Thus, these terms may be used somewhat interchangeably, and should be interpreted as possibly being a shutter length, an exposure time, and / or any other metric that controls the amount of signal response that results from light reaching the recording surface.

[0026] In some implementations or modes of operation, a camera may capture one or more still images each time image capture is triggered. In other implementations or modes of operation, a camera may capture a video image by continuously capturing images at a particular rate (e.g., 24 frames per second) as long as image capture remains triggered (e.g., while the shutter button is held down). Some cameras, when operating in a mode to capture a still image, may open the shutter when the camera device or application is activated, and the shutter may remain in this position until the camera device or application is deactivated. While the shutter is open, the camera device or application may capture and display a representation of a scene on a viewfinder (sometimes referred to as displaying a “preview frame”). When image capture is triggered, one or more distinct payload images of the current scene may be captured.

[0027] Cameras, including digital and analog cameras, may include software to control one or more camera functions and / or settings, such as aperture size, exposure time, gain, and so on. Additionally, some cameras may include software that digitally processes images during or after image capture. While the description above refers to cameras in general, it may beparticularly relevant to digital cameras. Digital cameras may be standalone devices e.g., a DSLR camera) or may be integrated with other devices.

[0028] Either or both of a front-facing camera and a rear-facing camera may include or be associated with an ambient light sensor (ALS) that may continuously or from time to time determine the ambient brightness of a scene that the camera can capture. In some devices, the ALS can be used to adjust the display brightness of a screen associated with the camera (e.g., a viewfinder). When the determined ambient brightness is high, the brightness level of the screen may be increased to make the screen easier to view. When the determined ambient brightness is low, the brightness level of the screen may be decreased, also to make the screen easier to view as well as to potentially save power. Additionally, the ambient light sensor’s input may be used to determine an exposure time of an associated camera, or to help in this determination.Example Rolling Band Detection

[0029] Figure 1 A is an example illustration of a presence of a rolling band, in accordance with example embodiments. A first computing device 105 is shown to display an image (e.g., a video) of a cat. When second computing device 110 is used to capture a video of the image displayed on the first computing device 105, one or more rolling bands in the form of horizontal bands 115 may appear on a display of the second computing device 110.

[0030] Figure IB is an example illustration of an absence of a rolling band, in accordance with example embodiments. As shown, first computing device 105 is shown to display the image (e.g., a video) of the cat, without any perceptible rolling bands.

[0031] Figure 2 is an example overview of a processing pipeline for rolling band detection and correction, in accordance with example embodiments. In some embodiments, the operations may involve displaying, by a graphical user interface of a computing device, a preview video of a scene being captured. In some embodiments, the computing device may be a mobile device. In some embodiments, the graphical user interface may be an interface that displays a captured video (e.g., a pre-recorded video stored on the computing device). In some embodiments, the graphical user interface may be a live-view interface that displays a live- view preview of a video.

[0032] Herein a "live-view preview" of a video should be understood to be a sequence of images (e.g., video) comprising consecutive frames that are generated and displayed based on an image data stream from an image sensor of an image-capture device. For instance, image data may be generated by a camera's image sensor (or a portion, subset, or sampling of the pixels on the image sensor). This image data is representative of the field-of-view (FOV) ofthe camera, and thus indicative of the sequence of images that will be captured if the user taps the camera's shutter button or initiates image capture in some other manner. To help a user to decide how to position a camera for image capture, a camera device or other computing device may generate and display a live-view preview image based on the image data stream from the image sensor. The live-view preview image can be a real-time image feed (e.g., video), such that the user is informed of the camera's FOV in real-time.

[0033] Block 205 involves receiving, from an image sensor of an image capturing device, a sensor row sum input. For example, the sensor row sum input may include aggregated light intensity data for two consecutive frames of a video. Generally, the aggregated light intensity data can be any type of statistical data e.g., a mean, median, statistical distribution, etc.) that indicates row-wise light intensities. In some embodiments the aggregated light intensity data may include respective row-sums of pixel intensities in the two consecutive frames. For example, a sensor row sum is a technique used to measure an amount of light that is received by each row of pixels on an image sensor.

[0034] The image sensor may include a plurality of light-sensing pixels that measure an intensity of light incident thereon and thereby collectively capture an image of an environment. The light-sensing pixels may be arranged in arrays, and may be grouped by position into pixel sensor groups. For example, an array of pixel sensors may be grouped into pixel sensor groups, where each pixel sensor group includes an array of pixel sensors. Such pixel sensor groups may be further grouped to define regions of interest (ROIs). Image processing circuitry groups may be configured to each receive pixel information from a corresponding pixel sensor group and further configured to perform image processing operations on the pixel information to provide processed pixel information during operation of the image sensor.

[0035] In the case of image processing, the input is typically either singular or a set of three channel images consisting of red, green, and blue (RGB) color channels. Such color images are commonly captured with an array of light sensors covered with different color filters, called a color filter array, which are generally arranged as a repeated mosaic. Examples of color filters can be, for example, a Bayer filter, a modified Bayer filter such as RGBE where a green filter is modified to an “emerald” filter, a red-yellow-yellow-blue (RYYB) filter, a cyan-yellow- yellow-magenta (CYYM) filter, a cyan -yellow-green-magenta (CYGM) filter, various modifications of the Bayer filter (e.g., RGBW where a green filter is modified to a “white” filter, a Quad Bayer filter (comprising 4x blue, 4x red, and 8x green filters), RYYB Quad Bayer (comprising 4x blue, 4x red, and 8x yellow filters), nonacell (comprising 9x blue, 9x red, and 18x green filters), RCCC filter (a monochrome sensor with a red channel to detect traffic lights,stop signs, etc.), RCCB filter (where the green pixels are clear), and others. For example, the color filter can also correspond to filters used in multispectral sensors. For low latency and memory bound applications it is sometimes desirable to process the raw color mosaic image as opposed to some RGB equivalent that was reconstructed from the raw image.

[0036] Some embodiments involve determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. For example, the pattern of light intensities may be based on a difference between the respective row-wise sums of light intensities in the respective pixel arrays. For example, a first row-wise delta difference may be determined based on a difference between a first sum of pixel intensities in a first row of a first frame, and a second sum of pixel intensities in a first row of a second frame (e.g., successive to the first frame). Likewise, a second row-wise delta difference may be determined based on a difference between a third sum of pixel intensities in a second row of the first frame, and a fourth sum of pixel intensities in a second row of the second frame (e.g., successive to the first frame). Continuing thus, row-wise delta differences may be determined for each row of the pixel array.

[0037] In some embodiments, determining the pattern of light intensities may involve applying a fast Fourier transform (FFT) to such row-wise delta differences over successive frames. This allows for an analysis of the aggregated light intensity data to a signal in a frequency domain. Generally speaking, in the event that the delta differences are substantially similar (e.g.,the signal is substantially linear or flat), this is indicative of an absence of a rolling band. However, in the event the delta differences resolve into a curve (e.g., the signal is substantially sinusoidal), this may be indicative of a presence of a rolling band.

[0038] Block 210 involves receiving ALS sensor input from an ALS sensor. For example, ambient data associated with the two consecutive frames may be received.

[0039] Block 215 involves detection of a rolling band. As described herein, such detection may be based on a combination of the row sum input and the ALS sensor input.

[0040] At block 220, detection of a rolling band may involve analyzing the row sum data. For example, a rolling band can be identified through signal processing by comparing row sums in the two consecutive frames. For example, a pattern of light intensities for the two consecutive frames may be determined based on a difference between the respective row-wise sums of light intensities in the respective pixel arrays.

[0041] Generally, the pattern of light intensities may be used to determine a band frequency. For example, as described herein, a FFT may be applied to the delta differences between the respective row-wise sums of light intensities in the respective pixel arrays to obtain a signal ina frequency domain. Within the frequency domain, a frequency for the signal may be obtained to determine the band frequency, and a peak value may be used to determine a band amplitude.

[0042] At block 225, a flicker light source may be detected using the ALS sensor data. For example, the ALS is generally constantly running and sampling at a higher frequency and detecting ambient light sources. In some embodiments, a flicker measurement indicating an amount of flicker in the video may be determined based on the ALS data.

[0043] Although row sum data or ALS data may be used independently to detect a rolling band, such an approach may not be reliable. For example, movement of the image capture device and / or movement of an object in the FOV may trigger a false positive - i.e., detection of a rolling band when there is no rolling band. For example, ALS data and / or row sum data may be indicative of motion, and not of a rolling band. Applying a band correction in such instances may degrade video quality. Accordingly, as described below, a more reliable band detection mechanism is based on correlating the pattern of light intensities with an amount of flicker to determine a presence or an absence of a rolling band in the video. For example, block 230 involves visible rolling band detection, and block 235 involves determining whether a rolling band is detected.

[0044] In some embodiments, the flicker measurement may be associated with a confidence level indicating a reliability of the flicker measurement. For example, the confidence level may be a normalized value in a range from 0 to 1, where 0 indicates a very low confidence, and 1 indicates a very high confidence. In some embodiments, a confidence threshold for the flicker measurement may be determined. Generally, the confidence threshold indicates a lowest acceptable confidence level for the flicker measurement, and may depend on factors such as the image capturing device, the ALS sensor, the ambient light, and so forth. In some embodiments, a flicker frequency may be determined from the amount of flicker. For example, the ALS data may indicate that the flicker frequency is at 100 Hertz (Hz), with a confidence level of 0.8.

[0045] Generally, a determination of the band frequency may not be as reliable as a determination of the flicker frequency. Accordingly, in some embodiments, the correlating of the pattern of light intensities and the amount of flicker involves determining whether the band frequency is within a range of the flicker frequency. For example, in the event that the flicker frequency is determined to be at 100 Hz, with a confidence level of 0.8 (with a confidence threshold at 0.7), and the band frequency is close to 100 Hz, then the band frequency may bedetermined to be within a range of the flicker frequency. In such a situation, it may be determined that a rolling band is detected at block 235.

[0046] Also, for example, based on a determination that the band frequency is not within a range of the flicker frequency, it may be determined that a rolling band is not detected at block 235. For example, in the event that the flicker frequency is determined to be at 100 Hz, with a confidence level of 0.72 (with a confidence threshold at 0.7), and the band frequency is close to 500 Hz, then the band frequency may be determined to be not within a range of the flicker frequency. This may correspond to a false detection of a rolling band where the row sum data indicates motion instead of a rolling band. In such a situation, it may be determined that a rolling band is not detected at block 235. As another example, in the event that the flicker frequency is determined to be at 100 Hz, with a confidence level of 0.6 (with a confidence threshold at 0.7), and the band frequency is close to 100 Hz, then the measurements may be deemed to be unreliable and it may be determined that a rolling band is not detected at block 235.

[0047] Some embodiments may involve, based on a determination that the confidence level of the flicker measurement exceeds the confidence threshold, determining the band amplitude from the pattern of light intensities, and determining whether the band amplitude exceeds a visibility threshold. Generally speaking, the visibility threshold indicates whether the rolling band is visible in the video. In the event the band amplitude does not exceed the visibility threshold, then there is likely no perceptible banding, and it may be determined that a rolling band is not detected at block 235.

[0048] In the event the band amplitude exceeds the visibility threshold, then there is likely to be perceptible banding. In such a situation, the correlating of the pattern of light intensities and the amount of flicker involves determining whether the band frequency is within a range of the flicker frequency, where the range may be a larger range of values. For example, in the event that the flicker frequency is determined to be at 100 Hz, with a confidence level of 0.8 (with a confidence threshold at 0.7), and the band frequency is determined to range between 50 Hz and 150 Hz, then, based on a detection of perceptible banding, the band frequency may be determined to be within a range of the flicker frequency. In such a situation, it may be determined that a rolling band is detected at block 235.Example Rolling Band Correction

[0049] Block 240 involves rolling band correction. Some embodiments involve, responsive to a determining of the presence of the rolling band in the video, applying a band removal strategyto potentially correct the presence of the rolling band in the video. In some embodiments, the applying of the band removal strategy involves determining whether to increase an exposure time for the image capturing device. Generally, an adjustment of the exposure time may be applied on a per frame basis during image capture (e.g., in near real-time).

[0050] Although existing approaches increase the exposure time upon detecting that there is a rolling band, such an automatic correction may not be desirable. For example, such an automatic correction may negatively impact a high dynamic range (HDR) of the video. An HDR display for an image can extend a range of user experience when viewing the image. For example, an image of a person in the dark may have a high composition of dark colors with a low luminance value. Likewise, an image of a sunlit sky may have a high composition of bright colors with a high luminance value. In some aspects, a ratio of respective luminance values may have a high dynamic range, for example, 1 : 1,000,000. Generally, the camera parameters may be adjusted to widen the exposure range to capture more detail in a scene (e.g., capturing details in both bright and dark areas). Increasing the exposure time can over-expose the brighter areas.

[0051] Block 245 involves sky detection. The “sky detection” is used herein as an example to illustrate how a rolling band correction may be applied to avoid an undesirable over-exposure of the brighter areas of a video. For example, a percentage of the sky in a current frame may be determined. In the event the percentage of the sky exceeds a threshold value, a rolling band correction may be applied at block 250 by increasing the exposure time. For example, although increasing the exposure time may add brightness to the sky regions, given that a large portion of the image is covered by the sky regions, the increased brightness may not degrade video quality.

[0052] In the event the percentage of the sky does not exceed the threshold value, a rolling band correction may not be applied at block 250, and the exposure time may be maintained (e.g., not increased). For example, increasing the exposure time may add brightness to the sky regions, given that a smaller portion of the image is covered by the sky regions, the increased brightness may degrade video quality. Accordingly, it is desirable to not increase the exposure time.

[0053] More generally, high brightness regions (e.g., sky) may be identified. The high brightness regions generally correspond to light intensities that exceed a brightness threshold. Also, for example, it may be determined whether an amount of the identified high brightness regions in the two consecutive frames exceeds a size threshold (e.g., whether a percentage of the sky exceeds a threshold value).

[0054] In some embodiments, based on a determination that the amount of the identified high brightness regions in the two consecutive frames exceeds the size threshold, a rolling band correction may be applied at block 250 by increasing the exposure time. For example, although increasing the exposure time may add brightness to the identified high brightness regions, given that a large portion of the image is covered by the identified high brightness regions, the increased brightness may not degrade video quality. Similarly, based on a determination that the amount of the identified high brightness regions in the two consecutive frames does not exceed the size threshold, a rolling band correction may not be applied at block 250, and the exposure time may be maintained. For example, increasing the exposure time may add brightness to the identified high brightness regions, given that a smaller portion of the image is covered by the identified high brightness regions, the increased brightness may degrade video quality. Accordingly, it is desirable to not increase the exposure time.

[0055] In some embodiments, the determining of whether to increase the exposure time for the image capturing device is based on a determining of whether the increase of the exposure time reduces a dynamic range of the video. Generally, upon a determination that the increase of the exposure time reduces the dynamic range of the video, a rolling band correction may not be applied at block 250, and the exposure time may be maintained (e.g., not increased). However, upon a determination that the increase of the exposure time does not reduce the dynamic range of the video, a rolling band correction may be applied at block 250, and the exposure time may be increased.

[0056] For example, the video may include objects of interest, regions of interest, and so forth. For example, one or more trained machine learning algorithms (e.g., an object detection algorithm, a face detection algorithm, a segmentation algorithm, and so forth) may be used to identify one or more regions of interest in the image. In some implementations, a user-approved facial recognition algorithm may be applied to identify one or more individuals in the image as likely objects of interest. For example, a user may enable access to a history of user preferences, and objects and / or individuals may be identified as being of high interest to the user. Accordingly, the determining of whether to increase the exposure time for the image capturing device may be based on a determining of whether the increase of the exposure time has a negative impact on a visibility and / or image characteristics of the objects of interest, regions of interest, and so forth. For example, increasing the exposure time may cause some of the objects and / or regions of interest to become bright enough to degrade their image quality. Similarly, decreasing the exposure time may cause some of the objects and / or regions of interest to become dark enough to degrade their image quality.

[0057] Some embodiments involve determining one or more characteristics of the object and / or region of interest. For example, the one or more characteristics may include textures such as jeans, leather, cotton, any kind of fabric, stone, brick, rough surfaces, smooth surfaces, rust, paint, tissue fabric, mouse pads, tin foil, ice cubes, foam, and bubbles. Accordingly, the determining of whether the increase of the exposure time reduces the dynamic range may be based on the one or more characteristics of the region of interest. For example, the exposure time may be adjusted to maintain a high quality capture of the textures described above. For example, increasing the exposure time may cause some of the textures to become bright enough to lose their distinctive appearance. Similarly, decreasing the exposure time may cause some of the textures to become dark enough to lose their distinctive appearance.Example Computing Device Architecture

[0058] Figure 3 is a block diagram of an example computing device 300, in accordance with example embodiments. In particular, computing device 300 shown in Figure 3 can be configured to perform at least one function described herein, including methods 400 and / or 500.

[0059] Computing device 300 may include a user interface module 301, a network communications module 302, one or more processors 303, data storage 304, rolling band module 312, one or more cameras 318, one or more sensors 320, and power system 322, all of which may be linked together via a system bus, network, or other connection mechanism 305.

[0060] User interface module 301 can be operable to send data to and / or receive data from external user input / output devices. For example, user interface module 301 can be configured to send and / or receive data to and / or from user input devices such as a touch screen, a computer mouse, a keyboard, a keypad, a touch pad, a trackball, a joystick, a voice recognition module, and / or other similar devices. User interface module 301 can also be configured to provide output to user display devices, such as one or more cathode ray tubes (CRT), liquid crystal displays, light emitting diodes (LEDs), displays using digital light processing (DLP) technology, printers, light bulbs, and / or other similar devices, either now known or later developed. User interface module 301 can also be configured to generate audible outputs, with devices such as a speaker, speaker jack, audio output port, audio output device, earphones, and / or other similar devices. User interface module 301 can further be configured with one or more haptic devices that can generate haptic outputs, such as vibrations and / or other outputs detectable by touch and / or physical contact with computing device 300. In some examples, user interface module 301 can be used to provide a graphical user interface (GUI) for utilizing computing device 300.

[0061] Network communications module 302 can include one or more devices that provide one or more wireless interfaces 307 and / or one or more wireline interfaces 308 that are configurable to communicate via a network. Wireless interface(s) 307 can include one or more wireless transmitters, receivers, and / or transceivers, such as a Bluetooth™ transceiver, a Zigbee® transceiver, a Wi-Fi™ transceiver, a WiMAX™ transceiver, an LTE™ transceiver, and / or other type of wireless transceiver configurable to communicate via a wireless network. Wireline interface(s) 308 can include one or more wireline transmitters, receivers, and / or transceivers, such as an Ethernet transceiver, a Universal Serial Bus (USB) transceiver, or similar transceiver configurable to communicate via a twisted pair wire, a coaxial cable, a fiberoptic link, or a similar physical connection to a wireline network.

[0062] In some examples, network communications module 302 can be configured to provide reliable, secured, and / or authenticated communications. For each communication described herein, information for facilitating reliable communications (e.g., guaranteed message delivery) can be provided, perhaps as part of a message header and / or footer (e.g., packet / message sequencing information, encapsulation headers and / or footers, size / time information, and transmission verification information such as cyclic redundancy check (CRC) and / or parity check values). Communications can be made secure (e.g., be encoded or encrypted) and / or decry pted / decoded using one or more cryptographic protocols and / or algorithms, such as, but not limited to, Data Encryption Standard (DES), Advanced Encryption Standard (AES), a Rivest-Shamir-Adelman (RSA) algorithm, a Diffie-Hellman algorithm, a secure sockets protocol such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS), and / or Digital Signature Algorithm (DSA). Other cryptographic protocols and / or algorithms can be used as well or in addition to those listed herein to secure (and then decry pt / decode) communications.

[0063] One or more processors 303 can include one or more general purpose processors (e.g., central processing unit (CPU), etc.), and / or one or more special purpose processors (e.g., digital signal processors, tensor processing units (TPUs), graphics processing units (GPUs), application specific integrated circuits, etc.). One or more processors 303 can be configured to execute computer-readable instructions 306 that are contained in data storage 304 and / or other instructions as described herein.

[0064] Data storage 304 can include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of one or more processors 303. The one or more computer-readable storage media can include volatile and / or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which can beintegrated in whole or in part with at least one of one or more processors 303. In some examples, data storage 304 can be implemented using a single physical device (e.g., one optical, magnetic, organic or other memory or disc storage unit), while in other examples, data storage 304 can be implemented using two or more physical devices.

[0065] Data storage 304 can include computer-readable instructions 306 and perhaps additional data. In some examples, data storage 304 can include storage required to perform at least part of the herein-described methods, scenarios, and techniques and / or at least part of the functionality of the herein-described devices and networks. In particular, computer-readable instructions 306 can include instructions that, when executed by processor(s) 303, enable computing device 300 to provide for some or all of the functionality described herein. For example, data storage 304 may store captured images, personalized correction factors, and so forth.

[0066] In some embodiments, computer-readable instructions 306 can include instructions that, when executed by processor(s) 303, enable computing device 300 to carry out operations. The operations may include receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video. The operations may also include determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data. The operations may additionally include receiving, from an ambient light sensor, ambient data associated with the two consecutive frames. The operations may also include determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video. The operations may further include correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0067] In some examples, computing device 300 can include rolling band module 312. Rolling band module 312 can be configured to detect and / or correct a rolling band in a video. For example, rolling band module 312 can be configured to correlate a pattern of light intensities (e.g., based on row sum data from one or more cameras 318) and an amount of flicker (e.g., based on ALS data from one or more sensors 320) to determine a presence or an absence of a rolling band in the video.

[0068] In some examples, computing device 300 can include one or more cameras 318. Camera(s) 318 can include one or more image capture devices, such as still and / or video cameras, equipped to capture light and record the captured light in one or more images; that is, camera(s) 318 can generate image(s) of captured light. The one or more images can be one or more still images and / or one or more images utilized in video imagery. Camera(s) 318 cancapture light and / or electromagnetic radiation emitted as visible light, infrared radiation, ultraviolet light, and / or as one or more other frequencies of light. Camera(s) 318 can include a wide camera, a tele camera, an ultrawide camera, and so forth. Also, for example, camera(s) 318 can be front-facing or rear-facing cameras with reference to computing device 300. Camera(s) 318 can include camera components such as, but are not limited to, an aperture, shutter, recording surface (e.g., photographic film and / or an image sensor), lens, and / or shutter button. The camera components may be controlled at least in part by software executed by one or more processors 303.

[0069] In some examples, computing device 300 can include one or more sensors 320. Sensors 320 can be configured to measure conditions within computing device 300 and / or conditions in an environment of computing device 300 and provide data about these conditions. For example, sensors 320 can include one or more of (i) sensors for obtaining data about computing device 300, such as, but not limited to, a thermometer for measuring a temperature of computing device 300, a battery sensor for measuring power of one or more batteries of power system 322, and / or other sensors measuring conditions of computing device 300; (ii) an identification sensor to identify other objects and / or devices, such as, but not limited to, a Radio Frequency Identification (RFID) reader, proximity sensor, one-dimensional barcode reader, two-dimensional barcode (e.g., Quick Response (QR) code) reader, and a laser tracker, where the identification sensors can be configured to read identifiers, such as RFID tags, barcodes, QR codes, and / or other devices and / or object configured to be read and provide at least identifying information; (iii) sensors to measure locations and / or movements of computing device 300, such as, but not limited to, a tilt sensor, a gyroscope, an accelerometer, a Doppler sensor, a GPS device, a sonar sensor, a radar device, a laser-displacement sensor, and a compass; (iv) an environmental sensor to obtain data indicative of an environment of computing device 300, such as, but not limited to, an infrared sensor, an optical sensor, a light sensor (e.g., an ambient light sensor), a biosensor, a capacitive sensor, a touch sensor, a temperature sensor, a wireless sensor, a radio sensor, a movement sensor, a microphone, a sound sensor, an ultrasound sensor and / or a smoke sensor; and / or (v) a force sensor to measure one or more forces (e.g, inertial forces and / or G-forces) acting about computing device 300, such as, but not limited to one or more sensors that measure: forces in one or more dimensions, torque, ground force, friction, and / or a zero moment point (ZMP) sensor that identifies ZMPs and / or locations of the ZMPs. Many other examples of sensors 320 are possible as well.

[0070] Power system 322 can include one or more batteries 324 and / or one or more external power interfaces 326 for providing electrical power to computing device 300. Each battery ofthe one or more batteries 324 can, when electrically coupled to the computing device 300, act as a source of stored electrical power for computing device 300. One or more batteries 324 of power system 322 can be configured to be portable. Some or all of one or more batteries 324 can be readily removable from computing device 300. In other examples, some or all of one or more batteries 324 can be internal to computing device 300, and so may not be readily removable from computing device 300. Some or all of one or more batteries 324 can be rechargeable. For example, a rechargeable battery can be recharged via a wired connection between the battery and another power supply, such as by one or more power supplies that are external to computing device 300 and connected to computing device 300 via the one or more external power interfaces. In other examples, some or all of one or more batteries 324 can be non-rechargeable batteries.

[0071] One or more external power interfaces 326 of power system 322 can include one or more wired-power interfaces, such as a USB cable and / or a power cord, that enable wired electrical power connections to one or more power supplies that are external to computing device 300. One or more external power interfaces 326 can include one or more wireless power interfaces, such as a Qi wireless charger, that enable wireless electrical power connections, such as via a Qi wireless charger, to one or more external power supplies. Once an electrical power connection is established to an external power source using one or more external power interfaces 326, computing device 300 can draw electrical power from the external power source the established electrical power connection. In some examples, power system 322 can include related sensors, such as battery sensors associated with the one or more batteries or other types of electrical power sensors.

[0072] One or more external power interfaces 326 of power system 322 can include one or more wired-power interfaces, such as a USB cable and / or a power cord, that enable wired electrical power connections to one or more power supplies that are external to computing device 300. One or more external power interfaces 326 can include one or more wireless power interfaces, such as a Qi wireless charger, that enable wireless electrical power connections, such as via a Qi wireless charger, to one or more external power supplies. Once an electrical power connection is established to an external power source using one or more external power interfaces 326, computing device 300 can draw electrical power from the external power source the established electrical power connection. In some examples, power system 322 can include related sensors, such as battery sensors associated with the one or more batteries or other types of electrical power sensors.Example Methods of Operation

[0073] Figure 4 is a flowchart of a method, in accordance with example embodiments. Method 400 may include various blocks or steps. The blocks or steps may be carried out individually or in combination. The blocks or steps may be carried out in any order and / or in series or in parallel. Further, blocks or steps may be omitted or added to method 400.

[0074] The blocks of method 400 may be carried out by various elements of computing device 300 as illustrated and described in reference to Figure 3.

[0075] Block 410 involves receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video.

[0076] Block 420 involves determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data.

[0077] Block 430 involves receiving, from an ambient light sensor, ambient data associated with the two consecutive frames.

[0078] Block 440 involves determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video.

[0079] Block 450 involves correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

[0080] Some embodiments involve determining a band frequency from the pattern of light intensities. Such embodiments involve determining a flicker frequency from the amount of flicker. The correlating of the pattern of light intensities and the amount of flicker involves determining whether the band frequency is within a range of the flicker frequency.

[0081] Some embodiments involve, based on a determination that the band frequency is not within a range of the flicker frequency, determining the absence of the rolling band in the video.

[0082] Some embodiments involve, based on a determination that the band frequency is within a range of the flicker frequency, determining whether a confidence level of the flicker measurement exceeds a confidence threshold.

[0083] Some embodiments involve, based on a determination that the confidence level of the flicker measurement does not exceed the confidence threshold, determining the absence of the rolling band in the video.

[0084] Some embodiments involve, based on a determination that the confidence level of the flicker measurement exceeds the confidence threshold, determining a band amplitude from the pattern of light intensities. Such embodiments involve determining whether the band amplitudeexceeds a visibility threshold, wherein the visibility threshold indicates whether the rolling band is visible in the video.

[0085] Some embodiments involve determining that the band amplitude exceeds the visibility threshold. Such embodiments involve determining the presence of the rolling band in the video.

[0086] Some embodiments involve determining that the band amplitude does not exceed the visibility threshold. Such embodiments involve determining the absence of the rolling band in the video.

[0087] Some embodiments involve, responsive to a determining of the presence of the rolling band in the video, applying a band removal strategy to potentially correct the presence of the rolling band in the video.

[0088] In some embodiments, the applying of the band removal strategy involves determining whether to increase an exposure time for the image capturing device.

[0089] Some embodiments involve identifying high brightness regions in the two consecutive frames, wherein the high brightness regions correspond to light intensities that exceed a brightness threshold. Such embodiments involve determining whether an amount of the identified high brightness regions in the two consecutive frames exceeds a size threshold.

[0090] Some embodiments involve, based on a determination that the amount of the identified high brightness regions in the two consecutive frames exceeds the size threshold, determining to increase the exposure time for the image capturing device to correct the presence of the rolling band in the video.

[0091] Some embodiments involve, based on a determination that the amount of the identified high brightness regions in the two consecutive frames does not exceed the size threshold, maintaining the exposure time for the image capturing device.

[0092] In some embodiments, the determining of whether to increase the exposure time for the image capturing device may be based on a determining of whether the increase of the exposure time reduces a dynamic range of the video.

[0093] Some embodiments involve, upon a determination that the increase of the exposure time reduces the dynamic range of the video, maintaining the exposure time for the image capturing device.

[0094] Some embodiments involve, upon a determination that the increase of the exposure time does not reduce the dynamic range of the video, increasing the exposure time for the image capturing device.

[0095] Some embodiments involve, detecting, based on a trained machine learning model, a region of interest in the video. Such embodiments involve determining one or morecharacteristics of the region of interest. The determining of whether the increase of the exposure time reduces the dynamic range is based on the one or more characteristics of the region of interest.

[0096] In some embodiments, the image capturing device may be a component of a mobile device.

[0097] In some embodiments, the aggregated light intensity data includes respective row-wise sums of pixel values in the respective pixel arrays for the two consecutive frames, and wherein the pattern of light intensities is based on a difference between the respective row-wise sums of pixel values in the respective pixel arrays.

[0098] Figure 5 is a flowchart of a method, in accordance with example embodiments. Method 500 may include various blocks or steps. The blocks or steps may be carried out individually or in combination. The blocks or steps may be carried out in any order and / or in series or in parallel. Further, blocks or steps may be omitted or added to method 500. Additionally, there may be additional relationships between the blocks. As such, Figure 5 illustrates one embodiment of the techniques described herein.

[0099] The blocks of method 500 may be carried out by various elements of computing device 300 as illustrated and described in reference to Figure 3.

[0100] Block 505 involves analyzing aggregated light intensity data.

[0101] Block 510 involves analyzing data from an ambient light sensor (ALS).

[0102] Block 515 involves correlating the row sum data and the ALS data.

[0103] Block 520 involves determining whether a band frequency is within range of a flicker frequency.

[0104] Upon a determination that the band frequency is not within range of the flicker frequency, method 500 proceeds to block 525.

[0105] Block 525 involves determining that there is an absence of a rolling band. Method 500 proceeds to block 530.

[0106] Block 530 involves determining that no correction is applied.

[0107] Upon a determination that the band frequency is within range of the flicker frequency, method 500 proceeds to block 535.

[0108] Block 535 involves determining that there is a presence of a rolling band. Method 500 proceeds to block 545.

[0109] Block 545 involves determining whether a band amplitude exceed a visibility threshold.

[0110] Upon a determination that the band amplitude does not exceed the visibility threshold, method 500 proceeds to block 530, and no correction is applied.

[0111] Upon a determination that the band amplitude exceeds the visibility threshold, method 500 proceeds to block 550.

[0112] Block 550 involves determining whether a size of a high brightness region exceeds a size threshold.

[0113] Upon a determination that the size of the high brightness region does not exceed the size threshold, method 500 proceeds to block 555.

[0114] Block 555 involves maintaining the exposure time.

[0115] Upon a determination that the size of the high brightness region exceeds the size threshold, method 500 proceeds to block 560.

[0116] Block 560 involves determining whether a HDR is negatively impacted.

[0117] Upon a determination that the HDR is negatively impacted, method 500 proceeds to block 555, and exposure time is maintained.

[0118] Upon a determination that the HDR is not negatively impacted, method 500 proceeds to block 565.

[0119] Block 565 involves increasing the exposure time.

[0120] The particular arrangements shown in the Figures should not be viewed as limiting. It should be understood that other embodiments may include more or less of each element shown in a given Figure. Further, some of the illustrated elements may be combined or omitted. Yet further, an illustrative embodiment may include elements that are not illustrated in the Figures.

[0121] A step or block that represents a processing of information can correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a step or block that represents a processing of information can correspond to a module, a segment, or a portion of program code (including related data). The program code can include one or more instructions executable by a processor for implementing specific logical functions or actions in the method or technique. The program code and / or related data can be stored on any type of computer readable medium such as a storage device including a disk, hard drive, or other storage medium.

[0122] The computer readable medium can also include non-transitory computer readable media such as computer-readable media that store data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer readable media can also include non-transitory computer readable media that store program code and / ordata for longer periods. Thus, the computer readable media may include secondary or persistent long-term storage, like read only memory (ROM), optical or magnetic disks, compact disc read only memory (CD-ROM), for example. The computer readable media can also be any other volatile or non-volatile storage systems. A computer readable medium can be considered a computer readable storage medium, for example, or a tangible storage device.

[0123] While various examples and embodiments have been disclosed, other examples and embodiments will be apparent to those skilled in the art. The various disclosed examples and embodiments are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method, comprising: receiving, from an image sensor of an image capturing device, aggregated light intensity data based on respective pixel arrays for two consecutive frames of a video; determining a pattern of light intensities for the two consecutive frames based on the aggregated light intensity data; receiving, from an ambient light sensor, ambient data associated with the two consecutive frames; determining, based on the ambient data, a flicker measurement indicating an amount of flicker in the video; and correlating the pattern of light intensities and the amount of flicker to determine a presence or an absence of a rolling band in the video.

2. The computer-implemented method of claim 1, further comprising: determining a band frequency from the pattern of light intensities; and determining a flicker frequency from the amount of flicker, and wherein the correlating of the pattern of light intensities and the amount of flicker comprises determining whether the band frequency is within a range of the flicker frequency.

3. The computer-implemented method of claim 2, further comprising: based on a determination that the band frequency is not within a range of the flicker frequency, determining the absence of the rolling band in the video.

4. The computer-implemented method of claim 2, further comprising: based on a determination that the band frequency is within a range of the flicker frequency, determining whether a confidence level of the flicker measurement exceeds a confidence threshold.

5. The computer-implemented method of claim 4, further comprising: based on a determination that the confidence level of the flicker measurement does notexceed the confidence threshold, determining the absence of the rolling band in the video.

6. The computer-implemented method of claim 4, further comprising: based on a determination that the confidence level of the flicker measurement exceeds the confidence threshold, determining a band amplitude from the pattern of light intensities; and determining whether the band amplitude exceeds a visibility threshold, wherein the visibility threshold indicates whether the rolling band is visible in the video.

7. The computer-implemented method of claim 6, further comprising: determining that the band amplitude exceeds the visibility threshold; and determining the presence of the rolling band in the video.

8. The computer-implemented method of claim 6, further comprising: determining that the band amplitude does not exceed the visibility threshold; and determining the absence of the rolling band in the video.

9. The computer-implemented method of claim 1, further comprising: responsive to a determining of the presence of the rolling band in the video, applying a band removal strategy to potentially correct the presence of the rolling band in the video.

10. The computer-implemented method of claim 9, wherein the applying of the band removal strategy comprises: determining whether to increase an exposure time for the image capturing device.

11. The computer-implemented method of claim 10, further comprising: identifying high brightness regions in the two consecutive frames, wherein the high brightness regions correspond to light intensities that exceed a brightness threshold; and determining whether an amount of the identified high brightness regions in the two consecutive frames exceeds a size threshold.

12. The computer-implemented method of claim 11, further comprising: based on a determination that the amount of the identified high brightness regions in the two consecutive frames exceeds the size threshold, determining to increase the exposuretime for the image capturing device to correct the presence of the rolling band in the video.

13. The computer-implemented method of claim 11, further comprising: based on a determination that the amount of the identified high brightness regions in the two consecutive frames does not exceed the size threshold, maintaining the exposure time for the image capturing device.

14. The computer-implemented method of claim 10, wherein the determining of whether to increase the exposure time for the image capturing device is based on a determining of whether the increase of the exposure time reduces a dynamic range of the video.

15. The computer-implemented method of claim 14, further comprising: upon a determination that the increase of the exposure time reduces the dynamic range of the video, maintaining the exposure time for the image capturing device.

16. The computer-implemented method of claim 14, further comprising: upon a determination that the increase of the exposure time does not reduce the dynamic range of the video, increasing the exposure time for the image capturing device.

17. The computer-implemented method of claim 14, further comprising: detecting, based on a trained machine learning model, a region of interest in the video; and determining one or more characteristics of the region of interest, and wherein the determining of whether the increase of the exposure time reduces the dynamic range is based on the one or more characteristics of the region of interest.

18. The computer-implemented method of claim 1, wherein the image capturing device is a component of a mobile device.

19. The computer-implemented method of claim 1, wherein the aggregated light intensity data comprises respective row-wise sums of pixel values in the respective pixel arrays for the two consecutive frames, and wherein the pattern of light intensities is based on a difference between the respective row-wise sums of pixel values in the respective pixel arrays.

20. A computing device, comprising: one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out functions comprising the computer-implemented method of any one of claims 1-19.

21. A computer program comprising instructions that, when executed by a computer, cause the computer to perform steps in accordance with the method of any one of claims 1-19.

22. An article of manufacture comprising one or more non-transitory computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to carry out functions that comprise the computer-implemented method of any one of claims 1-19.

23. A computing system, comprising: means for carrying out the computer-implemented method of any one of claims 1-19.

Images