SYSTEM AND METHOD FOR CONTROLLING AN IMAGE SENSOR - Patent application

Abstract

Systems and techniques for imaging are described. The imaging system includes an image sensor with a plurality of photodetectors grouped into a first photodetector group and a second photodetector group. The imaging system can reset the image sensor. The imaging system exposes the image sensor to light from a scene. The plurality of photodetectors convert the light into an electric charge. The imaging system stores analog photodetector signals corresponding to the electric charge from each photodetector. The imaging system reads first digital pixel data from a first subset of the analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the analog photodetector signals corresponding to the second photodetector group. The imaging system uses the first digital pixel data to generate an image of the scene.

Description

[Technical field]

[0001] This application relates to image capture and / or image processing, and more particularly to systems and methods for controlling whether a first readout of first image data is performed from a first subset of pixels of an image sensor to generate an image at a first resolution, or a second readout of second image data is performed from at least a second subset of pixels of the image sensor to generate an image at a second resolution. [Background technology]

[0002]

[0002] An image sensor includes an array of photodetectors, such as photodiodes. Each photodetector receives light from a scene and converts the light into a charge based on how much light reaches the photodetector. Reading out image data from an image sensor may involve operations such as amplifying the charge generated by the photodetectors, converting an analog signal associated with the charge to a digital signal using an analog to digital converter (ADC), and / or additional image processing operations. The amount of power consumed, bandwidth used, heat generated, and / or computational resources used for image capture may be related to the number of photodetectors read out.

[0003]

[0003] Portable electronic devices such as mobile handsets, cellular phones, head-mounted displays (HMDs), or wearable devices are becoming more and more prevalent. Portable electronic devices are typically powered by a portable power storage unit, such as a battery, that is part of the portable electronic device. To remain light and small, and therefore portable, portable electronic devices typically include a relatively small battery. Because portable electronic devices have limited battery life, it is beneficial to reduce the power usage of operations performed by the portable electronic device, where possible. Some portable electronic devices include a camera. Images of different resolutions may be useful for different use cases. In some use cases, an image having a first resolution may be sufficient. In other use cases, an image having a second resolution (e.g., higher than the first resolution) may be beneficial. Summary of the Invention

[0004] In some examples, systems and techniques are described herein for controlling the readout of an image sensor to control the resolution of a captured image and corresponding power consumption. The imaging system includes an imaging circuit. In some examples, the imaging circuit includes an image sensor with an array of photodetectors, such as photodiodes. In some examples, the photodetectors of the photodetector array of the image sensor are grouped such that each photodetector is one of a set of multiple distinct groups, for example based on a pattern. In an illustrative example, some of the photodetectors of the photodetector array are in a first group, some of the photodetectors of the photodetector array are in a second group distinct from the first group, and so on. According to some examples, the imaging system uses a global shutter. According to some examples, the imaging system uses a rolling shutter. The imaging system can reset the photodetectors of its image sensor. The imaging system exposes its image sensor to light from a scene. Each of the photodetectors of the image sensor converts light from the scene into an electric charge in response to exposure to the light. According to some examples, the imaging system receives and / or stores analog photodetector signals corresponding to the charges. For example, the imaging system in some examples stores the analog photodetector signals in storage units (e.g., storage diodes) of a storage array. The imaging system reads out first digital pixel data from a first subset of the analog photodetector signals corresponding to the first photodetector group without reading out second digital pixel data from a second subset of the analog photodetector signals corresponding to the second photodetector group.

[0005]

[0005] A readout of pixel data from some of the photodetectors of an image sensor without a readout of other photodetectors of the image sensor may be referred to as a sparse readout. A sparse readout reduces power consumption, bandwidth used, heat generated, and / or computational resources used compared to a dense readout of the photodetectors of the image sensor. A readout of more photodetectors of an image sensor than a sparse readout may be referred to as a dense readout. A readout of all, or at least a subset (e.g., at least a threshold amount) of the photodetectors of an image sensor may be referred to as a dense readout. In some examples, different types of photodetectors of an image sensor may be used for different functions. For example, a first type of photodetector (e.g., an image photodetector) may be used for image capture, and a second type of photodetector (e.g., a focus photodetector) may be used for focus, such as phase detection autofocus (PDAF). A readout of all or at least a subset (e.g., at least a threshold amount) of a type of photodetector of an image sensor (e.g., photodetectors for image capture but not for focus) may be referred to as a dense readout. The readout may include analog gain, analog-to-digital conversion (ADC), digital gain, demosaicing, and / or other operations. The imaging system may generate an image of the scene at a first resolution using the first digital pixel data, at least in part by combining the first digital pixel data with an image at the first resolution. In some examples, the first resolution may be different (e.g., less than the full resolution) than the full resolution that the image sensor is capable of. In some examples, the imaging system may reset the photodetectors of the image sensor after capturing an image at the first resolution, for example, before capturing a subsequent image using the image sensor. In some cases, the imaging system may determine that an image at a second resolution (e.g., higher than the first resolution) may be desired. The imaging system may read out the second digital pixel data and combine it with the first digital pixel data to generate an image at the second resolution.

[0006] In one example, an apparatus for imaging is provided. The apparatus includes a memory and one or more processors (e.g., implemented in a circuit) coupled to the memory. The one or more processors are configured and capable of: exposing a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors being configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another; storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and generating an image of the scene using at least the first digital pixel data.

[0007] In another example, a method for imaging is provided that includes exposing a plurality of photodetectors of an image sensor to light from a scene, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to the light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another, storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors, reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and generating an image of the scene using at least the first digital pixel data.

[0008]

[0008] In another example, a non-transitory computer readable medium is provided having instructions stored thereon that, when executed by one or more processors, are configured to cause the one or more processors to expose an image sensor to light from the scene, the image sensor having a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another, store a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors, read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading a second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and generate an image of the scene using at least the first digital pixel data.

[0009] In another example, an apparatus for image processing is provided that includes: means for exposing an image sensor to light from a scene, the image sensor having a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to the light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another; means for storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; means for reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and means for generating an image of the scene using at least the first digital pixel data.

[0010]

[0010] In some aspects, to read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, the one or more processors use the imaging circuitry to selectively apply a modification to a first subset of the plurality of analog photodetector signals without applying the modification to the second subset of the plurality of analog photodetector signals.

[0011] In some aspects, an analog photodetector signal of the plurality of analog photodetector signals indicates a voltage corresponding to a charge from one of the plurality of photodetectors.

[0012]

[0012] In some aspects, the methods, apparatus, and computer readable media described above further comprise an image sensor.

[0013] In some aspects, to read the first digital pixel data from the first subset of the plurality of analog photodetector signals, the one or more processors are configured to cause one or more amplifiers to amplify each of the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into a plurality of amplified analog photodetector signals using the one or more amplifiers, and the first digital pixel data is based on the plurality of amplified analog photodetector signals. In some aspects, the methods, apparatus, and computer-readable media described above further comprise one or more amplifiers.

[0014] In some aspects, to read the first digital pixel data from the first subset of the analog photodetector signals, the one or more processors are configured to cause one or more analog-to-digital converters (ADCs) to convert the plurality of analog signals based on the first subset of the analog photodetector signals corresponding to the first photodetector group into digital signals using the one or more ADCs, and the first digital pixel data is based on the digital signals. In some aspects, the method, apparatus, and computer-readable medium described above further comprise one or more ADCs. In some aspects, the one or more processors are configured to generate the first digital pixel data based on the digital signals by at least partially processing the digital signals using one or more image processing operations, and the one or more image processing operations include at least one of digital gain, demosaicing, pixel interpolation, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, and upsampling. In some aspects, the plurality of analog signals includes a plurality of amplified analog photodetector signals, and the one or more amplifiers generate the plurality of amplified analog photodetector signals at least in part by amplifying a first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors.

[0015]

[0015] In some aspects, the one or more processors are configured to, after generating an image, determine based on the image whether to read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0016] In some aspects, the one or more processors are configured to read the first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the second photodetector group after reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, and generate a second image of the scene using at least the first digital pixel data and the second digital pixel data, the second image having a second resolution higher than the first resolution of the image, and the reading of the second digital pixel data is based on the storage of the plurality of analog photodetector signals. In some aspects, the one or more processors are configured to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group after generating the image of the scene, to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group after reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first photodetector group. In some aspects, the one or more processors are configured to identify that the image includes an area having a saliency above a saliency threshold, and in response to identifying that the image includes an area having a saliency above the saliency threshold, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors. In some aspects, the one or more processors are configured to identify that the image exhibits motion relative to a previous image of the scene captured by the image sensor that exceeds a motion threshold, and in response to identifying that the image exhibits motion relative to a previous image of the scene captured by the image sensor that exceeds the motion threshold, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors.

[0017]

[0017] In some aspects, the one or more processors are configured to, after generating the second image, determine based on the second image whether to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group, wherein the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and the image and the second image are generated without the third digital pixel data. In some aspects, the one or more processors are configured to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group after reading the second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and generate a third image of the scene using at least the first digital pixel data and the second digital pixel data and the third digital pixel data, wherein a third resolution of the third image is higher than the second resolution of the second image, wherein reading the third digital pixel data is based on storing the plurality of analog photodetector signals, wherein the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and wherein the image and the second image are generated without the third digital pixel data.

[0018]

[0018] In some aspects, the one or more processors are configured to identify that the image includes a depiction of an object, and in response to identifying that the image includes a depiction of an object, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors. In some aspects, the object is a glyph that optically encodes information, and the glyph includes at least one of a one-dimensional barcode and a two-dimensional code. In some aspects, the object includes at least a portion of a person. In some aspects, the object includes at least a portion of a vehicle. In some aspects, the object includes one or more alphanumeric characters.

[0019] In some aspects, the photodetectors of the image sensor are arranged according to a grid of blocks, each block including a respective subset of the photodetectors, and each block of the grid of blocks includes at least one photodetector of the first group of photodetectors and at least one photodetector of the second group of photodetectors. In some aspects, the first group of photodetectors is arranged across the image sensor according to a first pattern, and the second group of photodetectors is arranged across the image sensor according to a second pattern, and the first pattern and the second pattern correspond to one or more grids.

[0020]

[0020] In some aspects, the image sensor includes a memory array with a plurality of storage units, and to store each of a plurality of analog photodetector signals corresponding to charges from each of the plurality of photodetectors, the image sensor is configured to store each of the plurality of analog photodetector signals in one of the plurality of storage units of the storage array.

[0021]

[0021] In some aspects, the image sensor is configured to reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing an image. In some aspects, the one or more processors are configured to analyze the image using an object detection algorithm, and the image sensor is configured to automatically reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array in response to one or more objects not being detected in the image by the object detection algorithm to reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing an image. In some aspects, the image sensor is configured to reset each of the plurality of photodetectors of the image sensor before exposing the plurality of photodetectors to light from the scene.

[0022]

[0022] In some aspects, the image sensor is configured to move a reset pointer at a predetermined speed from a first side of the image sensor to a second side of the image sensor across a plurality of reset pointer positions, and is configured to reset a first photodetector of the plurality of photodetectors in response to the reset pointer reaching one of a plurality of reset pointer positions where a first photodetector of the plurality of photodetectors is located to reset each of the plurality of photodetectors, and is configured to move a read pointer at a predetermined speed from the first side of the image sensor to the second side of the image sensor across a plurality of read pointer positions, and is configured to read a third digital pixel data corresponding to a second photodetector of the first photodetector group in response to the read pointer reaching one of a plurality of reset pointer positions where a second photodetector of the first photodetector group is located to read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, wherein the predetermined measure is associated with a rolling shutter.

[0023] In some aspects, the one or more processors are configured to output the image. In some aspects, the methods, apparatus, and computer-readable medium described above further comprise a display, and to output the image, the one or more processors are configured to display the image using the display. In some aspects, the methods, apparatus, and computer-readable medium described above further comprise a communication transceiver, and to output the image, the one or more processors are configured to transmit the image to a receiving device using the communication transceiver.

[0024] In some aspects, the plurality of photodetectors includes a plurality of photodiodes. In some aspects, the plurality of photodetectors of the image sensor includes a plurality of image photodiodes and a plurality of focus photodiodes, the plurality of focus photodiodes are configured for phase detection autofocus (PDAF), the first photodetector group includes a first subset of the plurality of image photodiodes, and the second photodetector group includes a second subset of the plurality of image photodiodes.

[0025] In some aspects, the device is, is a part of, and / or includes a wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a head mounted display (HMD) device, a wireless communication device, a mobile device (e.g., a mobile phone and / or mobile handset and / or a so-called "smartphone" or other mobile device), a camera, a personal computer, a laptop computer, a server computer, a vehicle or a computing device or component of a vehicle, another device, or a combination thereof. In some aspects, the device includes a camera or multiple cameras for capturing one or more images. In some aspects, the device further includes a display for displaying one or more images, notifications, and / or other displayable data. In some aspects, the devices described above may include one or more sensors (e.g., one or more inertial measurement units (IMUs), such as one or more gyrometers, one or more accelerometers, any combination thereof, and / or other sensors.

[0026]

[0026] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used independently to determine the scope of the claimed subject matter, which subject matter should be understood by reference to the entire specification of this patent, any or all drawings, and appropriate portions of each claim.

[0027]

[0027] The above, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings. [Brief description of the drawings]

[0028]

[0028] Exemplary embodiments of the present application are described in detail below with reference to the following drawings. [Figure 1]

[0029] 1 is a block diagram illustrating an example architecture of an image capture and processing system, in accordance with some examples. [Diagram 2]

[0030] FIG. 1 is a block diagram illustrating an example architecture of an imaging system that performs a process for image capture, at least in part, by storing analog photodetector signals and reading out digital pixel data from the photodetector, in accordance with some examples. [Figure 3A]

[0031] 1 is a conceptual diagram illustrating an example of an image sensor including a number of pixels labeled P0 through P63, according to some examples. [Figure 3B]

[0032] FIG. 3B is a conceptual diagram illustrating an example of a storage array that temporarily stores pixel data corresponding to each of the pixels of the image sensor of FIG. 3A, according to some examples. [Figure 4A]

[0033] 1 is a conceptual diagram illustrating capture of image data via an image sensor of an imaging system using a rolling shutter, according to some examples. [Figure 4B]

[0034] 1 is a conceptual diagram illustrating capture of image data via an image sensor of an imaging system using a global shutter, in accordance with some examples. [Figure 5A]

[0035] FIG. 1 is a conceptual diagram illustrating the organization of a pixel array of an image sensor into a grid of 24 blocks, with each block containing one pixel from each group, and the organization of each of the pixels of the pixel array into one of 16 groups, in accordance with some examples. [Figure 5B]

[0036] FIG. 1 is a conceptual diagram illustrating capture of a first resolution image by performing a sparse readout of an image sensor based on reading out only pixels in a first group of 16 groups, in accordance with some examples. [Figure 5C]

[0037] FIG. 1 is a conceptual diagram illustrating capture of a low resolution image by performing a sparse readout of an image sensor based on reading out only pixels in a second group of 16 groups, in accordance with some examples. [Figure 6A]

[0038] FIG. 1 is a conceptual diagram illustrating capture of an image at a first resolution, a first image at a second resolution, and / or a second image at a second resolution by performing a sparse readout of an image sensor having a pixel array organized into a grid of four blocks, with each of the pixels in the pixel array being organized into one of thirteen groups, with each block containing 16 pixels having at least one pixel from each group, in accordance with some examples. [Figure 6B]

[0039] FIG. 1 is a conceptual diagram illustrating capture of a first resolution image and / or a first image at a second resolution by performing a sparse readout of an image sensor having a pixel array organized into a grid of four blocks, with each of the pixels in the pixel array organized into one of six groups, with each block containing nine pixels having at least one pixel from each group, in accordance with some examples. [Figure 7]

[0040] FIG. 1 is a conceptual diagram illustrating capture of a first image at a first resolution, capture of a second image at the first resolution, capture of an image at a second resolution, and capture of a third image at the first resolution, in accordance with some examples. [Figure 8]

[0041] 1 is a conceptual diagram illustrating reset pointer and read pointer movement along an image sensor in a rolling shutter imaging system, according to some examples. [Figure 9]

[0042] FIG. 1 is a conceptual diagram illustrating a rolling shutter imaging system that performs a first image capture, a second image capture, and a third image capture, according to some examples. [Figure 10A]

[0043] FIG. 1 is a conceptual diagram illustrating image capture of six image frames, each at a first resolution, via a first readout pattern using a rolling shutter imaging system having four pixel groups, in accordance with some examples. [Figure 10B]

[0044] FIG. 13 is a conceptual diagram illustrating image capture of eight image frames, each at a second resolution, via a second readout pattern using a rolling shutter imaging system having four pixel groups, in accordance with some examples. [Figure 10C]

[0045] FIG. 1 is a conceptual diagram illustrating a transition from capturing an image frame at a second resolution via a second readout pattern to capturing an image frame at a first resolution via a first readout pattern using a rolling shutter imaging system having four pixel groups, in accordance with some examples. [Figure 11]

[0046] 1 is a conceptual diagram illustrating a rolling shutter readout pattern using a rolling shutter imaging system, according to some examples. [Figure 12]

[0047] FIG. 1 is a flow diagram illustrating operations for imaging, according to some examples. [Figure 13]

[0048] FIG. 1 illustrates an example of a computing system for implementing some aspects described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029]

[0049] Specific aspects and embodiments of the present disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. In the following description, for the purposes of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting.

[0030]

[0050] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments provides those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the present application as set forth in the appended claims.

[0031]

[0051] A camera is a device that uses an image sensor to receive light and capture image frames, such as still images or video frames. The terms "image," "image frame," and "frame" are used interchangeably herein. A camera may be configured with various image capture and image processing settings. Different settings result in images with different appearances. Some camera settings, such as ISO, exposure time, aperture size, F / stop, shutter speed, focus, and gain, are determined and applied before or during the capture of one or more image frames. For example, settings or parameters may be applied to an image sensor to capture one or more image frames. Other camera settings, such as contrast, brightness, saturation, sharpness, levels, curves, or color changes, may constitute post-processing of one or more image frames. For example, settings or parameters may be applied to a processor (e.g., an image signal processor or ISP) to process one or more image frames captured by the image sensor.

[0032]

[0052] An image sensor includes an array of photodetectors, such as photodiodes. Each photodetector receives light from a scene and converts the light into an electrical charge based on how much light reaches the photodetector. Photodetector data from a given photodetector may be processed (e.g., via analog gain, analog-to-digital conversion, digital gain, and / or demosaicing) to provide pixel data for a pixel of an image captured by the image sensor. In the case of a color image sensor, different photodetectors may provide pixel data corresponding to different color components (e.g., red, green, or blue) for a given pixel. Reading out image data from an image sensor may involve operations such as amplification of the electrical charge generated by the photodetectors (e.g., analog gain), conversion of an analog signal associated with the charge to a digital signal using an analog-to-digital converter (ADC) (e.g., digitization), demosaicing, digital gain, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, upsampling, additional image processing operations, or combinations thereof. Performing a readout for every photodetector of an image sensor can have a significant impact on an imaging device in terms of power consumption, bandwidth usage, heat generated, and / or computational resource usage (e.g., for ISP and / or image processing operations). Resetting a photodetector refers to clearing the charge of the photodetector before the photodetector accepts and detects light for the capture of a new image. Resetting an image sensor may refer to resetting at least a subset of the photodetectors of an image sensor. Resetting an image sensor may refer to resetting all of the photodetectors of an image sensor.

[0033]

[0053] Portable electronic devices are generally powered by a power storage unit, such as a battery, that is part of the portable electronic device. To remain lightweight and small, and therefore portable, portable electronic devices generally include a relatively small battery. Because portable electronic devices have limited battery life, it is beneficial to reduce the power usage of operations performed by the portable electronic device as much as possible. Thus, some portable electronic devices also have less processing power than devices that connect to a power outlet, to reduce power consumption and increase battery life. Some portable electronic devices include a camera. Images of different resolutions may be useful for different use cases. In some use cases, an image having a first resolution may be sufficient. In other use cases, an image having a second resolution (e.g., higher than the first resolution) may be beneficial. In some examples, reading out image data of a first resolution may consume less battery power than reading out image data of a second resolution, or vice versa.

[0034]

[0054] Electronic devices that draw and use power from a power source may generate heat from delivering power to components that use power to perform activities and from components that use power to perform activities. Electronic devices may include heat dissipation components that aid in heat dissipation, such as heat sinks, fans, liquid coolers, tubes that transport fluids (e.g., liquids and / or gases), channels that transport fluids (e.g., liquids and / or gases), etc. To remain lightweight and small, and therefore portable, portable electronic devices generally include little space for heat dissipation components. Thus, many portable electronic devices include little or no heat dissipation components. Portable electronic devices that include heat dissipation components are generally limited in terms of the types and sizes of heat dissipation components they may include, which may also limit the effectiveness of the heat dissipation components used in such portable electronic devices. If the heat of a component of an electronic device becomes too high (e.g., exceeds a threshold temperature or heat level), the electronic device may be modified and / or become defective. For example, at high heat levels, certain components or connections between components may melt, deform, transform, change state, become brittle, change shape, break, or even be modified. Additionally, portable electronic devices are often held by a user in the user's hand, worn by a user on the user's body (e.g., as a wearable device), or stored by a user in the user's pocket adjacent to the user's body. If components of the portable electronic device get too hot (e.g., exceed a threshold temperature or heat level), the portable electronic device may burn, injure, and / or cause discomfort to the user. Some portable electronic devices may have limited heat dissipation capabilities, so it is beneficial to reduce the heat generated by operations performed by the portable electronic device, when possible.Some portable electronic devices have less processing power than devices that connect to a power outlet to reduce heat generation, to reduce the heat dissipation required to maintain a threshold temperature or threshold heat level, to reduce the amount of heat any heat dissipation components dissipate to maintain a threshold temperature or threshold heat level, or a combination thereof. Some portable electronic devices include a camera. Images of different resolutions may be useful for different use cases. In some use cases, images having a first resolution may be sufficient. In other use cases, images having a second resolution (e.g., higher than the first resolution) may be beneficial. In some examples, reading out image data at a first resolution may generate less heat (and thus reduce the heat dissipation required to maintain a threshold temperature or threshold heat level) than reading out image data at a second resolution, or vice versa.

[0035]

[0055] A rolling shutter camera can capture each frame of a still image or video by rapidly scanning across a scene from one side of the image sensor to the other side of the image sensor. Typically, a rolling shutter camera scans the scene vertically, e.g., from the top of the image sensor to the bottom of the image sensor. In some cases, a rolling shutter can scan the scene horizontally instead. In some examples, using a rolling shutter, different parts of an image of a scene are recorded at different instants. In some examples, using a rolling shutter, not all parts of an image of a scene are recorded at exactly the same instant. Images captured using a rolling shutter can depict different parts of a scene as they appeared at slightly different times (e.g., instants) in the scene. A rolling shutter camera can reset its photodetectors line by line (e.g., row by row) from one side of the image sensor to the other side of the image sensor. After resetting their photodetectors, a rolling shutter can read out pixel data from its photodetectors line by line (e.g., row by row) from one side of the image sensor to the other side of the image sensor.

[0036]

[0056] A global shutter camera may be used to capture each frame of a still image or video by simultaneously scanning the entire area of ​​the image. A global shutter camera may include a storage array of storage units, such as storage diodes, that store photodetector data from each photodetector of the image sensor, and therefore for each pixel of a full resolution image that the image sensor can generate. The global shutter camera may stop the exposure on the photodetector array and move the accumulated charge from the photodetector array to the storage array. The storage units of the storage array may be reset after a period of time, for example, just before another image is captured by the global shutter camera. In some examples, the storage array may store the photodetector data as analog photodetector data (e.g., the charge on the individual photodetectors before or after amplification).

[0037]

[0057] Systems and techniques are described herein for controlling whether a camera performs a first readout from a first subset of pixels of its image sensor to generate an image of a first resolution at a first power consumption, or whether the camera performs a second readout from at least a second subset of pixels of its image sensor to generate an image of a second resolution at a second power consumption. The imaging system may include an image sensor with an array of photodetectors, such as photodiodes. The photodetectors of the photodetector array of the image sensor may be grouped such that each photodetector is in one of a set of multiple distinct groups. For example, some of the photodetectors may be in a first group, some of the photodetectors may be in a second group distinct from the first group, and so on. The photodetectors of the photodetector array may be grouped into different groups based on a predetermined pattern. For example, the photodetectors of the photodetector array may be grouped into a grid of blocks, each having a predetermined dimension (e.g., n photodetectors in width by m photodetectors in height), each including one or more photodetectors from each group, possibly arranged according to a predetermined order within the blocks. In this example, n and m are both integer values ​​equal to or greater than 1 and may be equal to or different from each other. For example, each block of photodetectors may include photodetectors in a first group, followed by photodetectors in a second group, and so on. The imaging system may use a global shutter or a rolling shutter.

[0038]

[0058] The imaging system can reset the photodetectors in the photodetector array of its image sensor line by line (e.g., a rolling shutter), globally (e.g., a global shutter), group by group, or a combination thereof. The imaging system can expose its image sensor and the photodetector array thereon to light from a scene. Each of the photodetectors is configured to and can convert light from the scene into an electric charge in response to exposure to light from the scene. The imaging system can receive and / or store an analog photodetector signal corresponding to the electric charge from each of the photodetectors. In some examples, the imaging system can use a global shutter and store each of the analog photodetector signals in one of the storage units of the storage array. In some examples, the imaging system can use a rolling shutter and store each of the analog photodetector signals in the photodetectors until read out from the photodetectors.

[0039]

[0059] The imaging system can read out the first digital pixel data from the first subset of analog photodetector signals corresponding to the first photodetector group without reading the second digital pixel data from the second subset of analog photodetector signals corresponding to the second photodetector group (and / or without reading additional digital pixel data from additional subsets of analog photodetector signals corresponding to additional photodetector groups, such as a third or fourth photodetector group). The imaging device reading out only digital pixel data from a subset of its photodetector groups may be referred to as a sparse readout. A sparse readout may also be referred to as a low power mode. The readout may include amplification (e.g., analog gain) of the analog photodetector signal. The readout may include conversion of the analog photodetector signal to a digital signal (e.g., via an ADC). In some examples, the digital signal from the ADC is the first digital pixel data. In some examples, the digital signal from the ADC is further processed via one or more image processing operations (e.g., digital gain, demosaicing, pixel interpolation, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balancing, automatic black balancing, downsampling, and upsampling) to become the first digital pixel data. The imaging system can capture a low-resolution image of the scene, at least in part, by combining the first digital pixel data into the low-resolution image. In some examples, the imaging system can reset the photodetectors of the image sensor after capturing the low-resolution image, e.g., before capturing a subsequent image.

[0040]

[0060] In some examples, the imaging system may determine that a high-resolution image may be desirable based on, for example, a user interface input by a user, based on the imaging device determining that the low-resolution image includes a representation of an object such as a barcode or quick response (QR) code that may be easier to scan and / or interpret in the high-resolution image, based on the imaging device determining that the low-resolution image includes a representation of an object such as a human face that may be easier to recognize using the high-resolution image, based on the imaging device determining that the low-resolution image includes a representation of an alphanumeric character that may be easier to analyze using the high-resolution image, based on the imaging device determining that the low-resolution image includes an image area that has a saliency above a saliency threshold (e.g., has at least a threshold size), based on the imaging device determining that the low-resolution image includes movement (e.g., above a movement threshold) compared to a previous image, or combinations thereof. After reading the first digital pixel data, the imaging device may read second digital pixel data from a second subset of the analog photodetector signals corresponding to a second group of photodetectors from the stored analog photodetector signals. The imaging device can capture a high-resolution image of the scene at least in part by combining both the first digital pixel data and the second digital pixel data into a high-resolution image. The high-resolution image has a higher resolution than the low-resolution image. When the imaging device reads out the digital pixel data from more of its photodetectors than in a sparse readout, it may be referred to as a dense readout. When the imaging device reads out the digital pixel data from all of its photodetectors, it may be referred to as a full readout. A dense readout may include a full readout. A dense readout may also be referred to as a high power mode.

[0041]

[0061] In some examples, the imaging device can read second digital pixel data from a second subset of analog photodetector signals corresponding to a second group of photodetectors from the stored analog photodetector signals before the photodetectors and / or the storage unit of the storage array are reset. Thus, the imaging device can determine on the fly to switch from a low-resolution image readout to a high-resolution image readout even before the capture of the next frame begins. This is a technical improvement over conventional cameras where a change in settings is generally not applied until several frames after a request for the change is made. The imaging device provides technical improvements such as reduced power consumption, reduced bandwidth usage, and / or reduced computational resource usage. These improvements are brought about by using a sparse readout when a low-resolution image is sufficient and using a dense readout (e.g., a full readout) when requested (e.g., by a user or by the imaging device itself) for operations where a high-resolution image is beneficial. In some examples, the sparse readout can be used for cameras that are left on and record for long periods of time, such as always-on (AON) cameras, and / or for persistent imaging applications to reduce power usage. The imaging system can switch from sparse to fine readout on demand (eg, by a user or by the imaging device itself) for operations where high resolution images are beneficial.

[0042]

[0062] Various aspects of the application are described with respect to the figures. Figure 1 is a block diagram illustrating the architecture of an image capture and processing system 100. The image capture and processing system 100 includes various components used to capture and process an image of a scene (e.g., an image of a scene 110). The image capture and processing system 100 can capture a standalone image (or photo) and / or can capture a video that includes multiple images (or video frames) in a particular sequence. A lens 115 of the system 100 faces the scene 110 and accepts light from the scene 110. The lens 115 bends the light towards an image sensor 130. The light received by the lens 115 passes through an aperture controlled by one or more control mechanisms 120 and is received by the image sensor 130.

[0043]

[0063] The one or more controls 120 may control exposure, focus, and / or zoom based on information from image sensor 130 and / or based on information from image processor 150. The one or more controls 120 may include multiple mechanisms and components. For example, the control 120 may include one or more exposure controls 125A, one or more focus controls 125B, and / or one or more zoom controls 125C. The one or more controls 120 may include additional controls beyond those shown, such as controls to control analog gain, flash, HDR, depth of field, and / or other image capture characteristics.

[0044]

[0064] The focus control mechanism 125B of the control mechanism 120 can obtain the focus setting. In some examples, the focus control mechanism 125B stores the focus setting in a memory register. Based on the focus setting, the focus control mechanism 125B can adjust the position of the lens 115 relative to the position of the image sensor 130. For example, based on the focus setting, the focus control mechanism 125B can move the lens 115 closer to or farther from the image sensor 130 by actuating a motor or servo, thereby adjusting the focus. In some cases, additional lenses, such as one or more microlenses on each photodetector (e.g., photodetector) of the image sensor 130, can be included in the system 100, each of which bends light received from the lens 115 toward a corresponding photodetector before the light reaches the photodetector. The focus setting can be determined via contrast detection autofocus (CDAF), phase detection autofocus (PDAF), or some combination thereof. The focus settings may be determined using the control mechanism 120, the image sensor 130, and / or the image processor 150. The focus settings may be referred to as image capture settings and / or image processing settings.

[0045]

[0065] The exposure control 125A of the control mechanism 120 can obtain an exposure setting. In some cases, the exposure control 125A stores the exposure setting in a memory register. Based on this exposure setting, the exposure control 125A can control the size of the aperture (e.g., aperture size or F / stop), the duration the aperture is open (e.g., exposure time or shutter speed), the sensitivity of the image sensor 130 (e.g., ISO speed or film speed), the analog gain applied by the image sensor 130, or any combination thereof. The exposure setting may be referred to as an image capture setting and / or an image processing setting.

[0046]

[0066] The zoom control 125C of the control mechanism 120 can obtain the zoom setting. In some examples, the zoom control 125C stores the zoom setting in a memory register. Based on the zoom setting, the zoom control 125C can control the focal length of an assembly of lens elements (lens assembly) including the lens 115 and one or more additional lenses. For example, the zoom control 125C can control the focal length of the lens assembly by actuating one or more motors or servos to move one or more of the lenses relative to each other. The zoom setting may be referred to as an image capture setting and / or an image processing setting. In some examples, the lens assembly may include a parfocal zoom lens or a variable focus zoom lens. In some examples, the lens assembly may include a focusing lens (which may be the lens 115 in some cases) that first accepts light from the scene 110, and then the light passes through an afocal zoom system between the focusing lens (e.g., the lens 115) and the image sensor 130 before the light reaches the image sensor 130. In some cases, an afocal zoom system may include two positive (e.g., converging, convex) lenses of equal or similar focal lengths (e.g., within a threshold difference) with a negative (e.g., diverging, concave) lens between them. In some cases, the zoom control 125C moves one or more of the lenses in the afocal zoom system, such as one or both of the negative and positive lenses.

[0047]

[0067] Image sensor 130 includes one or more arrays of photodetectors, such as photodiodes or other light-sensitive elements. Each photodetector measures an amount of light that ultimately corresponds to a particular pixel in the image generated by image sensor 130. In some cases, different photodetectors may be covered by different color filters, and thus may measure light that matches the color of the filter covering the photodetector. For example, a Bayer color filter includes red, blue, and green filters, and each pixel of the image is generated based on red light data from at least one photodetector covered in a red filter, blue light data from at least one photodetector covered in a blue filter, and green light data from at least one photodetector covered in a green filter. Other types of color filters may use yellow, magenta, and / or cyan (also called "emerald") color filters instead of or in addition to red, blue, and / or green filters. Some image sensors may be completely devoid of color filters, and instead use different photodetectors (possibly stacked vertically) throughout the pixel array. Different photodetectors across the pixel array may have different spectral sensitivity curves and therefore respond to different wavelengths of light. Monochrome image sensors may also lack color filters and therefore no color depth.

[0048]

[0068] In some cases, image sensor 130 may alternatively or additionally include an opaque and / or reflective mask that blocks light from reaching some photodetectors or portions of some photodetectors at certain times and / or from certain angles, which may be used for phase detection autofocus (PDAF). Image sensor 130 may also include an analog gain amplifier for amplifying an analog signal output by the photodetector, and / or an analog-to-digital converter (ADC) 132 for converting the analog signal output of the photodetector (and / or amplified by the analog gain amplifier) ​​to a digital signal. In some cases, instead or in addition, some components or functions discussed with respect to one or more of control mechanisms 120 may be included within image sensor 130. The image sensor 130 may be a charge-coupled device (CCD) sensor, an electron-multiplying CCD (EMCCD) sensor, an active-pixel sensor (APS), a complimentary metal-oxide semiconductor (CMOS), an n-type metal-oxide-semiconductor (NMOS), a hybrid CCD / CMOS sensor (e.g., sCMOS), or some other combination thereof.

[0049]

[0069] In some examples, the array of photodetectors of the image sensor 130 may include a focus photodetector and an image photodetector. Both the image photodetector and the focus photodetector may accept light from the scene being photographed. The photodiode data from the image photodetector may be used to capture an image of the scene as described herein. The photodiode data from the focus photodetector may be used for PDAF. Both the image photodetector and the focus photodetector may be under a microlens. In some examples, the microlens on the focus photodiode may be partially masked with an opaque or reflective mask such that one side of the microlens allows light from the scene to pass to the focus photodiode and the other side of the microlens blocks light from the scene from reaching the focus photodiode. This masking may limit the light reaching the focus photodiode to light approaching the focus photodiode from a particular angular range that may be useful for PDAF. In some examples, the microlens may be split between two or more focus photodiodes such that the light reaching each focus photodiode is limited to light that passes through a particular portion of the microlens and thus to light approaching the photodiode from a particular angular range that may be useful for PDAF. In some examples, image data from the focus photodiode of image sensor 130 may be discarded by image capture and processing system 100. ISP 154, host processor 152, and / or image processor 150 may treat pixel data from the focus photodiode as missing pixels and / or bad pixels that ISP 154, host processor 152, and / or image processor 150 may correct using missing pixel correction, bad pixel correction, and / or interpolation. In some examples, image data from the focus photodiode of image sensor 130 may be used by image capture and processing system 100 in generating a captured image, for example, with some correction based on neighboring pixels (e.g., using missing pixel correction, bad pixel correction, and / or interpolation). A pixel corresponding to an image photodiode may be referred to as an image pixel. A pixel corresponding to a focus photodiode may be referred to as a focus pixel.

[0050]

[0070] Image processor 150 may include one or more processors, such as one or more image signal processors (ISP) (including ISP 154), one or more host processors (including host processor 152), and / or one or more of any other types of processors 1310 discussed with respect to computing system 1300. Host processor 152 may be a digital signal processor (DSP) and / or other types of processors. In some implementations, image processor 150 is a single integrated circuit or chip (e.g., referred to as a system-on-chip or SoC) that includes host processor 152 and ISP 154. In some cases, the chip may include one or more input / output ports (e.g., input / output (I / O) ports 156), central processing units (CPUs), graphics processing units (GPUs), broadband modems (e.g., 3G, 4G or LTE, 5G, etc.), memory, connectivity components (e.g., Bluetooth, Global Positioning System (GPS), etc.), any combination thereof, and / or other components.The I / O ports 156 may include any suitable input / output ports or interfaces according to one or more protocols or specifications, such as an Inter-Integrated Circuit 2 (I2C) interface, an Inter-Integrated Circuit 3 (I3C) interface, a Serial Peripheral Interface (SPI) interface, a serial General Purpose Input / Output (GPIO) interface, a Mobile Industry Processor Interface (MIPI) (e.g., a MIPI CSI-2 physical (PHY) layer port or interface, etc.), an Advanced High-performance Bus (AHB) bus, any combination thereof, and / or other input / output ports. In one illustrative example, the host processor 152 may communicate with the image sensor 130 using an I2C port and the ISP 154 may communicate with the image sensor 130 using a MIPI port.

[0051]

[0071] In some examples, the image capture and processing system 100 may include a storage array 135 that includes an array of storage units. In some examples, the storage units may be storage diodes, memory diodes, memory units, or combinations thereof. The storage array 135 may store photodetector data from the photodetector array of the image sensor 130 before the photodetector data is converted to digital by the ADC 132. In some examples, each storage unit of the storage array 135 stores analog photodiode data (e.g., charge or a signal indicative of charge) of a different one of the photodetectors of the photodetector array of the image sensor 130, as shown, for example, with respect to the image sensor 310 of FIG. 3A and the corresponding storage array 320 of FIG. 3B. The storage array 135 may provide the stored analog photodiode data to the ADC 132 and / or the image processor 150 (e.g., the ISP 154 and / or the host processor 152) for readout, as shown, for example, with respect to the imaging system of FIG. 2.

[0052]

[0072] Image processor 150 may perform several tasks, such as demosaicing, color space conversion, image frame downsampling, pixel interpolation, automatic exposure (AE) control, automatic gain control (AGC), CDAF, PDAF, automatic white balance, merging image frames to form HDR images, image recognition, object recognition, feature recognition, accepting input, managing output, managing memory, or some combination thereof. Image processor 150 may store image frames and / or processed images in random access memory (RAM) 140 and / or 1320, read-only memory (ROM) 145 and / or 1325, a cache, a storage unit, another storage device, or some combination thereof.

[0053]

[0073] Various input / output (I / O) devices 160 may be connected to image processor 150. I / O devices 160 may include a display screen, a keyboard, a keypad, a touch screen, a track pad, a touch sensitive screen, a printer, any other output device 1335, any other input device 1345, or some combination thereof. In some cases, captions may be entered into image processing device 105B through a physical keyboard or keypad of I / O device 160 or through a virtual keyboard or keypad of a touch screen of I / O device 160. I / O 160 may include one or more ports, jacks, or other connectors that enable a wired connection between system 100 and one or more peripheral devices, through which system 100 may receive data from and / or send data to one or more peripheral devices. I / O 160 may include one or more wireless transceivers that enable a wireless connection between system 100 and one or more peripheral devices, through which system 100 may receive data from and / or transmit data to one or more peripheral devices. The peripheral devices may include any of the types of I / O devices 160 previously described, and may themselves be considered I / O devices 160 when coupled to a port, jack, wireless transceiver, or other wired and / or wireless connector.

[0054]

[0074] In some cases, image capture and processing system 100 may be a single device. In some cases, image capture and processing system 100 may be two or more separate devices including image capture device 105A (e.g., a camera) and image processing device 105B (e.g., a computing device coupled to a camera). In some implementations, image capture device 105A and image processing device 105B may be coupled, for example, via one or more wires, cables, or other electrical connectors and / or wirelessly via one or more wireless transceivers. In some implementations, image capture device 105A and image processing device 105B may be separate from one another.

[0055]

[0075] As shown in Figure 1, a vertical dashed line divides the image capture and processing system 100 of Figure 1 into two portions, which respectively represent image capture device 105A and image processing device 105B. Image capture device 105A includes lens 115, control mechanism 120, and image sensor 130. Image processing device 105B includes image processor 150 (including ISP 154 and host processor 152), RAM 140, ROM 145, and I / O 160. In some cases, some components shown in image capture device 105A, such as ISP 154 and / or host processor 152, may be included within image capture device 105A.

[0056]

[0076] The image capture and processing system 100 may include an electronic device, such as a mobile or fixed telephone handset (e.g., a smartphone, a mobile phone, etc.), a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the image capture and processing system 100 may include one or more wireless transceivers for wireless communication, such as cellular network communication, 802.11 wi-fi communication, wireless local area network (WLAN) communication, or any combination thereof. In some implementations, the image capture device 105A and the image processing device 105B may be different devices. For example, the image capture device 105A may include a camera device, and the image processing device 105B may include a computing device, such as a mobile handset, a desktop computer, or other computing device.

[0057]

[0077] Although image capture and processing system 100 is shown as including several components, one skilled in the art will appreciate that image capture and processing system 100 may include many more components than those shown in FIG. 1. The components of image capture and processing system 100 may include software, hardware, or one or more combinations of software and hardware. For example, in some implementations, the components of image capture and processing system 100 may include and / or be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable electronic circuits (e.g., microprocessors, GPUs, DSPs, CPUs, and / or other suitable electronic circuits), and / or may include and / or be implemented using computer software, firmware, or any combination thereof, to perform various operations described herein. The software and / or firmware may include one or more instructions stored on a computer-readable storage medium and executable by one or more processors of an electronic device implementing image capture and processing system 100.

[0058]

[0078] 2 is a block diagram illustrating an example architecture of an imaging system that implements a process 200 for image capture at least in part by storing an analog photodetector signal 250 and reading out digital pixel data 255 from a photodetector 215. The image capture system includes a photodetector 215, such as a photodiode or a phototransistor. The photodetector 215 may be a photodetector of a photodetector array of an image sensor, such as image sensor 130 or image sensor 310. The photodetector 215 may receive photons of light 205 from a scene 210. The scene 110 may be an example of a scene 210. The light 205 that reaches the photodetector 215 may be referred to as incident light. The photodetector 215 may convert the light 205 into an electric charge. The electric charge may form a voltage 225 and may flow as an analog photodetector signal (e.g., as a current). In some examples, a filter 220 (e.g., a low-pass filter, a high-pass filter, a band-pass filter), for example, formed using a capacitor to ground as shown in the imaging system of FIG. 2, can filter out noise from the analog photodetector signal from the photodetector 215. In some examples, the filter 220 can be omitted. The analog photodetector signal can flow to an amplifier 230, which can amplify the analog photodetector signal and therefore provide an analog gain to the analog photodetector signal. Here, the amplified analog photodetector signal can flow from the amplifier 230 to an analog-to-digital converter (ADC) 235. In some examples (not shown), another filter (similar to the filter 220, but separate from the filter 220) can filter the analog photodetector signal on the way from the amplifier 230 to the ADC 235. The ADC 235 can be an example of the ADC 132. The ADC 235 converts the analog photodetector signal into digital pixel data. In some examples, image processor 240 may process and / or modify the digital pixel data using one or more image processing operations, possibly based on other digital pixel data from ADC 235 (but from the same image sensor) corresponding to other photodiodes separate from photodiode 215.For example, image processor 240 may process and / or modify the digital pixel data at least in part by adjusting digital gain, demosaicing, pixel interpolation, performing missing pixel correction, performing bad pixel correction, adjusting brightness, adjusting contrast, adjusting saturation adjustment, adjusting histograms, performing color space conversion, performing automatic white balance (AWB), performing automatic black balance (ABB), downsampling, upsampling, or a combination thereof. Image processor 240 may be an example of ISP 154, host processor 152, image processor 150, image processing device 105B, or a combination thereof.

[0059]

[0079] The term digital pixel data 255 may refer to digital pixel data generated by ADC 235 without (yet) being processed by image processor 240, or digital pixel data generated by ADC 235 that is also processed and / or modified by image processor 240. Digital pixel data 255 may be combined with digital pixel data corresponding to other photodiodes and / or pixels of the image sensor to perform image capture 270 to capture an image.

[0060]

[0080] The analog photodetector signal 250 may refer to the analog photodetector signal output by the photodetector 215 but not yet filtered by the filter 220 or not yet amplified by the amplifier 230, the analog photodetector signal filtered by the filter 220 but not yet amplified by the amplifier 230, the analog photodetector signal amplified by the amplifier 230, the analog photodetector signal amplified by the amplifier 230 and then filtered by a second filter (not shown), or a combination thereof. The analog photodetector signal 250 may be stored in a storage unit 260 of the storage array 265. The storage array 265 may be an example of the storage array 135 and / or the storage array 320. The storage unit 260 may be an example of one of the storage units S0-S63 of FIG. 3B. The storage unit 260 may be a storage diode, a memory diode, a memory unit, or a combination thereof.

[0061]

[0081] FIG. 3A is a conceptual diagram illustrating an example of an image sensor 310 that includes a plurality of pixels labeled P0-P63. Image sensor 310 is nine pixels wide and nine pixels tall. The pixels are numbered sequentially from P0 to P63, counting from left to right within each row, starting with the top row and counting up toward the bottom row. Image sensor 310 may be an example of image sensor 130. Each of the individual pixels (P0-P63) of image sensor 310 may include one or more photodetectors, such as photodetector 215. In some examples, each of the elements labeled P0-P63 may be an individual photodetector, such as photodetector 215. In some examples, pixels P0-P63 of FIG. 3A are all image pixels. In some examples, photodetectors P0-P63 of FIG. 3A are all image photodetectors. In some examples, pixels P0-P63 of FIG. 3A include image pixels and focus pixels. In some examples, the photodetectors P0-P63 in FIG. 3A include image photodetectors and focus photodetectors.

[0062]

[0082] Figure 3B is a conceptual diagram illustrating an example of a storage array 320 that temporarily stores pixel data corresponding to each of the pixels of image sensor 310 of Figure 3A. Storage array 320 includes a number of storage units labeled S0 through S63. Storage array 320, like image sensor 310, is nine cells (pixels) wide and nine cells (pixels) high. The cells are numbered sequentially from left to right within each row, starting with the top row and counting upwards to the bottom row, from S0 through S63.

[0063]

[0083] Each pixel data set in each cell of the storage array 320 corresponds to a pixel in the image sensor 310. For example, the pixel data set stored in cell S0 in the storage array 320 corresponds to pixel P0 in the image sensor 310, the pixel data set stored in cell S1 in the storage array 320 corresponds to pixel P1 in the image sensor 310, and so on. The pixel data set in the storage array 320 may represent one or more analog pixel signals and / or one or more analog photodetector signals corresponding to one or more photodetectors (e.g., photodiodes) of the image sensor that represent the pixel data set for that pixel. In some examples, each pixel Pa of pixels P0-P63 of the image sensor 310 includes a set of b photodetectors, and a corresponding cell Ma of cells S0-S63 of the storage array 320 includes a set of b memory diodes that correspond to and store pixel data from the set of b photodetectors. In this example, a can be any integer between 0 and 63 (inclusive) and b can be any integer greater than or equal to 1.

[0064]

[0084] FIG. 4A is a conceptual diagram 400A illustrating the capture of image data via an image sensor of an imaging system using a rolling shutter. The capture of different portions of image data by the imaging system is shown relative to a horizontal time axis 490, where time flows from left to right, and relative to a vertical row axis 495, which shows different rows of the image sensor. The time axis 490 is sometimes referred to as a timeline. Different rows of the image sensor are shown along different horizontal lines that are perpendicular to the vertical row axis 495 and intersect the vertical row axis 495 at different times. Examples of image sensors may include image sensor 310 and / or image sensor 130. The image sensor may include an array of photodetectors and / or pixels arranged in rows. In some examples, the rolling shutter may be part of the control mechanism 120. In some examples, the rolling shutter may be an electronic rolling shutter that electronically controls when specific photodiodes and / or pixels (and / or their rows) are reset, exposed, and / or read out. In some examples, the rolling shutter may be a mechanical rolling shutter that physically blocks the image sensor from time to time and physically reveals the image sensor one row at a time (e.g., using a rolling or rotating motion of a physical shutter).

[0065]

[0085] Each row shown includes a reset block. The length of the reset block along the time axis 490 represents the period of time the row is being reset by the imaging system. Each row shown includes an exposure block followed by a reset block. The length of the exposure block along the time axis 490 represents the period of time the row (e.g., the photodiodes and / or pixels in the row) are exposed to light from the scene by the imaging system. Each row shown includes an exposure block followed by a readout block. The length of the readout block along the time axis 490 represents the period of time image data is being read out from the row (e.g., the photodiodes and / or pixels in the row) by the imaging system.

[0066]

[0086] Eight rows are shown, including row 402, row 412, row 422, row 432, row 442, row 452, row 462, and row 472. Reset block 404 in row 402 is the first operation performed along time axis 490. Reset block 414 in row 412 starts after reset block 404 in row 402. Reset block 424 in row 422 starts after reset block 414 in row 412. This pattern continues, with each successive reset block along row axis 495 starting after the reset block in the row above it. This pattern applies to reset block 404, reset block 414, reset block 424, reset block 434, reset block 444, reset block 454, reset block 464, and reset block 474. Some of the durations corresponding to different reset blocks may overlap.

[0067]

[0087] Similarly, each successive exposure block along the row axis 495 starts after the exposure block of the row above it. This pattern applies to exposure block 406, exposure block 416, exposure block 426, exposure block 436, exposure block 446, exposure block 456, exposure block 466, and exposure block 476. Some of the durations corresponding to different exposure blocks may overlap. Similarly, each successive readout block along the row axis 495 starts after the readout block of the row above it. This pattern applies to readout block 408, readout block 418, readout block 428, readout block 438, readout block 448, readout block 458, readout block 468, and readout block 478. Some of the durations corresponding to different readout blocks may overlap. An exposure block may include operations related to integration of data associated with a row. In some examples, the reset, integration, and / or readout operations scan across the image sensor in a raster scan order.

[0068]

[0088] FIG. 4B is a conceptual diagram 400B illustrating the capture of image data via an image sensor of an imaging system using a global shutter. The capture of image data using an imaging system with a global shutter in FIG. 4B is shown along the same time axis 490 and row axis 495 as in FIG. 4A. The capture of image data using an imaging system with a global shutter in FIG. 4B is shown using the same set of eight rows and reset blocks, exposure blocks, and readout blocks as in FIG. 4A, but arranged in a different manner permitted by the storage array. An example of a storage array includes storage array 320.

[0069]

[0089] In Figure 4B, the reset blocks (e.g., reset block 404, reset block 414, reset block 424, reset block 434, reset block 444, reset block 454, reset block 464, and / or reset block 474) may begin at approximately the same time. In Figure 4B, the reset blocks may end at approximately the same time. In Figure 4B, the duration along time axis 490 that each of the reset blocks spans may be approximately coincident, contemporaneous, and / or simultaneous.

[0070]

[0090] 4B, reading out multiple rows of an image sensor can amount to a significant amount of the time used in capturing image data using a global shutter. Thus, skipping readouts for certain rows, pixels, and / or photodiodes can result in faster and more efficient capture of image data.

[0071]

[0091] It should be understood that imaging systems with a global shutter may delay readout further in time after exposure than shown in FIG. 4B because charge from the photodiodes may be stored in the storage array for a period of time. In some examples, readout of a particular row, pixel, and / or photodiode may occur multiple times (e.g., two or more times) after exposure while charge from the photodiodes is still stored in the storage array. For example, in the context of FIG. 7, capture of image 735 at first resolution 727 and capture of image 755 at second resolution 757 may involve two readouts of a particular row, pixel, and / or photodiode after reset 732 (and / or integrate 730) and before reset 742. In some imaging systems with a global shutter, particular rows (e.g., two or more adjacent rows) may be read out simultaneously and / or contemporaneously.

[0072]

[0092] 5A is a conceptual diagram 500A illustrating the organization of the pixel array of image sensor 501 into a grid 502 of 24 blocks 504, with each block containing one pixel from each group, and the organization of each of the pixels of the pixel array into one of 16 groups (505-580). The pixel array of image sensor 501 is 24 pixels wide by 16 pixels high. Each of the blocks 504 is 4 pixels wide by 4 pixels high. Thus, the pixel array of image sensor 501 is 6 blocks wide by 4 blocks high.

[0073]

[0093] The blocks 504 are shown spaced apart with gaps between them to emphasize the organization of the grid 502. It should be understood that the blocks 504 may be adjacent to one another on the image sensor 501 itself without these gaps. The exemplary block 503 of the blocks 504 is outlined with a dashed rounded rectangle circle. Each of the blocks 504 includes a total of 16 pixels. Each of the blocks 504 includes one pixel from each of the 16 groups (505-580). Specifically, each of the blocks 504 includes one pixel of the first group 505, one pixel of the second group 510, one pixel of the third group 515, one pixel of the fourth group 520, one pixel of the fifth group 525, one pixel of the sixth group 530, one pixel of the seventh group 535, one pixel of the eighth group 540, one pixel of the ninth group 545, one pixel of the tenth group 550, one pixel of the eleventh group 555, one pixel of the twelfth group 560, one pixel of the thirteenth group 565, one pixel of the fourteenth group 570, one pixel of the fifteenth group 575, and one pixel of the sixteenth group 580. Each pixel in the pixel array is labeled in Figures 5A-5C and 6A-6B with the group to which it belongs. Within each block, the grouping of pixels goes from the top row down and then from left to right within each row, so the top left pixel in each block is in the first group 505 and the bottom right pixel in each block is in the sixteenth group 580.

[0074]

[0094] FIG. 5B is a conceptual diagram 500B illustrating capture of an image 585 at a first resolution 587 by performing a sparse readout 586 of the image sensor 501 based on reading out only pixels in a first group 505 of the 16 groups (505-580). A selection 588 is made to select the first group 505. The selection 588 may be made based on a user interface input received from a user interface. The selection 588 may be made automatically by the imaging system. In FIG. 5B, all pixels belonging to the first group 505 are highlighted using a thick black outline. The imaging system captures the image 585 at a first resolution 587 by performing a sparse readout 586 of the pixels belonging to the first group 505 and not performing a readout of any other pixels belonging to any of the other groups (510-580) and by combining pixel data from all of the pixels belonging to the first group 505 into the image 585 at the first resolution 587. Sparse readout allows capture of an image at a given resolution lower than the full resolution of the image sensor 501 is capable of, without downscaling or binning, as in sparse readout 586 of FIG. 5B or sparse readout 592 of FIG. 5C, with reduced power consumption, bandwidth usage, and computational resource usage. The pixels in the first group 505 may include image pixels, focus pixels, or a combination thereof. The pixels in the other groups (510-580) may include image pixels, focus pixels, or a combination thereof. The first resolution 587 is lower than the full resolution of the image sensor 501 is capable of, assuming the full resolution of the image sensor 501 is capable of using pixels from all 16 groups (505-580).

[0075]

[0095] FIG. 5C is a conceptual diagram 500C illustrating capture of a first resolution 587 image 590 by performing a sparse readout 592 of the image sensor based on reading out only pixels in a second group 510 of the 16 groups. A selection 594 is made to select the second group 510. The selection 594 may be made based on user interface input received from a user interface. The selection 594 may be made automatically by the imaging system. All of the pixels that belong to the second group 510 are highlighted using a thick black outline in FIG. 5C. The imaging system captures the first resolution 587 image 590 by performing a sparse readout 592 of the pixels that belong to the second group 510 and by combining the pixel data from all of the pixels that belong to the second group 510 into the first resolution 587 image 590 without performing a readout of any other pixels that belong to any of the other groups (505 and 515-580). The pixels in the second group 510 may include image pixels, focus pixels, or a combination thereof. The pixels in the other groups (505 and 515-580) may include image pixels, focus pixels, or a combination thereof. The first resolution 587 is less than the full resolution of which the image sensor 501 is capable.

[0076]

[0096] In some examples, the imaging system may alternate which group is selected for sparse readout in each subsequent frame in a sequence of image frames (e.g., for a video). For example, the imaging system may perform a sparse readout based on the first group 505 (as in FIG. 5B) for a first image frame, a sparse readout based on the second group 510 (as in FIG. 5C) for a second image frame, a sparse readout based on the third group 515 for a third image frame, and so on. The imaging system may cycle through the groups and eventually perform a sparse readout on all groups. In this way, the imaging system may gradually collect a full resolution view of the scene, at least for static portions of the scene.

[0077]

[0097] The blocks 504 in the lattice 502 of the conceptual diagrams 500A-500C are all uniform in shape and are all rectangular in shape. In some examples, at least some of the blocks 504 may be another shape, such as a rectangle or another polygonal shape. In some examples, at least some of the blocks 504 may be polyhedrons capable of forming a polyhedral lattice. In some examples, at least some of the blocks 504 may be regular polyhedrons capable of forming a regular polyhedral lattice. The blocks 504 in the lattice 502 of the conceptual diagrams 500A-500C are all uniform in size. In some examples, at least some of the blocks 504 may be another size. The lattice 502 may be referred to as a grid.

[0078]

[0098] FIG. 6A is a conceptual diagram 600A illustrating the capture of an image 610 at a first resolution 612, a first image 620 at a second resolution 622, and / or a second image 630 at a second resolution 622 by performing a sparse readout of an image sensor 601A with a pixel array organized into a grid 602A of four blocks 603A, with each of the pixels of the pixel array organized into one of 13 groups, with each block containing 16 pixels with at least one pixel from each group. The pixel array of the image sensor 601A is 8 pixels wide by 8 pixels high. Like the blocks 504 of FIGS. 5A-5C, each of the blocks 603A is 4 pixels wide by 4 pixels high. Thus, the pixel array of the image sensor 601A is 2 blocks wide by 2 blocks high.

[0079]

[0099] Each of the blocks 603A includes a total of 16 pixels. Each of the blocks 603A includes one pixel from each of the 13 groups. The 13 groups include the 16 groups of Figures 5A-5C without the third group 515, the ninth group 545, and the eleventh group 555. Each of the blocks 603A includes four pixels in the first group 505 and one pixel from each of the remaining groups of the 13 groups. Specifically, each of blocks 603A includes four pixels of the first group 505, one pixel of the second group 510, one pixel of the fourth group 520, one pixel of the fifth group 525, one pixel of the sixth group 530, one pixel of the seventh group 535, one pixel of the eighth group 540, one pixel of the tenth group 550, one pixel of the twelfth group 560, one pixel of the thirteenth group 565, one pixel of the fourteenth group 570, one pixel of the fifteenth group 575, and one pixel of the sixteenth group 580.

[0080]

[0100] All 16 pixels belonging to the first group 505 (4 per block 603A) are highlighted in Fig. 6A using a thick black outline. The imaging system captures the first resolution 612 image 610 by performing a sparse readout based on a selection 615 of the first group 505 without reading out any of the other groups (510, 520, 525, 530, 535, 540, 550, 560, 565, 570, 575, 580) and by combining all of the pixels belonging to the first group 505 into the first resolution 612 image 610.

[0081]

[0101] All four pixels belonging to the sixth group 530 (one per block 603A) are highlighted in Fig. 6A using a triple black line outline. The imaging system captures the first image 620 at the second resolution 622 by performing a sparse readout based on a selection 625 of the sixth group 530 without reading out any of the other groups (505, 510, 520, 525, 535, 540, 550, 560, 565, 570, 575, 580) and by combining all of the pixels belonging to the sixth group 530 into the first image 620 at the second resolution 622.

[0082]

[0102] All four pixels belonging to the sixteenth group 580 (one per block 603A) are highlighted in Fig. 6A using a dashed black outline. The imaging system captures the second image 630 at the second resolution 622 by combining all of the pixels belonging to the sixteenth group 580 into the second image 630 at the second resolution 622 by performing a sparse readout based on a selection 635 of the sixteenth group 580 without reading out any of the other groups (505, 510, 520, 525, 530, 535, 540, 550, 560, 565, 570, 575). In some examples, the readout of the first group 505 to capture the image 610 at the first resolution 612 may be referred to as a fine readout in contrast to the sparse readout of the sixth group 530 to generate the first image 620 at the second resolution 622 and / or the sparse readout of the sixteenth group 580 to generate the second image 630 at the second resolution 622.

[0083]

[0103] In some examples, the first resolution 612 is a higher resolution than the second resolution 622. The terms dense readout and / or sparse readout may be relative. For example, the term dense readout may refer to any readout that is denser than a specified sparse readout. The term sparse readout may refer to any readout that is less dense than a specified dense readout and / or full readout. In some examples, the readout of the first group 505 to capture the image 610 at the first resolution 612 may be referred to as a medium density readout.

[0084]

[0104] A third resolution image (not shown) may be captured, for example, by selecting and reading out pixel data from a plurality of groups. In one illustrative example, a third resolution image (not shown) may be captured by selecting and reading out pixel data from all of the groups (e.g., 505, 510, 520, 525, 530, 535, 540, 550, 560, 565, 570, 575, and 580). The reading out of a third resolution image may be referred to as a high density readout. The reading out of a third resolution image may be referred to as a full density readout when the third resolution readout includes selecting and reading out pixel data from all of the groups. In this manner, an imaging system with image sensor 601A may provide multiple levels of readout density. Multiple levels of readout density may include up to a full resolution readout (e.g., 16 pixels per block for block 603A in FIG. 6A), a sparsest readout (e.g., 1 pixel per block), a next readout with additional pixel data (e.g., 2 pixels per block), a next readout with additional pixel data (e.g., 3 pixels per block), a next readout with additional pixel data (e.g., 4 pixels per block as in group 505), a next readout with additional pixel data (e.g., 5 pixels per block), and so forth.

[0085]

[0105] In some examples, different groups may have different numbers of pixels in each of the blocks 603A. For example, one group may have one pixel per block, a second group may have two pixels per block, a third group may have three pixels per block, a fourth group may have four pixels per block, and a fifth group may again have one pixel per block. The pixels may have any arrangement within the blocks. In some examples, the pixels may be arranged in the same way (e.g., the same arrangement of groupings) in each of the blocks of the grid. In some examples, the pixels may be arranged in different ways (e.g., different arrangements of groupings) in different blocks of the grid.

[0086]

[0106] 6B is a conceptual diagram 600B illustrating capture of an image 610 at a first resolution 612 and / or a first image 620 at a second resolution 622 by performing a sparse readout of an image sensor 601B with a pixel array organized into a grid 602B of four blocks 603B, where each of the pixels of the pixel array is organized into one of six groups, with each block containing nine pixels with at least one pixel from each group. The pixel array of the image sensor 601B is six pixels wide by six pixels high. Each of the blocks 603B is three pixels wide by three pixels high. Thus, the pixel array of the image sensor 601B is two blocks wide by two blocks high.

[0087]

[0107] Despite the smaller size of the blocks 603B compared to the blocks 603A, the image sensor 601B of FIG. 6B is still able to capture the image 610 in the first resolution 612 and / or the first image 620 in the second resolution 622, as the image sensor 601A of FIG. 6A is able to do. This is because the blocks 603B still include four pixels of the first group 505 and only one pixel of the sixth group 530. In total, each of the blocks 603B includes four pixels of the first group 505, one pixel in the second group 510, one pixel in the fifth group 525, one pixel in the sixth group 530, one pixel in the seventh group 535, and one pixel in the tenth group 550. In some examples, the first resolution 612 is a higher resolution than the second resolution 622.

[0088]

[0108] It should be understood that Figures 5A-5C and 6A-6B represent non-limiting examples of grid, block, and pixel grouping structures, patterns, arrangements, and / or layouts. The image sensor of the imaging system may be arranged in a grid having more blocks, fewer blocks, or an equal number of blocks compared to any of the grids of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid having blocks having more pixels, fewer pixels, or an equal number of pixels compared to any of the blocks of the grids of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid having blocks having more photodiodes, fewer photodiodes, or an equal number of photodiodes compared to any of the blocks of the grids of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may have a pixel array having more pixels, fewer pixels, or an equal number of pixels compared to any of the pixel arrays of the image sensors of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid with blocks that are larger, smaller, or equal in size compared to any of the blocks in the grids of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid with blocks that include more, fewer, or an equal number of pixel groups compared to any of the blocks in the grids of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged on a grid with blocks that include more pixels in the pixel groups, fewer pixels in the pixel groups, or an equal number of pixels in the pixel groups compared to any of the pixel groups of Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid with one or more blocks that are square, such as the blocks in Figures 5A-5C and 6A-6B. The image sensor of the imaging system may be arranged in a grid with one or more blocks that are rectangular. The image sensor of the imaging system may be arranged in a grid with one or more blocks that are rectangular, with one side of the block longer (in terms of number of pixels) than the other side of the block.The image sensor of the imaging system may be arranged in a grid with blocks having pixels arranged in any pattern, arrangement, or layout of pixel grouping. In some examples, each block has its pixels arranged in the same pixel grouping pattern, such as the blocks of FIGS. 5A-5C and 6A-6B. In some examples, different blocks have pixels arranged in different pixel grouping patterns. Different patterns of pixel grouping may be repeated horizontally and / or vertically across the image sensor according to one or more grids, such as grid 502, grid 602A, or grid 602B. For example, a particular pixel grouping pattern may be used for every other block in the layout. Different pixel groups may be arranged across the image sensor according to different patterns. The different patterns for different pixel groupings may be arranged according to the same grid or different grids. In some examples, the grids, blocks, and pixel grouping structures, patterns, arrangements, and / or layouts of FIGS. 5A-5C and 6A-6B may represent examples of imaging systems with a global shutter. In some examples, the grid, block, and pixel grouping structures, patterns, arrangements, and / or layouts of FIGS. 5A-5C and 6A-6B can represent examples of imaging systems with rolling shutters.

[0089]

[0109] In some examples, the pixel arrays of FIGS. 5A-5C and 6A-6B are arrays of image pixels. In some examples, the pixel arrays of FIGS. 5A-5C and 6A-6B include only image pixels. In some examples, the pixel arrays of FIGS. 5A-5C and 6A-6B include image pixels and focus pixels. In some examples, one of the pixel groups of FIGS. 5A-5C and 6A-6B may be a group of image pixels. In some examples, one of the pixel groups of FIGS. 5A-5C and 6A-6B may include only image pixels. In some examples, one of the pixel groups of FIGS. 5A-5C and 6A-6B may include image pixels and focus pixels. In some examples, one of the pixel groups of FIGS. 5A-5C and 6A-6B may include focus pixels.

[0090]

[0110] 7 is a conceptual diagram 700 illustrating the capture of a first image 725 in a first resolution 727, a second image 735 in a first resolution 727, an image 755 in a second resolution 757, and a third image 745 in a first resolution 727. The capture of the first image 725 in the first resolution 727, the second image 735 in the first resolution 727, the image 755 in the second resolution 757, and the third image 745 in the first resolution 727 are shown relative to a horizontal time axis 790, with time running from left to right. The time axis 790 is sometimes referred to as a timeline.

[0091]

[0111] Three time durations are identified along the time axis 790: a first time 705, a second time 710 after the first time 705, and a third time 715 after the second time 710. At the beginning of the first time 705, all pixels of the image sensor of the imaging system are reset, as shown as reset 722 (e.g., a "reset all" function). At the end of the first time 705, the imaging system captures a first image 725 at a first resolution 727 based on a first sparse readout 728 during which the imaging system selectively reads out one or more designated pixel groups without reading out other pixel groups other than the one or more designated pixel groups. In some examples, during the first sparse readout 728, the imaging system reads out all of the image pixels of the one or more designated pixel groups. In some examples, during the first sparse readout 728, the imaging system reads out all of the image pixels of the one or more designated pixel groups and all of the focus pixels. The time between reset 722 and sparse readout 728 to capture a first image 725 at a first resolution 727 is called integration 720. Exposure of the image sensor to light from the scene over the exposure time, and the resulting accumulation of charge on the photodetectors of the image sensor, may occur during integration 720. During integration 720, analog pixel data from all of the pixels and / or photodetectors of the image sensor may be stored in storage array 135, shown in and discussed with reference to Figures 1, 2, and 3A-3B.

[0092]

[0112] At the beginning of the second time 710, all pixels of the image sensor of the imaging system are reset, as shown as reset 732 (e.g., a "reset all" function). At the end of the second time 710, the imaging system captures a second image 735 at the first resolution 727 based on a second sparse readout 738 during which the imaging system selectively reads out one or more designated groups of pixels without reading out other groups of pixels other than the one or more designated groups of pixels. In some examples, the one or more designated groups of pixels for the second sparse readout 738 are the same as for the first sparse readout 728. In some examples, the one or more designated groups of pixels for the second sparse readout 738 are different as compared to the first sparse readout 728. In some examples, the one or more designated groups of pixels for the second sparse readout 738 include at least one of the one or more designated groups of pixels for the first sparse readout 728. In some examples, the one or more designated pixel groups for the second sparse read 738 exclude at least one of the one or more designated pixel groups for the first sparse read 728. In some examples, during the second sparse read 738, the imaging system reads all of the image pixels of the one or more designated pixel groups. In some examples, during the second sparse read 738, the imaging system reads all of the image pixels of the one or more designated pixel groups and all of the focus pixels. The time between the reset 732 and the sparse read 738 to capture the second image 735 at the first resolution 727 is referred to as integration 730. Exposure of the image sensor to light from the scene over an exposure time, and the resulting accumulation of charge in the photodetectors of the image sensor, may occur during integration 730. During integration 730, analog pixel data from all of the pixels and / or photodetectors of the image sensor may be stored in the storage array 135 as shown in and discussed with respect to FIGS. 1, 2, and 3A-3B.

[0093]

[0113] At the beginning of a third time 715, all pixels of the image sensor of the imaging system are reset, as shown as reset 742 (e.g., a "reset all" function). At the end of the third time 715, the imaging system captures a third image 745 at the first resolution 727 based on a third sparse readout 748 during which the imaging system selectively reads out one or more designated groups of pixels without reading out other groups of pixels other than the one or more designated groups of pixels. In some examples, the one or more designated groups of pixels for the third sparse readout 748 are the same as for the first sparse readout 728 and / or the second sparse readout 738. In some examples, the one or more designated groups of pixels for the third sparse readout 748 are different as compared to the first sparse readout 728 and / or the second sparse readout 738. In some examples, the one or more designated pixel groups for the third sparse readout 748 include at least one of the one or more designated pixel groups for the first sparse readout 728 and / or the second sparse readout 738. In some examples, the one or more designated pixel groups for the third sparse readout 748 exclude at least one of the one or more designated pixel groups for the first sparse readout 728 and / or the second sparse readout 738. In some examples, during the third sparse readout 748, the imaging system reads all of the image pixels of the one or more designated pixel groups. In some examples, during the third sparse readout 748, the imaging system reads all of the image pixels of the one or more designated pixel groups and all of the focus pixels. The time between the reset 742 and the sparse readout 748 to capture the third image 745 at the first resolution 727 is referred to as integration 740. Exposure of the image sensor to light from the scene over an exposure time and the resulting accumulation of charge in the photodetectors of the image sensor may occur during integration 740. During integration 740, the analog pixel data from all of the pixels and / or photodetectors of the image sensor may be stored in storage array 135 as shown in and discussed with respect to Figures 1, 2, and 3A-3B.

[0094]

[0114] A request 750 is received between the second time 710 and the third time 715. The request 750 requests capture of an image 755 at a second resolution 757. Because data from all of the pixels and / or photodiodes of the image sensor are still stored in the storage array 135 following the integration 730 and have not yet been reset by reset 742, the imaging system can perform a readout on the remaining pixel data from the remaining pixels and / or photodiodes that have not yet been read out as part of the sparse readout 738, and thus perform a fine readout 758 of the image sensor and capture the image 755 at the second resolution 757. In some examples, the second resolution 757 is higher than the first resolution 727.

[0095]

[0115] In some examples, during the dense readout 758, the imaging system selectively reads out one or more designated groups of pixels without reading out other groups of pixels other than the one or more designated groups of pixels. In some examples, the one or more designated groups of pixels for the dense readout 758 include one or more designated groups of pixels for the second sparse readout 738. In some examples, the one or more designated groups of pixels for the dense readout 758 include one or more other groups of pixels other than the designated groups of pixels for the second sparse readout 738. In some examples, the one or more designated groups of pixels are the same for the dense readout 758 and include at least one of the one or more designated groups of pixels for the first sparse readout 728, and / or for the second sparse readout 738, and / or for the third sparse readout 748. In some examples, the one or more designated pixel groups are the same for the fine readout 758, except for at least one of the one or more designated pixel groups for the first sparse readout 728, and / or for the second sparse readout 738, and / or for the third sparse readout 748. In some examples, the one or more designated pixel groups for the fine readout 758 are different compared to the first sparse readout 728 and / or the second sparse readout 738 and / or the third sparse readout 748. In some examples, the one or more designated pixel groups for the fine readout 758 are the same as the first sparse readout 728 and / or the third sparse readout 748. In some examples, during the fine readout 758, the imaging system reads all of the pixels of the image sensor. In some examples, during the fine readout 758, the imaging system reads all of the pixels of the image sensor. In some examples, during the fine readout 758, the imaging system reads all of the image pixels of the one or more designated pixel groups. In some examples, during a fine readout 758, the imaging system reads all of the image pixels and all of the focus pixels of one or more designated pixel groups. In some examples, during a fine readout 758, the imaging system reads all of the image pixels of the image sensor. In some examples, during a fine readout 758, the imaging system reads all of the image pixels and all of the focus pixels of the image sensor. In some examples, the fine readout 758 may be a full readout of the image sensor.

[0096]

[0116] In some examples, the request 750 may be based on a user interface input received from a user interface. In some examples, the request 750 may be provided by the imaging system, received by the imaging system, or both. For example, the request 750 may be provided and / or received by the imaging system based on the imaging system detecting characteristics of the second image 735 in the first resolution 727 (and / or the first image 725 in the first resolution 727) that suggest an image at the second resolution 757 (or possibly at a higher resolution than the first resolution 727) may be desirable and / or beneficial for a use case. In some examples, the imaging system may provide and receive the request 750 based on the imaging system determining (e.g., using an object detection algorithm) that the second image 735 in the first resolution 727 (and / or the first image 725 in the first resolution 727) includes a depiction of an object. In some examples, the object includes glyphs that optically encode information (e.g., one-dimensional barcodes or two-dimensional (2D) barcodes) that may be easier for the imaging system (e.g., object detection algorithms and / or glyph detection and / or scanning and / or interpretation algorithms) to scan and / or interpret the image at the second resolution 757 (or possibly a resolution higher than the first resolution 727). Examples of 2D barcodes include Quick Response (QR) Codes, Aztec Codes, Data Matrices, PDF417 Codes, MaxiCodes, Codablock F Codes, Han Xin Codes, and DotCodes. In some examples, they may be referred to as 2D barcodes and 2D codes. An example 754 of detection 752 of a glyph (QR Code®) in the second image 735 at the first resolution 727 is shown as an example of an imaging system that triggers the request 750. In the example 754, the glyph is visible within the field of view of the camera of the mobile handset.A first representation of the glyph corresponding to a first resolution 727 is shown on a display of the mobile handset to indicate that the second image 735 at the first resolution 727 may include the first representation of the glyph based on the glyph being within the FOV of the mobile handset at the time of capture of the second image 735 at the first resolution 727. The first representation of the glyph may be too blurry and / or of low resolution for the imaging system to be able to interpret information optically encoded by the glyph from a representation of the glyph (hence, request 750) having a confidence value that exceeds the confidence threshold, but the first resolution 727 of the first representation of the glyph may be sufficient to detect that the glyph is in fact depicted in the second image 735 having a confidence value that exceeds the confidence threshold. The image 755 at the second resolution 757 includes a second representation of the glyph that may be sufficiently detailed, sharp, clear, and / or of high resolution for the imaging system to be able to interpret information encoded by the glyph from the second representation of the glyph having a confidence value that exceeds the confidence threshold. In some examples, the information encoded by the glyphs may include a uniform resource indicator (URI), such as a uniform resource locator (URL). In some examples, the information encoded by the glyphs may include alphanumeric characters.

[0097]

[0117] In some examples, the object includes a glyph, a part of a person (e.g., a human face, a human body), a part of a vehicle (e.g., including a license plate), a part of an animal (e.g., an animal face, an animal body), one or more alphanumeric characters (e.g., arranged in an alphanumeric string), or a combination thereof. The object detection algorithm may include a feature detection algorithm, a feature recognition algorithm, an object detection algorithm, an object recognition algorithm, a face detection algorithm, a face recognition algorithm, a person detection algorithm, a person recognition algorithm, an optical character detection algorithm, an optical character recognition (OCR) algorithm, a classifier, an optical glyph detector, an optical glyph scanner, or a combination thereof. The object detection algorithm may, in some examples, detect more effectively and / or consistently using an image at the second resolution 757 (or possibly a resolution higher than the first resolution 727) than using an image at the first resolution 727.

[0098]

[0118] In some examples, the object is a set of one or more alphanumeric characters that may be easier for an object detection algorithm to analyze using an image at the second resolution 757 (or possibly at a higher resolution than the first resolution 727). The object detection algorithm may include an optical character recognition (OCR) algorithm.

[0099]

[0119] In some examples, the object is an object and / or image region having a saliency above a saliency threshold. The imaging system can determine the saliency of the object and / or image region at least in part by generating a saliency map based on an image at a first resolution 727 (e.g., the first image 725 and / or the second image 735). In some examples, the imaging system can generate a saliency map based on an image at a first resolution 727 (e.g., the first image 725 and / or the second image 735) by inputting these images at the first resolution 727 as input images to a trained machine learning model. The trained machine learning model is trained to output a saliency map based on one or more input images. In some examples, the imaging system may determine the saliency of an object and / or image region based at least in part on determining that the second image 735 at the first resolution 727 includes movement (e.g., exceeding a movement threshold) compared to a previous image (e.g., the first image 725 at the first resolution 727), and in some examples, the imaging system may determine the saliency of an object and / or image region based at least in part on identifying the object or image region as having a unique color or pattern not found elsewhere in the image at the first resolution 727. In some examples, the imaging system may determine the saliency of an object and / or image region based at least in part on a distance (e.g., in color space and / or luminosity space) between color and / or luminosity values ​​corresponding to pixels of the object and / or image region and color and / or luminosity values ​​of one or more other pixels in the image at the first resolution 727. In some examples, the imaging system may identify the saliency of a given pixel in the second image 735 of the first resolution 727 based at least in part on the distance (e.g., in color space and / or luminosity space) between a color value and / or luminosity value corresponding to the given pixel in the image and the color values ​​and / or luminosity values ​​of one or more other pixels (e.g., other than the given pixel) in the image.In some examples, the color and / or brightness values ​​of one or more pixels in the image may include an average of the color and / or brightness values ​​of the pixels in the image, such that the imaging system finds a distance between a color and / or brightness value of a given pixel and an average of the color and / or brightness values ​​of the pixels in the image. In some examples, the color and / or brightness values ​​of one or more pixels in the image may include an average of the color and / or brightness values ​​of the pixels in the image other than the given pixel, such that the imaging system finds a distance between a color and / or brightness value of a given pixel and an average of the color and / or brightness values ​​of the pixels in the image other than the given pixel.

[0100]

[0120] Thus, the imaging device can determine on the fly to switch from a first sparse readout 738 (e.g., sparse and / or low resolution) to a second fine readout 758 (e.g., fine and / or high resolution) before time 715 and thus before reset 742 and / or integration 740 begin. This is a technical improvement over conventional cameras where a change in settings may require image sensor reconfiguration. Conventional image sensor reconfiguration to change settings is generally not applied until a time delay has elapsed after a request for the change has been made (during which subsequent image frames may be captured). The imaging device provides technical improvements such as reduced power consumption, reduced bandwidth usage, reduced heat generation, more effective heat dissipation, and / or reduced computational resource usage. These improvements come about by using sparse readouts for the first sparse readout 728, the second sparse readout 738, and the third sparse readout 748 when low resolution 727 images (e.g., image 725, image 735, and image 745) are sufficient, and using a second readout 758 (e.g., a dense readout or a full readout) at the second resolution 757 when an image at the second resolution 757 (e.g., image 755) is requested via request 750 (e.g., by a user or by the imaging device itself) for an operation that is useful or required.

[0101]

[0121] In some examples, the imaging system may select the same group of pixels for each sparse readout (e.g., first sparse readout 728, second sparse readout 738, and third sparse readout 748) of a series of images (e.g., image 725, image 735, and image 745) at the first resolution 727. For example, in the context of Figures 5A-5C, the imaging system may select the first group 505 for each sparse readout (e.g., first sparse readout 728, second sparse readout 738, and third sparse readout 748) of a series of images (e.g., image 725, image 735, and image 745) at the first resolution 727. In some examples, the imaging system may alternate which group is selected for each sparse readout (e.g., first sparse readout 728, second sparse readout 738, and third sparse readout 748) of a series of images (e.g., image 725, image 735, and image 745) at a first resolution 727. For example, the imaging system may perform a first sparse readout 728 based on the first group 505 (as in FIG. 5B) for a first image 725 at a first resolution 727, a second readout 738 based on the second group 510 (as in FIG. 5C) for a second image 735 at a first resolution 727, a third sparse readout 748 based on the third group 515 for a third image 735 at a first resolution 727, and so on. The imaging system may cycle through the groups, eventually performing sparse readouts on all groups, and then start the routine over with the first group. The imaging system can store the images thus generated. By performing sparse readout on all groups during this routine, the imaging system can store images captured using all groups. The imaging system can combine the images into a combined image having a higher resolution than the images that are combined to form it by arranging pixel data from the different images based on the layout of the pixel groups in the blocks of the image sensor. This combined image can have a higher resolution than the images that are combined to form it. In this way, the imaging system can gradually gather a high-resolution (or full-resolution) view of the scene by combining multiple images captured at low resolution that are generated via sparse readout.This combined image may still accurately depict the scene, at least for those portions of the scene that remain stationary during the capture of the multiple images.

[0102]

[0122] In some examples, the imaging device may attempt to use an object detection algorithm to perform object detection on other images of the first resolution 727 other than the second image 735 of the first resolution 727, such as the first image 725 of the first resolution 727 and / or the third image 745 of the first resolution 727. In these attempts to perform object detection, a request for an image of the second resolution 757 may not occur (as in request 750) because the attempt to perform object detection resulted in the object detection algorithm not detecting any of the objects that the object detection algorithm attempts to detect within a threshold time, such as before the next reset (e.g., reset 732, reset 742). In some examples, the imaging system may automatically perform a reset (e.g., reset 732, reset 742) in response to the object detection algorithm not detecting any of the objects that the object detection algorithm attempts to detect within a threshold time.

[0103]

[0123] In some examples, the second resolution 757 is higher than the first resolution 727. In some examples, the imaging system may delay the reset 742 for a period of time (e.g., in response to the request 750 and / or the detection 752) to give the imaging system sufficient time to perform the fine readout 758, for example, if the imaging system determines that the fine readout 758 will possibly be interrupted by the reset 742. The period of the delay may be a predetermined period of time. The period of the delay may be based on an amount of time to perform the fine readout 758, may be based on a scheduled time for the reset 742, and / or may be based on an additional predetermined buffer period. In some examples, the second resolution 757 is a full resolution capable of the image sensor of FIG. 7. In some examples, the first resolution 727 corresponds to a first set of one or more pixel groups per block, and the second resolution 757 corresponds to a second set of one or more pixel groups per block. In an illustrative example, the first resolution 727 corresponds to group 530 of Figures 6A-6B and the second resolution 757 corresponds to group 505 of Figures 6A-6B. In some examples, the dense readout 758 at the second resolution 757 may have any of the multiple levels of readout density discussed with respect to Figures 6A-6B that exceed the sparsest readout density level. For example, a medium density readout, such as the readout of the first image 610 of Figures 6A-6B based on the selection 615 of the first group 505, may be sufficient to read glyphs, or for face recognition, or for object recognition, or for OCR, or for another function that the imaging device performs using the image 755.

[0104]

[0124] In some examples, a request such as request 750 may be used to trigger a second camera and / or a second image sensor other than the image sensor capturing image 725, image 735, and / or image 745. For example, image 755 may be captured using a second camera and / or a second image sensor. In some examples, the second camera and / or the second image sensor may have a different resolution than the camera and / or the image sensor used to capture image 725, image 735, and / or image 745. In some examples, the second camera and / or the second image sensor may have a higher resolution than the camera and / or the image sensor used to capture image 725, image 735, and / or image 745. In some examples, image data (e.g., images 725, 735, and / or images 745) from the camera and / or image sensor used to capture images 725, 735, and / or 745 may be used to set image capture settings (e.g., exposure settings, focus settings, white balance settings, zoom settings, flash settings, color balance settings, tone settings, saturation settings, gain settings, image processing settings, etc.) for the second camera and / or second image sensor. This may be used, for example, to speed up exposure convergence of the second camera and / or second image sensor.

[0105]

[0125] It should be understood that references to pixels herein may refer to photodiodes. It should be understood that references to groups of pixels herein may refer to groups of photodiodes.

[0106]

[0126] 8 is a conceptual diagram 800 illustrating movement of a reset pointer 840 and a read pointer 845 along an image sensor 830 in a rolling shutter imaging system. The movement of the reset pointer 840 and the read pointer 845 along the image sensor 830 is shown relative to a horizontal time axis 890, with time flowing from left to right. The time axis 890 is sometimes referred to as a timeline.

[0107]

[0127] Both the reset pointer 840 and the readout pointer 845 move downward along the image sensor 830 from the top of the image sensor 830 to the bottom of the image sensor 830. Both the reset pointer 840 and the readout pointer 845 may move downward along the image sensor 830 at the same rate. Thus, the reset pointer 840 and the readout pointer 845 may maintain a gap of time between them, called the integration time 835. Exposure of the image sensor 830 to light from the scene over the exposure time, and the resulting accumulation of charge in the photodetectors of the image sensor 830, may occur during the integration time 835.

[0108]

[0128] Four times are identified along the time axis 890: a first time 805, a second time 810 after the first time 805, a third time 815 after the second time 810, and a fourth time 820 after the third time 815. At the first time 805, a reset pointer 840, shown as a thick horizontal dashed black line, is near the top of the image sensor 830 and is resetting rows near the top of the image sensor 830. At the second time 810, the reset pointer 840 moves down toward the center of the image sensor 830 and is resetting rows near the center of the image sensor 830. A readout pointer 845, shown as a thick horizontal solid black line, is near the top of the image sensor 830 and is reading out pixel data from rows near the top of the image sensor 830. The spacing between the reset pointer 840 and the readout pointer 845 along the image sensor 830 represents the distance that can be traversed during the integration time 835 at the rate that the reset pointer 840 and the readout pointer 845 are moving. At a third time 815, the reset pointer 840 moves down towards the bottom of the image sensor 830, resetting rows near the bottom of the image sensor 830. The readout pointer 845 is near the center of the image sensor 830, reading out pixel data from rows near the center of the image sensor 830. The distance between the reset pointer 840 and the readout pointer 845 along the image sensor 830 still corresponds to the integration time 835, since the velocities of the reset pointer 840 and the readout pointer 845 are consistent. At a fourth time 820, the reset pointer 840 moves down past the bottom of the image sensor 830, wrapping around to the top of the image sensor 830, resetting rows near the top of the image sensor 830. The readout pointer 845 is near the bottom of the image sensor 830, reading out pixel data from rows near the bottom of the image sensor 830.

[0109]

[0129] When the read pointer 845 reaches the bottom of the image sensor 830 and reads out the last pixel data from the image sensor 830, the image sensor 830 may capture an image based on the readout of pixel data from the image sensor 830. The time axis 890 may then reset back to time 805 and / or time 810 for depiction of the reset, integration, and readout of the next image captured by the image sensor 830. In some examples, the reset pointer 840 scans across the image sensor in a raster scan order. In some examples, the read pointer 845 scans across the image sensor in a raster scan order.

[0110]

[0130] 9 is a conceptual diagram 900 illustrating a rolling shutter imaging system performing a first image capture 920, a second image capture 925, and a third image capture 930. The first image capture 920, the second image capture 925, and the third image capture 930 are shown relative to a horizontal time axis 990, with time flowing from left to right. The time axis 990 is sometimes referred to as a timeline.

[0111]

[0131] Three time durations are identified along the time axis 990: a first time 905, a second time 910 after the first time 905, and a third time 915 after the second time 910. The imaging system performs a first image capture 920 during the first time 905, a second image capture 925 during the second time 910, and a third image capture 925 during the third time 915. The first image capture 920 may cause the imaging system to capture a first image, the second image capture 925 may cause the imaging system to capture a second image, and the third image capture 930 may cause the imaging system to capture a third image. The first image, second image, and third image may be successive image frames of a video.

[0112]

[0132] The first image capture 920, the second image capture 925, and the third image capture 930 are each shown as a parallelogram with the top row further left (earlier along the time axis 990) than the bottom row and the bottom row further right (later along the time axis 990) than the top row. This is because the top row of image sensors is reset, integrated, and / or read out earlier than the bottom row of image sensors. The left end of the parallelogram includes a thick, dashed, black, downward arrow representing the resetting of the image sensor by a reset pointer 940. The reset pointer 940 in FIG. 9 moves from the top of the image sensor to the bottom of the image sensor, similar to the reset pointer 840 in FIG. 8. The left end of the parallelogram includes a thick, dashed, black, downward arrow representing the reading of pixel data from the image sensor by a read pointer 945. The read pointer 945 in FIG. 9 moves from the top of the image sensor to the bottom of the image sensor, similar to the read pointer 845 in FIG. 8. Each parallelogram includes a right-facing arrow representing an integration 935 between a reset on the left edge of the parallelogram (e.g., corresponding to a reset pointer 940) and a readout on the right edge of the parallelogram (e.g., corresponding to a readout pointer 945). Exposure of the image sensor to light from the scene over an exposure time, and the resulting accumulation of charge on the photodetectors of the image sensor, may occur during integration 935. In some examples, the reset pointer 940 scans across the image sensor in a raster scan order. In some examples, the readout pointer 945 scans across the image sensor in a raster scan order.

[0113]

[0133] Figure 10A is a conceptual diagram 1000A illustrating image capture of six image frames 1050A-1050F, each at a first resolution 1096, via a first readout pattern 1055 using a rolling shutter imaging system having four pixel groups. The four pixel groups include a first group 1010, a second group 1020, a third group 1030, and a fourth group 1040. An example of an image sensor 1070 (or a portion of the image sensor 1070) is shown in Figure 10A. The image sensor 1070, or the portion of the image sensor 1070 shown, has dimensions of 6 pixels wide by 6 pixels high.

[0114]

[0134] The pixels of the image sensor 1070 are arranged according to the grid 1072 of blocks 1074, similar to the grid 502 of blocks 504 of FIGS. 5A-5C, the grid 602A of blocks 603A of FIG. 6A, and / or the grid 602B of blocks 603B of FIG. 6B. Each of the blocks 1074 includes one pixel from each of the four pixel groups. Thus, each of the blocks 1074 includes a pixel from the first group 1010, a pixel from the second group 1020, a pixel from the third group 1030, and a pixel from the fourth group 1040. The pixels from each of the four groups are arranged in the same order, pattern, and / or arrangement within each of the blocks 1074. An exemplary block 1073 of the blocks 1074 is outlined by a dashed rounded rectangle. For example, the top left pixel in the exemplary block 1073 is in a first group 1010, the top right pixel in the exemplary block 1073 is in a second group 1020, the bottom left pixel in the exemplary block 1073 is in a third group 1030, and the bottom right pixel in the exemplary block 1073 is in a fourth group 1040, and the four pixel groups are arranged according to this order, pattern, and / or arrangement within each of the blocks 1074.

[0115]

[0135] In some examples, a block 1074 may have more than one pixel in a particular group. Examples of blocks having more than one pixel in a particular group include blocks 603A and 603B of Figures 6A-6B, each having four pixels in the first group 505. In some examples, one of the blocks 1074 may have a different order, pattern, and / or arrangement of pixel groups compared to another one of the blocks 1074.

[0116]

[0136] The first readout pattern 1055 is shown with respect to a horizontal time axis 1090, with time flowing from left to right. The time axis 1090 may be referred to as a timeline. Six image frames are captured sequentially according to the first readout pattern 1055, with each image frame captured at a first resolution 1060. The six image frames include, from front to back along the time axis 1090, image frame 1050A, image frame 1050B, image frame 1050C, image frame 1050D, image frame 1050E, and image frame 1050F. Each image frame is shown in Figures 10A-10C as a column of parallelograms along the time axis 1090 including a first group 1010, a second group 1020, a third group 1030, and a fourth group 1040. Each parallelogram represents all of the pixels across the image sensor 1070 that belong to the group to which the parallelogram is labeled. Specifically, the parallelogram labeled first group 1010 represents all of the pixels across the image sensor 1070 in the first group 1010, the parallelogram labeled second group 1020 represents all of the pixels across the image sensor 1070 in the second group 1020, the parallelogram labeled third group 1030 represents all of the pixels across the image sensor 1070 in the third group 1030, and the parallelogram labeled fourth group 1040 represents all of the pixels across the image sensor 1070 in the fourth group 1040. The pixel data within each group may be read out from top to bottom and left to right according to a raster scan order.

[0117]

[0137] According to the first readout pattern 1055, all pixels across the image sensor 1070 are reset, exposed, integrated, and read out for all six image frames. For all six image frames, the pixels in the first group 1010, the pixels in the second group 1020, the pixels in the third group 1030, and the pixels in the fourth group 1040 are all reset, exposed, integrated, and read out. In Figures 10A-10C, the parallelograms along the time axis 1090 all include a thick black dashed line along their left side, which represents the image sensor 1070 reset of pixels in the group corresponding to the group label on the parallelogram. In Figure 10A, all of the parallelograms along the time axis 1090 include a thick black solid line along their right side, which represents the readout of pixel data from pixels in the group corresponding to the group label on the parallelogram. The area of ​​the parallelogram itself represents the exposure and / or integration between reset and readout.

[0118]

[0138] 10B is a conceptual diagram 1000B illustrating image capture of eight image frames 1052A-1052H, each at a second resolution 1062, via a second readout pattern 1057 using a rolling shutter imaging system having four pixel groups. The eight image frames include, from front to back along a time axis 1090, image frame 1052A, image frame 1052B, image frame 1052C, image frame 1052D, image frame 1052E, image frame 1052F, image frame 1052G, and image frame 1052H. As in FIG. 10A, each image frame is shown as a column of parallelograms along the time axis 1090, including a first group 1010, a second group 1020, a third group 1030, and a fourth group 1040. However, in Figure 10B, some of the parallelograms along the time axis 1090 lack a thick solid black line along their right side, indicating that those groups of pixels are not read out for that image frame according to the second readout pattern 1057. To help highlight those groups that are not read out according to the second readout pattern 1057, their parallelograms are also shaded with a half-tone shading pattern.

[0119]

[0139] The second readout pattern 1057 reads a different group of pixels in each image frame until all four groups of pixels have been read out, and then repeats the pattern of capture of different groups of pixels for the next round of image frames. In Figure 10B, the second readout pattern 1057 captures a first group 1010 in image frame 1052A, a second group 1020 in image frame 1052B, a third group 1030 in image frame 1052C, and a fourth group 1040 in image frame 1052D. The diagonal group readout pattern established in image frames 1052A-1052D is repeated in image frames 1052E-1052H.

[0120]

[0140] According to the second readout pattern 1057, only the first group of pixels 1010 is read out for image frame 1052A and image frame 1052E. The second group of pixels 1020, the third group of pixels 1030, and the fourth group of pixels 1040 are not read out for image frame 1052A and image frame 1052E. According to the second readout pattern 1057, only the second group of pixels 1020 is read out for image frame 1052B and image frame 1052F. The first group of pixels 1010, the third group of pixels 1030, and the fourth group of pixels 1040 are not read out for image frame 1052B and image frame 1052F. According to the second readout pattern 1057, only the third group of pixels 1030 is read out for image frame 1052C and image frame 1052G. For image frames 1052C and 1052G, the first group of pixels 1010, the second group of pixels 1020, and the fourth group of pixels 1040 are not read out. For image frames 1052D and 1052H, only the fourth group of pixels 1040 is read out according to a second read pattern 1057. For image frames 1052D and 1052H, the first group of pixels 1010, the second group of pixels 1020, and the third group of pixels 1030 are not read out.

[0121]

[0141] Because only one group of pixels is read out for each image frame, the image data captured for each individual image frame is read out and captured at a second resolution 1062. The second resolution 1062 is different from and lower than the first resolution 1060 of image frames 1050A-1050F of FIG. 10A in which all four groups are read out. Because readout is skipped for certain groups of pixels in certain image frames, capturing image frames with the second readout pattern 1057 of FIG. 10B uses less power, uses less bandwidth (e.g., to convey image data), uses less computational resources (e.g., for image processing), and generates less heat than capturing image frames with the first readout pattern 1055 of FIG. 10A.

[0122]

[0142] In some examples, an imaging system including image sensor 1070 can combine pixel data from different groups read out for different image frames into a single combined image having a resolution higher than the second resolution 1062. For example, an imaging system including image sensor 1070 can combine pixel data from the first group 1010 from image frame 1052A, the second group 1020 from image frame 1052B, the third group 1030 from image frame 1052C, and / or the fourth group 1040 from image frame 1052D into a single combined image. Similarly, the imaging system can combine pixel data from the first group 1010 from image frame 1052E, the second group 1020 from image frame 1052F, the third group 1030 from image frame 1052G, and / or the fourth group 1040 from image frame 1052H into a single combined image. When the imaging system combines the pixel data from all four groups into a combined image, the combined image has the first resolution 1060. In this way, the imaging system can still obtain high quality and high resolution images while still maintaining advantages such as using less power, using less bandwidth, using less computational resources, and generating less heat.

[0123]

[0143] This approach to generating a combined image works particularly well when the imaging system is capturing images of a scene that includes static regions. Because the static regions of the scene do not change significantly as time moves along the time axis 1090, they will appear similar while image frame 1052A is captured, while image frame 1052B is captured, while image frame 1052C is captured, while image frame 1052D is captured, while image frame 1052E is captured, and / or while image frame 1052B is captured. Thus, for static regions of the scene, the combined image may appear indistinguishable from one of image frames 1050A-1050F of FIG. 10A in which all of the pixels are read out.

[0124]

[0144] Some visual artifacts may result from the technique for generating a combined image when the imaging system is capturing images of a scene that includes a dynamic region. The visual artifacts may be caused by the combination of different pixel groups that are read out at different times associated with the capture of different image frames. The visual artifacts may be examples of, or may appear similar to, visual artifacts associated with interlaced video, such as combing, ghosting, interline tweeter, moiré, aliasing effects, or combinations thereof. In some examples, an imaging system that includes an image sensor 1070 may reduce or eliminate visual artifacts by applying one or more de-interlacing algorithms to the combined image, to other combined images adjacent to the combined image along the time axis 1090, and / or to pixel data associated with individual pixel groups used to generate the combined image. The one or more deinterlacing algorithms may include, for example, field blending deinterlacing, field expansion deinterlacing, motion compensated deinterlacing, edge detection, blending, selective blending, weaving, inverse telecine, half-sizing, line doubling, block motion compensation, scene change detection, decimation, video filters, denoising filters, deblocking filters, or combinations thereof.

[0125]

[0145] In some examples, the imaging system may use a third readout pattern (not shown) that may be similar to the second readout pattern 1057, but in a different order, pattern, and / or sequence, and may read out only certain groups without reading out other groups. In one illustrative example, the third readout pattern may capture the fourth group 1040 in image frame 1052A, the third group 1030 in image frame 1052B, the second group 1020 in image frame 1052C, and the fourth group 1040 in image frame 1052D. In some examples, the third readout pattern may capture only a subset of the pixel groups. In another illustrative example, the third readout pattern captures image data and / or pixel data only from one, two, or three of the groups (e.g., selected from the first group 1010, the second group 1020, the third group 1030, and the fourth group 1040). In some examples, the third readout pattern may include some image frames that are skipped entirely such that no image data or pixel data is read out during those frames, hi another illustrative example, the third readout pattern captures image data and / or pixel data from only every other one of the image frames.

[0126]

[0146] 10C is a conceptual diagram 1000C illustrating the transition from capturing image frames at a second resolution 1062 via a second readout pattern 1057 to capturing image frames at a first resolution 1060 via a first readout pattern 1055 using a rolling shutter imaging system having four pixel groups. The image frames include, from front to back along a time axis 1090, image frame 1054A, image frame 1054B, image frame 1054C, image frame 1054D, image frame 1054E, image frame 1054F, image frame 1054G, and image frame 1054H. As in FIGS. 10A-10B, each image frame is shown as a column of parallelograms along the time axis 1090 including a first group 1010, a second group 1020, a third group 1030, and a fourth group 1040.

[0127]

[0147] The first five image frames, including image frame 1054A, image frame 1054B, image frame 1054C, image frame 1054D, and image frame 1054E, are captured at the second resolution 1062 according to the second readout pattern 1057, similar to FIG. 10B. At a time along the time axis 1090 after capture of image frame 1054E, the imaging system with image sensor 1070 receives a request 1085 for image data 1087 at the first resolution 1060. After the request 1085 is received, the imaging device transitions from the second readout pattern 1057 to the first readout pattern 1055. The imaging system captures image frame 1054G and image frame 1054H according to the first readout pattern 1055, similar to FIG. 10A. Because all pixel groups are reset every image frame even if they are not read out, the transition between the second readout pattern 1057 and the first readout pattern 1055 can be instantaneous or near instantaneous since there is no need to delay before the reset can be performed. The reset for every image frame does not add significant delay. In some examples (not shown), the image sensor 1070 of the imaging system can skip resetting certain image frames, for example, to reset every other image frame.

[0128]

[0148] Because the request 1085 is received midway through the capture and / or readout of different pixel groups of the image frame 1054F, the capture of the image frame 1054F includes the readout of more groups than under the second readout pattern 1057 and includes the readout of fewer groups than under the first readout pattern 1055. In an illustrative example (not shown), the request 1085 may be received earlier than shown in FIG. 10C and the image frame 1054F may be fully captured according to the first readout pattern 1055, thus resulting in the readout of the first group 1010. In another illustrative example (not shown), the request 1085 may be received later than shown in FIG. 10C and the image frame 1054F may be fully captured according to the second readout pattern 1057, thus resulting in the readout of the third group 1030 and the fourth group 1040 being skipped.

[0129]

[0149] In some examples (not shown), an imaging system with image sensor 1070 can additionally or alternatively transition from the first readout pattern 1055 to the second readout pattern 1057. In some examples (not shown), an imaging system with image sensor 1070 can additionally or alternatively transition to and / or from one of the third readout patterns discussed with respect to FIG.

[0130]

[0150] In some examples, the request 1085, or a similar request to transition from one readout pattern to another, may be triggered by a manual request received via a user interface. In some examples, the request 1085 may be triggered automatically by detection by an imaging device with an image sensor 1070 of one or more objects in one or more of the image frames 1054A-1054E captured under the second readout pattern 1057 (and / or in a combined image generated based on pixel data from different pixel groups read out from different image frames). The detection may include any type of detection discussed with respect to detection 752 of FIG. 7. The detection may include, for example, detection of a glyph, as in example 754 of detection 752 of FIG. 7. As in FIG. 7, the detection of FIG. 10C may include, for example, detection of an object using an object detection algorithm. The object may include, for example, a glyph, a part of a person (e.g., a human face, a human body), a part of a vehicle (including, for example, a license plate), a part of an animal (e.g., an animal face, an animal body), one or more alphanumeric characters (e.g., arranged in an alphanumeric string), or a combination thereof. The object detection algorithm may include a feature detection algorithm, a feature recognition algorithm, an object detection algorithm, an object recognition algorithm, a face detection algorithm, a face recognition algorithm, a person detection algorithm, a person recognition algorithm, an optical character detection algorithm, an optical character recognition (OCR) algorithm, a classifier, an optical glyph detector, an optical glyph scanner, or a combination thereof.

[0131]

[0151] In some examples, the object is an object and / or image region having a saliency that exceeds a saliency threshold. The imaging system can determine the saliency of the object and / or image region by generating a saliency map based on at least one or more of the image frames 1054A-1054E captured under the second readout pattern 1057 (and / or in a combined image generated based on pixel data from different pixel groups read out from different image frames). In some examples, the imaging system can generate a saliency map based on these images by inputting these images as input images into a trained machine learning model. The trained machine learning model is trained to output a saliency map based on the one or more input images. In some examples, the imaging system can determine the saliency of the object and / or image region based at least in part on determining that the image includes motion (e.g., exceeding a motion threshold) as compared to a previous image of one or more of the image frames 1054A-1054E. In some examples, the imaging system can determine the saliency of the object and / or image region based at least in part on identifying the object or image region as having a unique color or pattern not found elsewhere in the image. In some examples, the imaging system may determine the saliency of an object and / or image region based at least in part on a distance (e.g., in color space and / or luminosity space) between a color value and / or luminosity value corresponding to the pixel of the object and / or image region and the color value and / or luminosity value of one or more other pixels in the image. In some examples, the imaging system may identify the saliency of a given pixel in the image based at least in part on a distance (e.g., in color space and / or luminosity space) between a color value and / or luminosity value corresponding to the given pixel in the image and the color value and / or luminosity value of one or more other pixels (e.g., other than the given pixel) in the image. In some examples, the color value and / or luminosity value of one or more pixels in the image may include an average of the color values ​​and / or luminosity values ​​of the pixels in the image, such that the imaging system finds a distance between the color value and / or luminosity value of the given pixel and an average of the color values ​​and / or luminosity values ​​of the pixels in the image.In some examples, the color and / or brightness value of one or more pixels in the image may include an average of the color and / or brightness values ​​of pixels in the image other than the given pixel, such that the imaging system finds the distance between the color and / or brightness value of the given pixel and the average of the color and / or brightness values ​​of pixels in the image other than the given pixel.

[0132]

[0152] FIG. 11 is a conceptual diagram illustrating a rolling shutter readout pattern 1100 using a rolling shutter imaging system. The rolling shutter readout pattern 1100 is shown relative to a horizontal time axis 1190, where time flows from left to right, and relative to a vertical row axis 1192, which shows different rows of the image sensor. The time axis 1190 is sometimes referred to as a timeline. Different rows of the image sensor are shown along different horizontal lines that are perpendicular to the vertical row axis 1192 and intersect the vertical row axis 1192 at different times. The rolling shutter readout pattern 1100 includes a reset 1112 of the group 1010 followed by a readout 1115 of the group 1010, with an integration 1110 of the group 1010 between them. The rolling shutter readout pattern 1100 includes a reset 1122 of the group 1020 followed by a readout 1125 of the group 1020, with an integration 1120 of the group 1020 between them. The rolling shutter readout pattern 1100 includes a reset 1132 of a group 1030 followed by a readout 1135 of the group 1030, with an integration 1130 of the group 1030 between them. The rolling shutter readout pattern 1100 includes a reset 1142 of a group 1040 followed by a readout 1145 of the group 1040, with an integration 1140 of the group 1040 between them. Different groups can be reset at different times. There is a time overlap between the integrations of adjacent groups.

[0133]

[0153] 12 is a flow diagram illustrating operations for imaging. Process 1200 may be performed by an imaging system. In some examples, the imaging system may be, for example, an imaging system including image capture and processing system 100, image capture device 105A, image processing device 105B, image processor 150, ISP 154, host processor 152, an imaging system of FIG. 2 performing at least a subset of process 200, an imaging system including image sensor 310 and / or storage array 320, an imaging system including image sensor 410 and rolling shutter 420, an imaging system including image sensor 501, an imaging system including image sensor 601A, an imaging system with a rolling shutter of FIG. 4A, an imaging system with a global shutter of FIG. 4B, an imaging system with image sensor 501 of FIG. 5A-5C, an imaging system with image sensor 601A of FIG. 6A, The imaging system may include an imaging system having image sensor 601B of FIG. 6B, an imaging system capturing at least a subset of the images of FIG. 7 (e.g., image 725, image 735, image 745, image 755), an imaging system including image sensor 830 of FIG. 8, an imaging system performing at least a subset of the image captures of FIG. 9 (e.g., image capture 920, image capture 925, image capture 930), an imaging system performing readout according to first readout pattern 1055 of FIGS. 10A and 10C, an imaging system performing readout according to second readout pattern 1057 of FIGS. 10B-10C, an imaging system applying rolling shutter readout pattern 1100 of FIG. 11, a computing system 1300, a processor 1310, or combinations thereof.

[0134]

[0154] In operation 1205, the imaging system is configured and capable of exposing a plurality of photodetectors of the image sensor to light from the scene. Each of the plurality of photodetectors is configured to convert light from the scene into an electric charge in response to exposure to light from the scene. The plurality of photodetectors includes at least a first group of photodetectors and a second group of photodetectors that are distinct from one another. In some examples, the imaging system includes an image sensor. Examples of image sensors include image sensor 130, an image sensor including photodetector 215, image sensor 310, an image sensor of the imaging system of FIG. 4A, an image sensor of the imaging system of FIG. 4B, image sensor 501, image sensor 601A, image sensor 601B, an image sensor of the imaging system of FIG. 7, image sensor 830, an image sensor of the imaging system of FIG. 9, an image sensor of any of the imaging systems of FIGS. 10A-10C, an image sensor of the imaging system of FIG. 11, another image sensor, or a combination thereof. Examples of photodetectors include photodetector 215, photodetectors P0-P63 of image sensor 310, photodetectors corresponding to pixels of image sensor 501, photodetectors corresponding to pixels of image sensor 601A, photodetectors corresponding to pixels of image sensor 601B, photodetectors corresponding to pixels of image sensor 830, photodetectors corresponding to pixels of any of the image sensors of Figures 10A-10C, or combinations thereof. An example of a scene is scene 110. 5A-5C and 6A-6B include pixels in the first group 505, the second group 510, the third group 515, the fourth group 520, the fifth group 525, the sixth group 530, the seventh group 535, the eighth group 540, the ninth group 545, the tenth group 550, the eleventh group 555, the twelfth group 560, the thirteenth group 565, the fourteenth group 570, the fifteenth group 575, and the sixteenth group 580. Examples of separate photodetector groups in the context of FIGS. 10A-10C include the first group 1010, the second group 1020, the third group 1030, and the fourth group 1040. Exposing a photodetector of the image sensor to light from the scene may include actuating one or more actuators and / or motors to open a shutter associated with the image sensor and / or open an aperture associated with the image sensor.Following exposing the image sensor's photodetector to light from the scene, the image sensor may be caused to terminate or terminate the exposure of the image sensor's photodetector to light from the scene, which may include actuating one or more actuators and / or motors to close a shutter associated with the image sensor and / or close an aperture associated with the image sensor.

[0135]

[0155] In some aspects, the plurality of photodetectors includes a plurality of photodiodes. In some aspects, the plurality of photodetectors of the image sensor includes a plurality of image photodetectors (e.g., image photodiodes) and a plurality of focus photodetectors (e.g., focus photodiodes). The plurality of focus photodetectors are configured for phase detection autofocus (PDAF). The first photodetector group includes a first subset of the plurality of image photodiodes. The first photodetector group may include a first subset of the plurality of focus photodetectors. The second photodetector group includes a second subset of the plurality of image photodiodes. The second photodetector group may include a second subset of the plurality of focus photodetectors.

[0136]

[0156] In operation 1210, the imaging system is configured to and can store a plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors. In some aspects, the image sensor includes a storage array with a plurality of storage units. To store each of the plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors, the imaging system (e.g., the image sensor) is configured to and can store each of the plurality of analog photodetector signals in one of the plurality of storage units of the storage array. Examples of storage arrays include storage array 265 and storage array 320. Examples of storage units include storage units S0-S63 of storage unit 260 and storage array 320.

[0137]

[0157] In some embodiments, an analog photodetector signal of the plurality of analog photodetector signals indicates a voltage corresponding to a charge from one of the plurality of photodetectors. An example of an analog photodetector signal is analog photodetector signal 250.

[0138]

[0158] In some aspects, the imaging system (e.g., an image sensor) is configured to and capable of resetting each of the photodetectors of the image sensor and / or the storage units of the storage array prior to exposing the photodetectors to light from the scene in operation 1205. Examples of resetting the photodetectors and / or the storage units include reset block 404, reset block 414, reset block 424, reset block 434, reset block 444, reset block 454, reset block 464, reset block 474, reset 722, reset 732, reset 742, reset using reset pointer 840, reset using reset pointer 940, reset using any of the resets (indicated by dashed lines) in Figures 10A-10C, reset 1112, reset 1122, reset 1132, reset 1142, or combinations thereof.

[0139]

[0159] In operation 1215, the imaging system is configured to and capable of reading first digital pixel data from a first subset of the analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the analog photodetector signals corresponding to the second photodetector group. An example of the first digital pixel data includes digital pixel data 255. Reading the first digital pixel data may be referred to as reading the first digital pixel data. Examples of reading or reading the first digital pixel data include reading from the image capture device 105A by the image processing device 105A, reading the digital pixel data 255, reading from the image sensor 310, reading from the storage array 320, read block 408, read block 418, read block 428, read block 438, read block 448, read block 458, read block 468, read block 478, sparse readout, and the like. 6A, the two reads of FIG. 6B, a first sparse read 728, a second sparse read 738, a third sparse read 748, a dense read 758, a read with read pointer 845, a read with read pointer 945, a read with first read pattern 1055, a read with second read pattern 1057, a read 1115, a read 1125, a read 1135, a read 1145, or a combination thereof.

[0140]

[0160] In some examples, the imaging system may include a connector coupled to the image sensor. Reading the first digital pixel data may include receiving the first digital pixel data using the connector. The connector may include a port, a jack, a wire, an input / output (I / O) pin, a conductive trace on a printed circuit board (PCB), any other type of connector discussed herein, or any combination thereof. In some examples, the imaging system may include an image sensor. In some examples, causing the image sensor to expose a plurality of photodetectors of the image sensor to light from the scene, as in Operation 1205, may include transmitting a signal from one or more processors of the imaging system to the image sensor via the connector. In some examples, causing the image sensor to store a plurality of analog photodetector signals corresponding to charges from each of the plurality of photodetectors, as in Operation 1210, may include transmitting a signal from one or more processors of the imaging system to the image sensor via the connector. In some examples, as in operation 1215, reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors includes receiving and / or reading the first digital pixel data and / or the first subset of the plurality of analog photodetector signals (and / or a processed version thereof) via a connector.

[0141]

[0161] In some aspects, the imaging system is configured and can use imaging circuitry to selectively apply a modification to a first subset of the analog photodetector signals corresponding to a first group of photodetectors without applying the modification to a second subset of the analog photodetector signals to read a first digital pixel data from a first subset of the analog photodetector signals corresponding to a second group of photodetectors. The modification may include, for example, amplification of the analog photodetector signals via one or more analog gain amplifiers, such as amplifier 230. The modification may include filtering the analog photodetector signals via one or more filters, such as, for example, a high-pass filter, a low-pass filter, a band-pass filter, or a combination thereof. Examples of modification using such filters are discussed with respect to FIG. 2. The modification may include conversion of the analog photodetector signals to digital pixel data, for example, via one or more ADCs, such as analog-to-digital converter (ADC) 235 and / or ADC 132. The modifications may include processing operations performed by ISP 154, such as, for example, demosaicing, digital gain, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, upsampling, additional image processing operations, or combinations thereof.

[0142]

[0162] In some aspects, the imaging system (e.g., an image sensor and / or imaging circuitry associated with the image sensor) includes one or more amplifiers. An example of the one or more amplifiers includes amplifier 230. To read the first digital pixel data from the first subset of the analog photodetector signals, the imaging system is configured to and can cause one or more amplifiers to amplify each of the first subset of the analog photodetector signals corresponding to the first photodetector group into a plurality of amplified analog photodetector signals. An example of the amplified analog photodetector signal includes analog photodetector signal 250 once amplified by amplifier 230. In some examples, the first digital pixel data is based on the plurality of amplified analog photodetector signals. An example of the digital pixel data here includes digital pixel data 255 based on photodetector signal 250 amplified by amplifier 230.

[0143]

[0163] In some aspects, the imaging system (e.g., image sensor) includes one or more analog-to-digital converters (ADCs). Examples of the one or more ADCs include ADC 132 and ADC 235. To read the first digital pixel data from the first subset of the analog photodetector signals, the imaging system is configured and can cause the one or more ADCs to convert a plurality of analog signals based on the first subset of the analog photodetector signals corresponding to the first photodetector group into digital signals. The first digital pixel data is based on the digital signals. Examples of the digital signals include digital pixel data 255 and / or the output of ADC 235 to image processor 240.

[0144]

[0164] In some aspects, the imaging system is configured and capable of generating the first digital pixel data based on the digital signal at least in part by processing the digital signal using one or more image processing operations. The one or more image processing operations may be performed by an image processor, such as ISP 154, host processor 152, image processor 150, image processor 240, processor 1310, or a combination thereof. An example of generating the first digital pixel data by processing the digital signal using one or more image processing operations performed by an image processor includes generating the digital pixel data 255 by processing the digital signal output by ADC 235 using one or more image processing operations performed by image processor 240. The one or more image processing operations may include digital gain, demosaicing, pixel interpolation, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, upsampling, or a combination thereof. In some aspects, the plurality of analog signals includes a plurality of amplified analog photodetector signals. The one or more amplifiers may generate the multiple amplified analog photodetector signals at least in part by amplifying a first subset of the multiple analog photodetector signals corresponding to the first group of photodetectors. Examples of the one or more amplifiers include amplifier 230.

[0145]

[0165] In operation 1220, the imaging system is configured and capable of generating an image of the scene using at least the first digital pixel data.Examples of images include images captured by image capture and processing system 100, images captured via image capture 270 based on digital pixel data 255, images captured using image sensor 310 and / or storage array 320, images captured via an imaging device with a rolling shutter as in FIG. 4A, images captured via an imaging device with a global shutter as in FIG. 4B, images captured using image sensor 501 (e.g., images captured using groups 505-580 based on digital pixel data from one or more of groups 505-580), images captured using a 3D image sensor 502 (e.g., images captured using groups 505-580 based on digital pixel data from one or more of groups 505-580), images captured using a 3D image sensor 503 (e.g., images captured using groups 505-580 based on digital pixel data from one or more of groups 505-580), images captured using a 3D image sensor 504 (e.g., images captured using groups 505-580 based on digital pixel data from one or more of groups 505-580), images captured using a 3D image sensor 505 (e.g., images captured using groups 505-580 based on digital pixel data from one or more of groups 505-580), images captured using a 3D image sensor 506 (e.g., images captured using groups 506-580 based on digital pixel data from one or more of groups 506-580), images captured using a 3D image sensor 508 (e.g., images captured using groups 506-580 based on digital pixel data from one or more of groups 506-580), images captured using a 3D image sensor 509 (e.g., images captured using groups 508-580 based on digital pixel data from one or more of groups 508-580), images captured using a 3D image sensor 509 (e. 580), an image 585 at a first resolution 587, an image 590 at a first resolution 587, an image 610 at a first resolution 612, a first image 620 at a second resolution 622, a second image 630 at a second resolution 622, an image captured using image sensor 601A (e.g., based on digital pixel data from one or more of groups 505-580, without digital pixel data from a different one or more of groups 505-580), an image captured using image sensor 601B (e.g., based on digital pixel data from one or more of groups 505-580, without digital pixel data from a different one or more of groups 505-580), an image captured using an image sensor 830 based on pixel data read according to a read pointer 845; an image captured using an image capture 920 based on pixel data read according to a read pointer 945; an image captured using an image capture 920 based on pixel data read according to a read pointer 945; an image captured using an image capture 930 based on pixel data read according to a read pointer 945; an image captured using an image capture 940 based on pixel data read according to a read pointer 945; The captured images may include an image captured using image capture 925 based on pixel data output, an image captured using image capture 930 based on pixel data read according to read pointer 945, an image captured according to a first read pattern 1055, an image captured according to a second read pattern 1057, an image captured using image sensor 1070, an image captured according to read 1115, an image captured according to read 1125, an image captured according to read 1135, an image captured according to read 1145, or a combination thereof.

[0146]

[0166] In some aspects, an imaging system (e.g., an image sensor) is configured to and can reset a plurality of photodetectors of the image sensor and / or a plurality of storage units of a storage array after capturing an image. Examples of resetting the photodetectors and / or storage units include reset block 404, reset block 414, reset block 424, reset block 434, reset block 444, reset block 454, reset block 464, reset block 474, reset 722, reset 732, reset 742, reset using reset pointer 840, reset using reset pointer 940, reset using any of the resets (indicated by dashed lines) in Figures 10A-10C, reset 1112, reset 1122, reset 1132, reset 1142, or combinations thereof.

[0147]

[0167] In some aspects, the imaging system is configured and capable of analyzing the image using an object detection algorithm. The object detection algorithm may include a feature detection algorithm, a feature recognition algorithm, an object detection algorithm, an object recognition algorithm, a face detection algorithm, a face recognition algorithm, a person detection algorithm, a person recognition algorithm, an optical character detection algorithm, an optical character recognition (OCR) algorithm, a classifier, an optical glyph detector, an optical glyph scanner, or a combination thereof. The object detection algorithm may be executed by one or more processors of the imaging system. The object detection algorithm may include one or more trained machine learning (ML) systems that may receive as input one or more input images (e.g., including images) and may determine whether the one or more input images depict a particular type of object, such as an optical glyph, a face, a person, at least a portion of a person (e.g., a hand, a face), one or more alphanumeric characters (e.g., arranged in a string), at least a portion of a vehicle, at least a portion of an animal, at least a portion of a plant, at least a portion of a structure (e.g., a building), or a combination thereof. The one or more trained ML systems may include one or more trained ML models, one or more trained neural networks (NN), one or more trained convolutional neural networks (CNN), one or more trained time delay neural networks (TDNN), one or more deep networks, one or more deep belief nets (DBN), one or more recurrent neural networks (RNN), one or more generative adversarial networks (GAN), one or more trained support vector machines (SVM), one or more trained random forests (RF), or a combination thereof.

[0148]

[0168] In some examples, if the object detection algorithm detects an object, the imaging system can read second digital pixel data from a second subset of the analog photodetector signals corresponding to the second group of photodetectors. In some examples, if the object detection algorithm fails to detect any objects of the object type for at least a threshold period, the imaging system can reset the photodetectors of the image sensor and / or the storage units of the storage array. In some examples, the threshold period is a period that ends at a moment when the reset of the photodetectors and / or the storage array is scheduled to be performed by the imaging system. In some examples, the imaging system (e.g., the image sensor) is configured and can automatically reset the photodetectors of the image sensor and the storage units of the storage array in response to the failure to detect one or more objects in the image by the object detection algorithm.

[0149]

[0169] In some aspects, the imaging system is configured and capable of determining whether to read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group after generating the image in ACT 1220. In some aspects, the imaging system is configured and capable of determining whether to read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group based on the image generated in ACT 1220. For example, the decision regarding whether to read the second digital pixel data may be based on analysis of the image generated in ACT 1220 using an object detection algorithm.

[0150]

[0170] In some aspects, the imaging system is configured to and capable of reading first digital pixel data from a first subset of the analog photodetector signals corresponding to the first photodetector group, followed by reading second digital pixel data from a second subset of the analog photodetector signals corresponding to the second photodetector group. An example of the second digital pixel data includes digital pixel data 255. Reading the second digital pixel data may be referred to as reading the second digital pixel data. Examples of reading or reading the second digital pixel data include reading from the image capture device 105A by the image processing device 105A, reading the digital pixel data 255, reading from the image sensor 310, reading from the storage array 320, read block 408, read block 418, read block 428, read block 438, read block 448, read block 458, read block 468, read block 478, sparse readout, and the like. 6A, the two reads of FIG. 6B, a first sparse read 728, a second sparse read 738, a third sparse read 748, a dense read 758, a read with read pointer 845, a read with read pointer 945, a read with first read pattern 1055, a read with second read pattern 1057, a read 1115, a read 1125, a read 1135, a read 1145, or a combination thereof.

[0151]

[0171] In some aspects, the reading of the second digital pixel data by the imaging system is based on the storage of the plurality of analog photodetector signals in operation 1210. An example of the reading of the second digital pixel data by the imaging system based on the storage of the analog photodetector signals may include reading of the digital pixel data 255 based on the stored analog photodetector signals 250 stored in the storage unit 260 of the storage array 265, optionally amplified by the amplifier 230, converted by the ADC 235, and optionally processed by the image processor 240. In some aspects, the imaging system is configured and capable of generating a second image of the scene using at least the first digital pixel data and the second digital pixel data. The second image has a second resolution that is different from the first resolution of the image generated in operation 1220. In some aspects, the second image has a second resolution that is higher (e.g., larger) than the first resolution of the image generated in operation 1220. In some aspects, the imaging system is configured to and capable of reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors after generating an image of the scene in operation 1220.

[0152]

[0172] In some aspects, the imaging system is configured and capable of identifying that the image includes an area having a saliency above a saliency threshold. For example, the imaging system can generate a saliency map based on the image. In some examples, the imaging system can generate the saliency map by using the image as an input into one or more trained machine learning (ML) systems trained to generate a saliency map corresponding to an input image that is input to the one or more trained ML systems. In some examples, the imaging system can generate the saliency map by identifying areas having motion compared to a previous image captured by the image sensor prior to the image. In some examples, the imaging system can generate the saliency map by identifying areas having a unique color or pattern not found elsewhere in the image. In some examples, the imaging system can identify the saliency of a given pixel in the image based at least in part on a distance (e.g., in color space and / or luminosity space) between a color value and / or luminosity value corresponding to the given pixel in the image and a color value and / or luminosity value of one or more other (e.g., other than the given pixel) pixels in the image. In some examples, the color and / or brightness values ​​of one or more pixels in the image may include an average of the color and / or brightness values ​​of the pixels in the image, such that the imaging system finds a distance between the color and / or brightness value of the given pixel and an average of the color and / or brightness values ​​of the pixels in the image. In some examples, the color and / or brightness values ​​of one or more pixels in the image may include an average of the color and / or brightness values ​​of the pixels in the image other than the given pixel, such that the imaging system finds a distance between the color and / or brightness value of the given pixel and an average of the color and / or brightness values ​​of the pixels in the image other than the given pixel. The imaging system may use the saliency map to identify areas having saliency above a saliency threshold. In some aspects, the imaging system is configured to and may read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors in response to identifying that the image includes an area having saliency above a saliency threshold.In some examples, if the imaging system generates a saliency map for an image and identifies that there are no areas in the image with saliency above a saliency threshold, the imaging system may reset the photodiodes and / or the storage array. The identification of areas with saliency above a saliency threshold, or lack thereof, may be a trigger for some of the same actions as object detection using an object detection algorithm.

[0153]

[0173] In some aspects, the imaging system is configured to and can identify that the image exhibits motion relative to a previous image of the scene captured by the image sensor, the motion exceeding a motion threshold. In some examples, the image sensor can capture a sequence of images. For example, the sequence of images can be video frames of a video. In some examples, the previous image is an image in a sequence of images that immediately precedes the image in the sequence of images, with no additional images between the previous image and the image in the sequence of images. In some examples, the previous image is an image in a sequence of images that is two or more images before the image in the sequence of images, with one or more additional images between the previous image and the image in the sequence of images. In the context of FIG. 7, if the image generated in operation 1220 is the third image 745 in the first resolution 727, the previous image can be the first image 725 in the first resolution 727, the second image 735 in the first resolution 727, or the image 755 in the second resolution 757. In the context of FIG. 7, if the image generated in operation 1220 is a second image 735 at a first resolution 727, the previous image may be the first image 725 at a first resolution 727. The motion may include motion of an object, such as motion of a person from a previous image to an image. The motion threshold may indicate a minimum distance (e.g., in pixels) for an object to move between a previous image and the image. The imaging system may be configured to, and may read, second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors in response to identifying that the image indicates motion relative to a previous image of the scene captured by the image sensor, where the motion exceeds the motion threshold.

[0154]

[0174] In some aspects, the imaging system is configured and capable of determining, after generating the second image, whether to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group based on the second image. The plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group. The image is generated in operation 1220 without the third digital pixel data. The second image is also generated without the third digital pixel data.

[0155]

[0175] In some aspects, the imaging system is configured to and can read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group after reading the second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group. Reading the third digital pixel data is based on the storage of the plurality of analog photodetector signals in operation 1210. An example of reading the third digital pixel data by the imaging system based on the storage of the analog photodetector signals may include reading digital pixel data 255 based on the stored analog photodetector signals 250 stored in the storage unit 260 of the storage array 265, optionally amplified by the amplifier 230, converted by the ADC 235, and optionally processed by the image processor 240. The plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group. The image is generated in operation 1220 without the third digital pixel data. The second image is also generated without the third digital pixel data. The imaging system is configured to and can generate a third image of the scene using at least the first digital pixel data, the second digital pixel data, and the third digital pixel data. The third image has a third resolution that is higher (e.g., greater) than the second resolution of the second image and / or the first resolution of the first image generated in OPERATION 1220. In some aspects, the imaging system is configured to and can read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to the third group of photodetectors after generating the image of the scene in OPERATION 1220 and / or after generating the second image.

[0156]

[0176] An example of the third digital pixel data includes digital pixel data 255. Reading the third digital pixel data may be referred to as reading the third digital pixel data. Examples of the first reading or reading of the third pixel data include reading from image capture device 105A by image processing device 105A, reading digital pixel data 255, reading from image sensor 310, reading from storage array 320, read block 408, read block 418, read block 428, read block 438, read block 448, read block 458, read block 468, read block 478, sparse reads, and the like. 586, sparse read 592, the three reads of FIG. 6A, the two reads of FIG. 6B, a first sparse read 728, a second sparse read 738, a third sparse read 748, a dense read 758, a read with read pointer 845, a read with read pointer 945, a read with first read pattern 1055, a read with second read pattern 1057, a read 1115, a read 1125, a read 1135, a read 1145, or a combination thereof.

[0157]

[0177] In some aspects, the imaging system is configured and capable of identifying that the image includes a depiction of an object, for example, using an object detection algorithm. In response to identifying that the image includes a depiction of an object, the imaging system is configured and capable of reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors. In some aspects, the object is a glyph that optically encodes information. The glyph may include at least one one-dimensional barcode, at least one 2D code, or a combination thereof. In some aspects, the object includes at least a portion of a person and / or animal. For example, the object may include a face, a hand, an arm, a finger, a fingerprint, a handprint, a leg, a foot, a footprint, an eye, a head, a gait, a pose, or a combination thereof. In some aspects, the object includes at least a portion of a vehicle. For example, the object may include a shape of a vehicle, a license plate of the vehicle, a collection of headlights of the vehicle, or a combination thereof. In some aspects, the object includes one or more alphanumeric characters. For example, one or more alphanumeric characters may be arranged in a row, such as on a piece of paper, a signature, a card, a display, a projection surface, or a combination thereof.

[0158]

[0178] In some embodiments, the photodetectors of the image sensor are arranged according to a grid of blocks, each block including a distinct subset of the photodetectors. Each block of the grid of blocks includes at least one photodetector of the first group of photodetectors and at least one photodetector of the second group of photodetectors. Examples of grids of blocks include grid 502 of block 504, grid 602A of block 603A, grid 602B of block 603B, grid 1072 of block 1074 of pixels of FIGS. 10A-10C, or combinations thereof.

[0159]

[0179] In some embodiments, the first group of photodetectors is arranged across the image sensor according to a first pattern, and the second group of photodetectors is arranged across the image sensor according to a second pattern. The first pattern and the second pattern correspond to one or more grids. The first pattern may be repeated horizontally and / or vertically according to one or more grids. The second pattern may be repeated horizontally and / or vertically according to one or more grids. Examples of the one or more grids include grid 502, grid 602A, grid 602B, grid 1072 of block 1074 of pixels in FIGS. 10A-10C, or combinations thereof. In some examples, the first pattern and the second pattern both correspond to the same grid. In some examples, the first pattern corresponds to the first grid and the second pattern corresponds to the second grid. The first grid and the second grid are distinct from each other. For example, in the context of Figures 5A-5C, the first lattice may be a lattice 502 of blocks 504, while the second lattice is a lattice of clusters of multiple adjacent blocks 504 (e.g., each cluster is a 2 block by 2 block cluster of four adjacent blocks 504).

[0160]

[0180] In some aspects, the imaging system is configured and capable of moving a reset pointer from a first side of the image sensor to a second side of the image sensor at a predetermined speed across a plurality of reset pointer positions. An example of the movement of the reset pointer across a plurality of reset pointer positions is shown in FIG. 8, where the first side of the image sensor is at the top of the image sensor 830 of FIG. 8 and the second side of the image sensor is at the bottom of the image sensor 830 of FIG. 8. Examples of resetting using a reset pointer also include at least FIGS. 4A, 9, 10A-10C, and 11. To reset each of the plurality of photodetectors, the imaging system is configured and capable of resetting a first photodetector of the plurality of photodetectors in response to the reset pointer reaching one of the plurality of reset pointer positions where the first photodetector of the plurality of photodetectors is located. The readout pointer is moved from a first side of the image sensor to a second side of the image sensor at a predetermined speed across a plurality of readout pointer positions. In some examples, the imaging system is configured and capable of reading out third digital pixel data corresponding to the second photodetector of the first group of photodetectors in response to the read pointer reaching one of a plurality of reset pointer positions in which the second photodetector of the first group of photodetectors is located. The predetermined speed may be associated with a rolling shutter, such as the rolling shutters of the imaging systems of FIGS. 4A, 8, 9, 10A-10C, and 11.

[0161]

[0181] In some aspects, the imaging system is configured and capable of outputting an image. The imaging system may include a display. To output the image, the imaging system is configured and capable of displaying the image using the display. The imaging system may include a communication transceiver. To output the image, the one or more processors are configured and capable of transmitting the image to a receiving device using the communication transceiver. Examples of displays, communication transceivers, and / or other output devices for outputting images include output device(s) 1335, communication interface 1340, devices connected to I / O 156, devices connected to I / O 160, or combinations thereof.

[0162]

[0182] In some aspects, the imaging system includes means for exposing an image sensor to light from the scene, a plurality of photodetectors of the image sensor, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to the light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group that are distinct from one another; means for storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; means for reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and means for generating an image of the scene using at least the first digital pixel data.

[0163]

[0183] The means for exposing the multiple light detectors of the image sensor may include image sensor 130, an image sensor including light detector 215, image sensor 310, an image sensor of the imaging system of Figure 4A, an image sensor of the imaging system of Figure 4B, image sensor 501, image sensor 601A, image sensor 601B, an image sensor of the imaging system of Figure 7, image sensor 830, an image sensor of the imaging system of Figure 9, an image sensor of any of the imaging systems of Figures 10A to 10C, an image sensor of the imaging system of Figure 11, a control mechanism 120, an actuating motor that controls the opening and / or closing of a shutter, an actuating motor that controls the opening and / or closing of an aperture, or a combination thereof. The means for storing a plurality of analog photodetector signals in the image sensor includes memory unit 260, memory array 265, memory array 320, memory units S0-S63, image sensor 130, an image sensor including photodetector 215, image sensor 310, the image sensor of the imaging system of FIG. 4A, the image sensor of the imaging system of FIG. 4B, image sensor 501, image sensor 601A, image sensor 601B, the image sensor of the imaging system of FIG. 7, image sensor 830, the image sensor of the imaging system of FIG. 9, the image sensor of any of the imaging systems of FIGS. 10A-10C, the image sensor of the imaging system of FIG. 11, or a combination thereof. The means for reading the first digital pixel data without reading the second digital pixel data includes amplifier 230, filter 220, ADC 235, storage unit 260, storage array 265, image processor 240, read block 408, read block 418, read block 428, read block 438, read block 448, read block 458, read block 468, read block 478, sparse Includes read 586, sparse read 592, the three reads of FIG. 6A, the two reads of FIG. 6B, the first sparse read 728, the second sparse read 738, the third sparse read 748, the dense read 758, read pointer 845, read pointer 945, a read according to the first read pattern 1055, a read according to the second read pattern 1057, a read 1115, a read 1125, a read 1135, a read 1145, or any combination thereof.The means for generating an image may include image capture and processing system 100, image capture device 105A, image processing device 105B, image processor 150, ISP 154, host processor 152, the imaging system of FIG. 2 implementing at least a subset of process 200, image processor 240, an imaging system including image sensor 310 and / or storage array 320, an imaging system including image sensor 410 and rolling shutter 420, an imaging system including image sensor 501, an imaging system including image sensor 601A, an imaging system with a rolling shutter of FIG. 4A, an imaging system with a global shutter of FIG. 4B, an imaging system with image sensor 501 of FIGS. 5A-5C, an imaging system with image sensor 601 of FIG. 6A, 1A, an imaging system with image sensor 601B of FIG. 6B, an imaging system capturing at least a subset of the images of FIG. 7 (e.g., image 725, image 735, image 745, image 755), an imaging system including image sensor 830 of FIG. 8, an imaging system performing at least a subset of the image captures of FIG. 9 (e.g., image capture 920, image capture 925, image capture 930), an imaging system performing readout according to a first readout pattern 1055, an imaging system performing readout according to a second readout pattern 1057, an imaging system applying the rolling shutter readout pattern 1100 of FIG. 11, a computing system 1300, a processor 1310, or combinations thereof.

[0164]

[0184] In some examples, the processes described herein (e.g., the process of process diagram 7200, the process of FIG. 7, the process of FIG. 8, the process of FIG. 9, the process of FIG. 11, process 1200, and / or other processes described herein) may be performed by a computing device or apparatus. In some examples, the process 200, the process of FIG. 7, the process of FIG. 8, the process of FIG. 9, the process of FIG. 11, and / or the process of FIG. 7 may be performed by the image capture and processing system 100, the imaging system of FIG. 2, the imaging system of FIG. 4A, the imaging system of FIG. 4B, the imaging system of FIG. 7, or a combination thereof. In another example, the process 200, the process of FIG. 7, the process of FIG. 8, the process of FIG. 9, the process of FIG. 11, and / or the process of FIG. 7 may be performed by a computing device including the computing system 1300 shown in FIG. 13.

[0165]

[0185] The computing device may include any suitable device, such as a mobile device (e.g., a mobile phone), a desktop computing device, a tablet computing device, a wearable device (e.g., a VR headset, an AR headset, AR glasses, a network-connected watch or smartwatch, or other wearable device), a server computer, an autonomous vehicle or autonomous vehicle computing device, a robotic device, a television, and / or any other computing device with resource capabilities to perform the processes described herein, including process 200, the process of FIG. 7, the process of FIG. 8, the process of FIG. 9, the process of FIG. 11, and / or the process of FIG. 7. In some cases, the computing device or apparatus may include various components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and / or other components configured to perform steps of the processes described herein. In some examples, the computing device may include a display, a network interface configured to communicate and / or receive data, any combination thereof, and / or other components. The network interface may be configured to communicate and / or receive Internet Protocol (IP)-based data or other types of data.

[0166]

[0186] Components of a computing device may be implemented in circuitry, for example, components may include and / or be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable electronic circuitry (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuitry), and / or may include and / or be implemented using computer software, firmware, or any combination thereof, to perform various operations described herein.

[0167]

[0187] Process 200, the process of FIG. 7, the process of FIG. 8, the process of FIG. 9, the process of FIG. 11, and / or the process of FIG. 7 are illustrated as logical flow diagrams, block diagrams, or conceptual diagrams whose operations represent a sequence of operations that may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform a particular function or implement a particular data type. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0168]

[0188] Additionally, the process of Figure 7, the process of Figure 8, the process of Figure 9, the process of Figure 11, process 1200, and / or other processes described herein may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that collectively execute on one or more processors, by hardware, or a combination thereof. As mentioned above, the code may be stored on a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

[0169]

[0189] Fig. 13 illustrates an example of a system for implementing some aspects of the present technology. In particular, Fig. 13 illustrates an example of a computing system 1300, which may be, for example, an internal computing system, a remote computing system, a camera, or any computing device constituting any of the components thereof, in which the components of the system communicate with each other using a connection 1305. The connection 1305 may be a physical connection using a bus, or a direct connection to a processor 1310, such as in a chipset architecture. The connection 1305 may also be a virtual connection, a network connection, or a logical connection.

[0170]

[0190] In some embodiments, computing system 1300 is a distributed system in which the functionality described in this disclosure may be distributed across one data center, multiple data centers, a peer network, etc. In some embodiments, one or more of the system components described represent many components, each performing some or all of the functionality that is the subject of the component description. In some embodiments, the components may be physical or virtual devices.

[0171]

[0191] The exemplary system 1300 includes at least one processing unit (CPU or processor) 1310 and connections 1305 coupling various system components to the processor 1310, including system memory 1315, such as read only memory (ROM) 1320 and random access memory (RAM) 1325. The computing system 1300 may include a cache 1312 of high speed memory, either directly connected to the processor 1310, in close proximity to the processor 1310, or integrated as part of the processor 1310.

[0172]

[0192] Processor 1310 may include any general purpose processor, as well as hardware or software services, such as services 1332, 1334, and 1336, stored in storage device 1330 and configured to control processor 1310, as well as special purpose processors whose software instructions are built into the actual processor design. Processor 1310 may essentially be a completely self-contained computing system, including multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors may be symmetric or asymmetric.

[0173]

[0193] To enable user interaction, computing system 1300 includes input devices 1345, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, speech, etc. Computing system 1300 may also include output devices 1335, which can be one or more of several output mechanisms. In some cases, a multi-modal system may enable a user to provide multiple types of input / output to communicate with computing system 1300. Computing system 1300 may generally include a communications interface 1340, which can govern and manage user input and system output.The communications interface may be any of the following: audio jack / plug, microphone jack / plug, universal serial bus (USB) port / plug, Apple® Lightning® port / plug, Ethernet® port / plug, fiber optic port / plug, proprietary wired port / plug, BLUETOOTH® wireless signal transmission, BLUETOOTH low energy (BLE) wireless signal transmission, IBEACON® wireless signal transmission, radio-frequency identification (RFID) wireless signal transmission, near-field communications (NFC) wireless signal transmission, dedicated short range communication (DSRC) wireless signal transmission, 802.11 Wi-Fi wireless signal transmission, wireless local area network (WLAN) signal transmission, Visible Light Communication (VLC), Worldwide Interoperability for Microwave Access (WLAN), and Bluetooth® wireless signal transmission. The wireless communication device may perform or facilitate the reception and / or transmission of wired or wireless communications using wired and / or wireless transceivers, including those utilizing WiMAX (Wireless Access), infrared (IR) communications wireless signal transmission, Public Switched Telephone Network (PSTN) signal transmission, Integrated Services Digital Network (ISDN) signal transmission, 3G / 4G / 5G / LTE cellular data network wireless signal transmission, ad-hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or any combination thereof.The communication interface 1340 may include one or more Global Navigation Satellite System (GNSS) receivers or transceivers used to determine a position of the computing system 1300 based on reception of one or more signals from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the United States Global Positioning System (GPS), the Russian Global Navigation Satellite System (GLONASS), the Chinese BeiDou Navigation Satellite system (BDS), and the European Galileo GNSS. There is no constraint to operate with any particular hardware arrangement, and therefore the basic features herein may be easily substituted for improved hardware or firmware arrangements as they are developed.

[0174]

[0194] The storage device 1330 may be a non-volatile and / or non-transitory and / or computer readable memory device, such as a magnetic cassette, a flash memory card, a solid state memory device, a digital versatile disk, a cartridge, a floppy disk, a flexible disk, a hard disk, a magnetic tape, a magnetic strip / stripe, any other magnetic storage medium, a flash memory, a memristor memory, any other solid state memory, a compact disc read only memory (CD-ROM) optical disk, a rewritable compact disc (CD) optical disk, a digital video disk (DVD) optical disk, a blu-ray disc (BDD) optical disk, a holographic optical disk, another optical medium, a secure digital (SD) card, a micro secure digital (microSD) card, a memory stick card, a smart card chip, an EMV chip, a subscriber identity module (SIM) card, a mini / micro / nano / pico SIM card, another integrated circuit (ICC), circuit (IC) chips / cards, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash EPROM (FLASH EPROM), cache memory (L1 / L2 / L3 / L4 / L5 / L#), resistive random access memory (RRAM),The memory may be a hard disk or other type of computer readable medium capable of storing data that is accessible by a computer, such as random-access memory (RRAM / ReRAM), phase change memory (PCM), spin transfer torque RAM (STT-RAM), another memory chip or cartridge, and / or a combination thereof.

[0175]

[0195] The storage devices 1330 may include software services, servers, services, etc. that cause the system to perform functions when code defining such software is executed by the processor 1310. In some embodiments, hardware services that perform particular functions may include software components stored in a computer-readable medium in conjunction with the necessary hardware components, such as the processor 1310, connections 1305, output devices 1335, etc., to perform the functions.

[0176]

[0196] The term "computer-readable medium" as used herein includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, storing, or transporting instructions and / or data. Computer-readable media may also include non-transitory media on which data is stored and does not include carrier waves and / or transitory electronic signals propagating wirelessly or via wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory, or memory devices. Computer-readable media may have code and / or machine-executable instructions stored thereon, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted using any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0177]

[0197] In some embodiments, computer readable storage devices, media, and memories may include cables or wireless signals containing bit streams, etc. However, when mentioned, non-transitory computer readable storage media specifically excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0178]

[0198] Specific details have been provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For clarity of explanation, in some cases, the technology may be presented as including individual functional blocks, including devices, device components, steps or routines in a method embodied in software, or functional blocks comprising a combination of hardware and software. Additional components may be used other than those shown in the figures and / or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments.

[0179]

[0199] Particular embodiments may be described above as a process or method that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although the flowcharts may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when the operations are completed, but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or to the main function.

[0180]

[0200] The processes and methods according to the examples described above may be implemented using computer-executable instructions stored or otherwise available from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a processing device to perform a function or group of functions, or in some cases configure a general-purpose computer, a special-purpose computer, or a processing device to perform a function or group of functions. Portions of the computer resources used may be accessible over a network. The computer-executable instructions may be, for example, binary, intermediate format instructions, such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during the methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, network-attached storage devices, etc.

[0181]

[0201] Devices implementing the processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments (e.g., computer program product) to perform the necessary tasks may be stored in a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Typical examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, stand-alone devices, etc. The functionality described herein may be embodied in a peripheral device or an add-in card. Such functionality may be implemented on a circuit board among different chips, or on different processes executing in a single device, as further examples.

[0182]

[0202] The instructions, media for carrying such instructions, computing resources for executing such instructions, and other structures for supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0183]

[0203] In the above description, aspects of the present application are described with reference to specific embodiments thereof, but those skilled in the art will recognize that the present application is not limited thereto. Thus, while exemplary embodiments of the present application have been described in detail herein, it should be understood that the inventive concepts may be embodied and employed in various other ways, and that the appended claims are intended to be construed to include such variations, except as limited by the prior art. The various features and aspects of the present application described above may be used individually or jointly. Moreover, the embodiments may be utilized in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the present specification. Thus, the present specification and drawings should be regarded as illustrative and not restrictive. For purposes of illustration, methods have been described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in an order different from that described.

[0184]

[0204] Those skilled in the art will appreciate that the less than ("<") and greater than (">") symbols or terminology used herein may be replaced with the less than or equal to ("≦") and greater than or equal to ("≧") symbols, respectively, without departing from the scope of the present specification.

[0185]

[0205] When a component is described as being "configured to" perform a particular operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0186]

[0206] The phrase "coupled to" refers to any component that is physically connected, either directly or indirectly, to another component and / or that is in communication, either directly or indirectly, with another component (e.g., connected to the other component via a wired or wireless connection and / or other suitable communication interface).

[0187]

[0207] Claim language or other language reciting "at least one of" a set and / or "one or more" of a set indicates that one member of the set or multiple members of the set (in any combination) satisfy the claim. For example, a claim language reciting "at least one of A and B" means A, B, or A and B. In another example, a claim language reciting "at least one of A, B, and C" means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The language "at least one of" a set and / or "one or more" of a set does not limit the set to the items listed in the set. For example, a claim language reciting "at least one of A and B" can mean A, B, or A and B, and can further include items not listed in the set of A and B.

[0188]

[0208] The various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, the various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0189]

[0209] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Any features described as modules or components may be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise a memory or data storage medium, such as a random access memory (RAM), such as a synchronous dynamic random access memory (SDRAM), a read-only memory (ROM), a non-volatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a FLASH memory, a magnetic or optical data storage medium, etc. The techniques may additionally or alternatively be realized at least in part by a computer-readable communications medium, such as a propagated signal or wave, which may carry or communicate program code in the form of instructions or data structures and which may be accessed, read, and / or executed by a computer.

[0190]

[0210] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, or alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, the term "processor" as used herein may refer to any of the above structures, any combination of the above structures, or any other structure or apparatus suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided in dedicated software or hardware modules configured for encoding and decoding, or may be incorporated in a combined video encoder-decoder (CODEC).

[0191]

[0211] Exemplary aspects of the present disclosure include the following.

[0212] Aspect 1. An apparatus for imaging comprising a memory and one or more processors coupled to the memory, the one or more processors being configured to: expose an image sensor to light from the scene, the image sensor including a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another; store in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and generate an image of the scene using at least the first digital pixel data.

[0192]

[0213] Embodiment 2. The apparatus of embodiment 1, wherein the one or more processors use the imaging circuitry to selectively apply a correction to a first subset of the plurality of analog photodetector signals without applying the correction to the second subset of the plurality of analog photodetector signals to read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to a first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second photodetector group.

[0193]

[0214] Example 3. The apparatus of any of Examples 1 or 2, wherein one analog photodetector signal of the plurality of analog photodetector signals indicates a voltage corresponding to a charge from one of the plurality of photodetectors.

[0194]

[0215] Example 4. The apparatus of any of Examples 1 to 3, further comprising an image sensor.

[0195]

[0216] Embodiment 5. The apparatus of any of embodiments 1-4, wherein to read the first digital pixel data from the first subset of the plurality of analog photodetector signals, the one or more processors are configured to cause one or more amplifiers to amplify each of the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into a plurality of amplified analog photodetector signals using the one or more amplifiers, and the first digital pixel data is based on the plurality of amplified analog photodetector signals.

[0196]

[0217] Embodiment 6. The apparatus of embodiment 5, further comprising one or more amplifiers.

[0197]

[0218] Embodiment 7. The apparatus of any of embodiments 1-6, wherein to read the first digital pixel data from the first subset of the plurality of analog photodetector signals, the one or more processors are configured to cause one or more analog-to-digital converters (ADCs) to convert a plurality of analog signals based on the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into digital signals using the one or more ADCs, and the first digital pixel data is based on the digital signals.

[0198]

[0219] Embodiment 8. The apparatus of embodiment 7, further comprising one or more ADCs.

[0199]

[0220] Example 9. The apparatus of any of Examples 7 or 8, wherein the plurality of analog signals includes a plurality of amplified analog photodetector signals, and wherein the one or more amplifiers generate the plurality of amplified analog photodetector signals by at least partially amplifying a first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors.

[0200]

[0221] Aspect 10. The apparatus of any of aspects 7 to 9, wherein the one or more processors are configured to generate first digital pixel data based on the digital signal by at least partially processing the digital signal using one or more image processing operations, the one or more image processing operations including at least one of digital gain, demosaicing, pixel interpolation, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, and upsampling.

[0201]

[0222] Embodiment 11. The apparatus of any of embodiments 1 to 10, wherein the one or more processors are configured to, after generating an image, determine, based on the image, whether to read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0202]

[0223] Embodiment 12. The apparatus of any of embodiments 1 to 11, wherein the one or more processors are configured to read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, then read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and use at least the first digital pixel data and the second digital pixel data to generate a second image of the scene, wherein a second resolution of the second image is higher than the first resolution of the image, and wherein reading the second digital pixel data is based on storing the plurality of analog photodetector signals.

[0203]

[0224] Embodiment 13. The apparatus of embodiment 12, wherein the one or more processors are configured to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group after generating an image of the scene, after reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, and to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0204]

[0225] Example 14. The apparatus of any of Examples 12 or 13, wherein the one or more processors are configured to identify that the image includes an area having a saliency above a saliency threshold, and in response to identifying that the image includes an area having a saliency above the saliency threshold, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0205]

[0226] Embodiment 15. The apparatus of any of embodiments 12-14, wherein the one or more processors are configured to identify that the image exhibits motion relative to a previous image of the scene captured by the image sensor that exceeds a motion threshold, and in response to identifying that the image exhibits motion relative to a previous image of the scene captured by the image sensor that exceeds the motion threshold, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0206]

[0227] Embodiment 16. The apparatus of any of embodiments 12-15, wherein the one or more processors are configured to, after generating the second image, determine based on the second image whether to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group, and the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and the image and the second image are generated without the third digital pixel data.

[0207]

[0228] Embodiment 17. The apparatus of any of embodiments 12-16, wherein the one or more processors are configured to, after reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group, and generate a third image of the scene using at least the first digital pixel data and the second digital pixel data and the third digital pixel data, wherein a third resolution of the third image is higher than the second resolution of the second image; wherein reading the third digital pixel data is based on storing the plurality of analog photodetector signals, and the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and wherein the image and the second image are generated without the third digital pixel data.

[0208]

[0229] Embodiment 18. The apparatus of any of embodiments 12-17, wherein the one or more processors are configured to identify that the image includes a depiction of an object, and in response to identifying that the image includes a depiction of the object, read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0209]

[0230] Aspect 19. The apparatus of aspect 18, wherein the object is a glyph that optically encodes information, the glyph including at least one of a one-dimensional barcode and a two-dimensional code.

[0210]

[0231] Aspect 20. The apparatus of any of aspects 18 or 19, wherein the object includes at least a portion of a person.

[0211]

[0232] Embodiment 21. The apparatus of any one of embodiments 18 to 20, wherein the object includes at least a portion of a vehicle.

[0212]

[0233] Embodiment 22. The device of any of embodiments 18 to 21, wherein the object includes one or more alphanumeric characters.

[0213]

[0234] Embodiment 23. The apparatus of any of embodiments 1 to 22, wherein the multiple photodetectors of the image sensor are arranged according to a grid of blocks, each block including a distinct subset of the multiple photodetectors, and each block of the grid of blocks includes at least one photodetector of the first photodetector group and at least one photodetector of the second photodetector group.

[0214]

[0235] Example 24. The apparatus of any of Examples 1-23, wherein a first group of photodetectors is arranged across the image sensor according to a first pattern, and a second group of photodetectors is arranged across the image sensor according to a second pattern, the first pattern and the second pattern corresponding to one or more gratings.

[0215]

[0236] Embodiment 25. The apparatus of any of embodiments 1-24, wherein the image sensor includes a memory array having a plurality of memory units, and wherein the image sensor is configured to store each of the plurality of analog photodetector signals in one of the plurality of memory units of the memory array to store each of the plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors.

[0216]

[0237] Example 26. The apparatus of example 25, wherein the image sensor is configured to reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing the image.

[0217]

[0238] Aspect 27. The device of aspect 26, wherein the one or more processors are configured to analyze the image using an object detection algorithm, and wherein the image sensor is configured to automatically reset the multiple photodetectors of the image sensor and the multiple storage units of the storage array in response to one or more objects not being detected in the image by the object detection algorithm, after capturing the image, to reset the multiple photodetectors of the image sensor and the multiple storage units of the storage array.

[0218]

[0239] Embodiment 28. The apparatus of any of embodiments 1-27, wherein the image sensor is configured to reset each of the multiple photodetectors of the image sensor prior to exposing the multiple photodetectors to light from the scene.

[0219]

[0240] Embodiment 29. The apparatus of embodiment 28, wherein the image sensor is configured to move a reset pointer across a plurality of reset pointer positions at a predetermined speed from a first side of the image sensor to a second side of the image sensor, and configured to reset a first photodetector of the plurality of photodetectors in response to the reset pointer reaching one of the plurality of reset pointer positions where a first photodetector of the plurality of photodetectors is located to reset each of the plurality of photodetectors, and configured to move a readout pointer across a plurality of readout pointer positions from the first side of the image sensor to the second side of the image sensor at a predetermined speed, and configured to read a third digital pixel data corresponding to a second photodetector of the first photodetector group in response to the readout pointer reaching one of the plurality of reset pointer positions where a second photodetector of the first photodetector group is located to read a first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, wherein the predetermined speed is associated with a rolling shutter.

[0220]

[0241] Embodiment 30. The device of any of embodiments 1-29, wherein the one or more processors are configured to output an image.

[0221]

[0242] Example 31. The apparatus of example 30, further comprising a display, wherein the one or more processors are configured to display the image using the display to output the image.

[0222]

[0243] Aspect 32. The apparatus of any of aspects 30 or 31, further comprising a communications transceiver, wherein the one or more processors are configured to transmit the image to a receiving device using the communications transceiver to output the image.

[0223]

[0244] Embodiment 33. The apparatus of any of embodiments 1-32, wherein the multiple photodetectors of the image sensor include multiple image photodiodes and multiple focus photodiodes, the multiple focus photodiodes are configured for phase detection autofocus (PDAF), the first photodetector group includes a first subset of the multiple image photodiodes, and the second photodetector group includes a second subset of the multiple image photodiodes.

[0224]

[0245] Embodiment 34. The apparatus of any of embodiments 1-33, wherein the plurality of photodetectors comprises a plurality of photodiodes.

[0225]

[0246] Example 35. The apparatus of any of Examples 1-34, wherein the apparatus includes at least one of a mobile handset, a wireless communication device, and a head mounted display.

[0226]

[0247] Aspect 36. A method for imaging comprising: exposing an image sensor to light from a scene, the image sensor including a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group that are distinct from one another; storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and generating an image of the scene using at least the first digital pixel data.

[0227]

[0248] Embodiment 37. The method of embodiment 36, wherein reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group includes selectively applying a correction to the first subset of the plurality of analog photodetector signals without applying the correction to the second subset of the plurality of analog photodetector signals.

[0228]

[0249] Embodiment 38. The method of any of embodiments 36 or 37, wherein one analog photodetector signal of the plurality of analog photodetector signals indicates a voltage corresponding to a charge from one of the plurality of photodetectors.

[0229]

[0250] Embodiment 39. The method of any of embodiments 36-38, wherein the method is implemented by an imaging system including an image sensor.

[0230]

[0251] Embodiment 40. The method of any of embodiments 36-39, wherein reading the first digital pixel data from the first subset of the plurality of analog photodetector signals includes causing one or more amplifiers to amplify each of the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into a plurality of amplified analog photodetector signals, and the first digital pixel data is based on the plurality of amplified analog photodetector signals.

[0231]

[0252] Embodiment 41. The method of embodiment 40, wherein the method is implemented by an imaging system including one or more amplifiers.

[0232]

[0253] Embodiment 42. The method of any of embodiments 36-41, wherein reading the first digital pixel data from the first subset of the plurality of analog photodetector signals includes causing one or more analog-to-digital converters (ADCs) to convert a plurality of analog signals based on the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into digital signals, and the first digital pixel data is based on the digital signals.

[0233]

[0254] Embodiment 43. The method of embodiment 42, wherein the method is implemented by an imaging system including one or more ADCs.

[0234]

[0255] Example 44. The method of any of Examples 42 or 43, wherein the plurality of analog signals includes a plurality of amplified analog photodetector signals, and one or more amplifiers generate the plurality of amplified analog photodetector signals by at least partially amplifying a first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors.

[0235]

[0256] Aspect 45. The method of any of aspects 42-44, further comprising generating first digital pixel data based at least in part on the digital signal by processing the digital signal using one or more image processing operations, the one or more image processing operations including at least one of digital gain, demosaicing, pixel interpolation, missing pixel correction, bad pixel correction, brightness adjustment, contrast adjustment, saturation adjustment, histogram adjustment, color space conversion, automatic white balance adjustment, automatic black balance adjustment, downsampling, and upsampling.

[0236]

[0257] Embodiment 46. The method of any of embodiments 36 to 45, further comprising, after generating the image, determining, based on the image, whether to read second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to a second group of photodetectors.

[0237]

[0258] Embodiment 47. The method of any of embodiments 36-46, further comprising: after reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and using at least the first digital pixel data and the second digital pixel data to generate a second image of the scene, wherein a second resolution of the second image is higher than the first resolution of the image, wherein reading the second digital pixel data is based on storing the plurality of analog photodetector signals.

[0238]

[0259] Embodiment 48. The method of embodiment 47, wherein reading the first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group, after reading the second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, includes reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, after generating an image of the scene.

[0239]

[0260] Aspect 49. The method of any of aspects 47 or 48, further comprising identifying that the image includes an area having saliency above a saliency threshold, and reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group in response to identifying that the image includes an area having saliency above the saliency threshold, comprising reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0240]

[0261] Embodiment 50. The method of any of embodiments 47-49, further comprising identifying that the image exhibits motion exceeding a motion threshold relative to a previous image of the scene captured by the image sensor, and reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group in response to identifying that the image exhibits motion exceeding a motion threshold relative to a previous image of the scene captured by the image sensor, comprising reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0241]

[0262] Embodiment 51. The method of any of embodiments 47 to 50, further comprising, after generating the second image, determining based on the second image whether to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group, wherein the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and the image and the second image are generated without the third digital pixel data.

[0242]

[0263] Embodiment 52. The method of any of embodiments 47-51, further comprising: after reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, reading third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group; and using at least the first digital pixel data and the second digital pixel data and the third digital pixel data to generate a third image of the scene, wherein a third resolution of the third image is higher than the second resolution of the second image. The method of any of embodiments 47-51, wherein reading the third digital pixel data is based on storing the plurality of analog photodetector signals, and the plurality of photodetectors includes a third photodetector group different from the first photodetector group and the second photodetector group, and the image and the second image are generated without the third digital pixel data.

[0243]

[0264] Embodiment 53. The method of any of embodiments 47-52, further comprising identifying that the image includes a depiction of an object, and wherein reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group comprises reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group in response to identifying that the image includes a depiction of the object.

[0244]

[0265] Aspect 54. The method of aspect 53, wherein the object is a glyph that optically encodes information, and the glyph includes at least one of a one-dimensional barcode and a two-dimensional code.

[0245]

[0266] Embodiment 55. The method of any of embodiments 53 or 54, wherein the object comprises at least a part of a person.

[0246]

[0267] Embodiment 56. The method of any one of embodiments 53 to 55, wherein the object includes at least a portion of a vehicle.

[0247]

[0268] Embodiment 57. The method of any one of embodiments 53 to 56, wherein the object includes one or more alphanumeric characters.

[0248]

[0269] Embodiment 58. The method of any of embodiments 36 to 57, wherein the multiple photodetectors of the image sensor are arranged according to a grid of blocks, each block including a distinct subset of the multiple photodetectors, and each block of the grid of blocks includes at least one photodetector of the first photodetector group and at least one photodetector of the second photodetector group.

[0249]

[0270] Embodiment 59. The method of any of embodiments 36-58, wherein a first group of photodetectors is arranged across the image sensor according to a first pattern, and a second group of photodetectors is arranged across the image sensor according to a second pattern, the first pattern and the second pattern corresponding to one or more gratings.

[0250]

[0271] Embodiment 60. The method of any of embodiments 36-59, wherein the image sensor includes a memory array having a plurality of storage units, and storing each of the plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors includes storing each of the plurality of analog photodetector signals in one of the plurality of storage units of the memory array.

[0251]

[0272] Embodiment 61. The method of embodiment 60, further comprising resetting the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing the image.

[0252]

[0273] Aspect 62. The method of aspect 61, further comprising analyzing the image using an object detection algorithm, and resetting the multiple photodetectors of the image sensor and the multiple storage units of the storage array after capturing the image includes automatically resetting the multiple photodetectors of the image sensor and the multiple storage units of the storage array in response to one or more objects not being detected in the image by the object detection algorithm.

[0253]

[0274] Embodiment 63. The method of any of embodiments 36-62, further comprising resetting each of the multiple photodetectors of the image sensor prior to exposing the multiple photodetectors to light from the scene.

[0254]

[0275] Embodiment 64. The method of embodiment 63, further comprising: moving a reset pointer at a predetermined speed from a first side of the image sensor to a second side of the image sensor across a plurality of reset pointer positions, where resetting each of the plurality of photodetectors includes resetting a first photodetector of the plurality of photodetectors in response to the reset pointer reaching one of the plurality of reset pointer positions where a first photodetector of the plurality of photodetectors is located; and moving a readout pointer at a predetermined speed from the first side of the image sensor to the second side of the image sensor across a plurality of readout pointer positions, where reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group includes reading third digital pixel data corresponding to a second photodetector of the first photodetector group in response to the readout pointer reaching one of the plurality of reset pointer positions where a second photodetector of the first photodetector group is located, wherein the predetermined speed is associated with a rolling shutter.

[0255]

[0276] Embodiment 65. The method of any one of embodiments 36 to 64, further comprising outputting the image.

[0256]

[0277] Embodiment 66. The method of embodiment 65, wherein outputting the image includes displaying the image using a display.

[0257]

[0278] Aspect 67. The method of any of aspects 65 or 66, wherein outputting the image includes transmitting the image to a receiving device using a communications transceiver.

[0258]

[0279] Embodiment 68. The method of any of embodiments 36-67, wherein the multiple photodetectors of the image sensor include multiple image photodiodes and multiple focus photodiodes, the multiple focus photodiodes are configured for phase detection autofocus (PDAF), the first photodetector group includes a first subset of the multiple image photodiodes, and the second photodetector group includes a second subset of the multiple image photodiodes.

[0259]

[0280] Embodiment 69. The method of any of embodiments 36-68, wherein the plurality of photodetectors comprises a plurality of photodiodes.

[0260]

[0281] Aspect 70. A non-transitory computer readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to expose a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another, store a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors, read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group, and generate an image of the scene using at least the first digital pixel data.

[0261]

[0282] Embodiment 71. The non-transitory computer-readable medium of embodiment 70, wherein execution of the instructions by one or more processors further causes the one or more processors to perform operations according to any of embodiments 2-35 and / or any of embodiments 37-69.

[0262]

[0283] Aspect 72. An apparatus for imaging comprising: means for exposing an image sensor to light from a scene, the image sensor comprising a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group that are distinct from one another; means for storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; means for reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; and means for generating an image of the scene using at least the first digital pixel data.

[0263]

[0284] Example 73. The apparatus of example 72, further comprising means for performing the operations according to any of examples 2-35 and / or any of examples 37-69.

[0264]

[0285] Aspect 74. An apparatus for imaging comprising a memory and one or more processors coupled to the memory, the one or more processors being configured to: expose a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors being configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another; store in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; generate an image of the scene using at least the first digital pixel data; and after generating the image, determine based on the image whether to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0265]

[0286] Embodiment 75. The apparatus of embodiment 74, wherein the one or more processors are configured to perform one or more operations according to any of embodiments 2-35.

[0266]

[0287] Aspect 76. A method for imaging comprising: exposing a plurality of photodetectors of an image sensor to light from a scene, each of the plurality of photodetectors being configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group that are distinct from each other; storing in the image sensor a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; generating an image of the scene using at least the first digital pixel data; and after generating the image, determining based on the image whether to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0267]

[0288] Example 77. The method of example 76, further comprising one or more operations according to any of examples 37-69.

[0268]

[0289] Aspect 78. A non-transitory computer readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: expose a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group distinct from one another; cause the image sensor to store a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; read first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; generate an image of the scene using at least the first digital pixel data; and determine, after generating the image, based on the image, whether to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0269]

[0290] Aspect 79. The non-transitory computer-readable medium of aspect 78, wherein execution of the instructions by one or more processors further causes the one or more processors to perform operations according to any of aspects 2-35 and / or any of aspects 37-69.

[0270]

[0291] Aspect 80. An apparatus for imaging comprising: exposing an image sensor to light from a scene, the image sensor comprising a plurality of photodetectors, each of the plurality of photodetectors configured to convert light from the scene into an electric charge in response to exposure to light from the scene, the plurality of photodetectors including at least a first photodetector group and a second photodetector group that are distinct from one another; means for causing the image sensor to store a plurality of analog photodetector signals corresponding to the electric charges from each of the plurality of photodetectors; means for reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first photodetector group without reading second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group; means for generating an image of the scene using at least the first digital pixel data; and means for determining, after generating the image, based on the image, whether to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second photodetector group.

[0271]

[0292] Example 81. The apparatus of example 80, further comprising means for performing the operations according to any of examples 2-35 and / or any of examples 33-69.

Claims

1. Memory and one or more processors coupled to the memory; 1. An apparatus for imaging, comprising: exposing a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors configured to convert the light from the scene into an electric charge in response to exposure to the light from the scene, the plurality of photodetectors including at least a first group of photodetectors and a second group of photodetectors that are distinct from one another; causing the image sensor to store a plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors; reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors; generating a first image of the scene using at least the first digital pixel data; Identifying the first image as including an area having a saliency above a saliency threshold or identifying the first image as including a depiction of an object; reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors after reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors in response to identifying the first image including an area having a saliency above a saliency threshold or in response to identifying the image including a depiction of an object, wherein reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors, wherein reading the second digital pixel data is based on storing the plurality of analog photodetector signals. generating a second image of the scene using at least the first digital pixel data and the second digital pixel data, the second image having a second resolution greater than the first resolution of the first image; It is configured as follows: Device.

2. 2. The apparatus of claim 1, wherein the one or more processors use imaging circuitry to selectively apply a modification to the first subset of the plurality of analog photodetector signals without applying the modification to the second subset of the plurality of analog photodetector signals to read the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors without reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors, wherein the modification includes one or more processing operations by an image processor or filtering by one or more filters.

3. 2. The apparatus of claim 1, wherein one analog photodetector signal of the plurality of analog photodetector signals indicates a voltage corresponding to the charge from one of the plurality of photodetectors.

4. the image sensor The apparatus of claim 1 further comprising:

5. 2. The apparatus of claim 1 , wherein to read the first digital pixel data from the first subset of the plurality of analog photodetector signals, the one or more processors are configured to cause one or more amplifiers to amplify each of the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into a plurality of amplified analog photodetector signals using the one or more amplifiers, and the first digital pixel data is based on the plurality of amplified analog photodetector signals.

6. the one or more amplifiers The apparatus of claim 5 further comprising:

7. 2. The apparatus of claim 1 , wherein to read the first digital pixel data from the first subset of the plurality of analog photodetector signals, the one or more processors are configured to cause one or more analog-to-digital converters (ADCs) to convert, using the one or more ADCs, a plurality of analog signals based on the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors into digital signals, the first digital pixel data being based on the digital signals, and the apparatus further comprising the one or more ADCs.

8. 8. The apparatus of claim 7, wherein the plurality of analog signals comprises a plurality of amplified analog photodetector signals, and wherein one or more amplifiers generate the plurality of amplified analog photodetector signals by at least partially amplifying the first subset of the plurality of analog photodetector signals that correspond to the first group of photodetectors.

9. the one or more processors:

2. The apparatus of claim 1, further configured to identify that the first image exhibits motion relative to a previous image of the scene captured by the image sensor that exceeds a motion threshold, and to read the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors in response to identifying that the first image exhibits the motion relative to the previous image of the scene captured by the image sensor that exceeds the motion threshold.

10. the one or more processors: after generating the second image, determining based on the second image whether to read third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third photodetector group, wherein the plurality of photodetectors includes the third photodetector group different from the first photodetector group and the second photodetector group, and the first image and the second image are generated without the third digital pixel data; or the one or more processors: after reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors, reading third digital pixel data from a third subset of the plurality of analog photodetector signals corresponding to a third group of photodetectors; generating a third image of the scene using at least the first digital pixel data, the second digital pixel data, and the third digital pixel data, wherein a third resolution of the third image is greater than the second resolution of the second image. It is structured as follows:

2. The apparatus of claim 1, wherein reading the third digital pixel data is based on storing the plurality of analog photodetector signals, the plurality of photodetectors including the third group of photodetectors different from the first group of photodetectors and the second group of photodetectors, and the image and the second image are generated without the third digital pixel data.

11. The device of claim 1, wherein the first image includes a depiction of an object, the object being a glyph that optically encodes information, the glyph including at least one of a one-dimensional barcode and a two-dimensional code, or the object including at least one of a part of a person, a part of a vehicle, and an alphanumeric character.

12. the plurality of photodetectors of the image sensor are arranged according to a grid of blocks, each block comprising a distinct subset of the plurality of photodetectors, each block of the grid of blocks comprising at least one photodetector of the first group of photodetectors and at least one photodetector of the second group of photodetectors; or 2. The apparatus of claim 1, wherein the first group of photodetectors is arranged across the image sensor according to a first pattern and the second group of photodetectors is arranged across the image sensor according to a second pattern, the first pattern and the second pattern corresponding to one or more gratings.

13. the image sensor includes a storage array having a plurality of storage units, and the image sensor is configured to store each of the plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors in one of the plurality of storage units of the storage array, the image sensor is configured to reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing the image, and the one or more processors:

2. The apparatus of claim 1, further configured to analyze the image using an object detection algorithm, and to reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array after capturing the image, wherein the image sensor is further configured to automatically reset the plurality of photodetectors of the image sensor and the plurality of storage units of the storage array in response to one or more objects not being detected in the image by the object detection algorithm.

14. The image sensor configured to reset each of the plurality of photodetectors of the image sensor before exposing the plurality of photodetectors to the light from the scene; configured to move a reset pointer from a first side of the image sensor to a second side of the image sensor at a predetermined speed over a plurality of reset pointer positions, and configured to reset a first photodetector of the plurality of photodetectors in response to the reset pointer reaching one of the plurality of reset pointer positions at which a first photodetector of the plurality of photodetectors is located, to reset each of the plurality of photodetectors; a readout pointer is configured to move from the first side of the image sensor to the second side of the image sensor at a predetermined speed over a plurality of readout pointer positions to read the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors, and the image sensor is configured to read third digital pixel data corresponding to the second photodetector of the first group of photodetectors in response to the readout pointer reaching one of the plurality of reset pointer positions at which a second photodetector of the first group of photodetectors is located; The apparatus of claim 1 , wherein the predetermined speed is associated with a rolling shutter.

15. the one or more processors: configured to output the image, wherein the device further comprises a display, and wherein, to output the image, the one or more processors are configured to display the image using the display.

10. The apparatus of claim 1.

16. 2. The apparatus of claim 1, wherein the plurality of photodetectors of the image sensor comprises a plurality of image photodiodes and a plurality of focus photodiodes, the plurality of focus photodiodes being configured for phase detection autofocus (PDAF), the first group of photodetectors comprising a first subset of the plurality of image photodiodes, and the second group of photodetectors comprising a second subset of the plurality of image photodiodes.

17. exposing a plurality of photodetectors of the image sensor to light from the scene, each of the plurality of photodetectors configured to convert the light from the scene into an electric charge in response to exposure to the light from the scene, the plurality of photodetectors including at least a first group of photodetectors and a second group of photodetectors that are distinct from one another; causing the image sensor to store a plurality of analog photodetector signals corresponding to the charge from each of the plurality of photodetectors; reading first digital pixel data from a first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors without reading second digital pixel data from a second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors; generating a first image of the scene using at least the first digital pixel data; identifying the first image as including an area having a saliency above a saliency threshold, identifying the first image as including an area having a saliency above a saliency threshold, or identifying the first image as including a depiction of an object; reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the first group of photodetectors after reading the first digital pixel data from the first subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors in response to identifying the first image including an area having a saliency above a saliency threshold or in response to identifying the image including a depiction of an object, wherein reading the second digital pixel data from the second subset of the plurality of analog photodetector signals corresponding to the second group of photodetectors, wherein reading the second digital pixel data is based on storing the plurality of analog photodetector signals. generating a second image of the scene using at least the first digital pixel data and the second digital pixel data, the second image having a second resolution greater than the first resolution of the first image; An imaging method comprising:

Images