Lower power operation of differential time-of-flight sensor pixels

Abstract

An imaging system includes a sensor array of differential pixels. A controller operates a first differential pixel of the sensor array in a first, lower power mode. The controller supplies a first clock signal for a first duration to selectively activate a first collection terminal of the first differential pixel, and a second clock signal for a second duration to selectively activate a second collection terminal of the first differential pixel. In an analog domain, a first amount of charge accumulated at the first collection terminal over the first duration is read out and compared to a readout of a second amount of charge accumulated at the second collection terminal over the second duration. In response to the first amount of charge differing from the second amount of charge by more than a threshold, the first differential pixel of the sensor array is operated in a second, higher power mode.

Description

Background Technology

[0001] Two-dimensional (2D) cameras can rely on ambient lighting of the scene to acquire 2D passively lit images. The digital computations performed to process and analyze the data from a 2D camera are likely to be relatively small. These factors make 2D cameras relatively power-efficient. In contrast, three-dimensional (3D) cameras, such as time-of-flight (ToF) cameras, operate by actively illuminating the scene and determining the depth of objects in the scene that reflect the active illumination back to the 3D camera by measuring one or more parameters of the active illumination. The generation of active illumination and the computations used to process and analyze the data from a 3D camera can be relatively resource-intensive. Summary of the Invention

[0002] This summary is provided to present, in a simplified form, the selection of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to the implementation of solutions to any or all the shortcomings pointed out in any part of this disclosure.

[0003] Examples relating to adjusting the power utilized when operating a ToF camera are disclosed. One example provides an imaging system comprising a sensor array of differential pixels. A controller operates the first differential pixels of the sensor array in a first lower power mode. The controller supplies a first clock signal for a first duration to selectively activate a first acquisition terminal of the first differential pixel, and supplies a second clock signal for a second duration to selectively activate a second acquisition terminal of the first differential pixel. In the analog domain, a first charge accumulated at the first acquisition terminal during the first duration is read out and compared with a readout of a second charge accumulated at the second acquisition terminal during the second duration. In response to a difference between the first and second charge amounts exceeding a threshold, the first differential pixels of the sensor array are operated in a second higher power mode. Attached Figure Description

[0004] Figure 1A-1C An example imaging system is illustrated schematically.

[0005] Figure 2 An example circuit for a transmission gate time-of-flight pixel is schematically shown.

[0006] Figure 3 An example timing diagram of the global shutter operation of the transmission gate time-of-flight pixels is shown.

[0007] Figure 4 An example timing diagram of a single row of transmission gate time-of-flight pixels under global shutter operation is shown.

[0008] Figure 5A flowchart illustrating an example method for manipulating differential time-of-flight pixels is shown.

[0009] Figure 6 It shows how to use Figure 5 Example timing diagram of the method for manipulating differential time-of-flight pixels.

[0010] Figure 7 An example graph showing changes in brightness in the environment is shown.

[0011] Figure 8 An example comparator circuit is shown schematically.

[0012] Figure 9 An example differential circuit is shown schematically.

[0013] Figure 10A A pixel array operating at full resolution is shown schematically.

[0014] Figure 10B The extracted pixel array is shown schematically.

[0015] Figure 11 A flowchart depicting an example method for manipulating a pixel array that includes regions of differential time-of-flight pixels is shown.

[0016] Figure 12 A flowchart is shown to illustrate an example method for detecting a step change in brightness at a differential pixel.

[0017] Figure 13 It shows how to use Figure 12 Example timing diagram of the method for manipulating pixel arrays.

[0018] Figure 14 An example computing system is illustrated schematically. Detailed Implementation

[0019] In some cases, operating a time-of-flight (ToF) camera can be both computationally expensive and power-intensive. This is especially true for battery-powered imaging systems, where it may be necessary to reduce power consumption while maintaining frame rate requirements for accurate depth imaging.

[0020] ToF cameras utilizing differential pixel operation have various power consumptions, including clock switching between A and B differentials, analog-to-digital converter (ADC) operation, and operation of the internal illumination source (which may include a laser). However, sometimes the scene being imaged, or a portion of the scene, may be characterized by slowly moving or stationary objects. Therefore, in some cases, continuously illuminating the entire scene and processing depth data at each pixel at a relatively high rate can be wasteful of power.

[0021] To conserve power, the camera can be configured to react to the surrounding scene, illuminating only the parts of the scene of direct interest. Schemes such as "progressive wake" and depth-on-demand have been considered, consuming computational resources and power only when needed. However, most such schemes involve reading one frame, storing it in memory, and then reading another frame and performing a digital comparison. This requires the simultaneous use of both the ADC and memory.

[0022] Therefore, systems and methods for providing scene-based operating modes are disclosed, which can provide power savings at pixels, illumination sources, and ADCs when a relatively static scene or an uninteresting part of a scene is presented.

[0023] As an example, the pixel operates in a first, lower power mode as the default condition. A low-frequency image of the pixel is obtained and compared in the analog domain to determine if there is a brightness change in the pixel's field of view. A brightness change greater than a threshold causes a wake-up signal, resulting in the pixel operating in a higher power mode. By comparing analog voltages, there is no need to call the ADC, and the storage of the previous image in memory can be omitted.

[0024] If the brightness becomes static during operation in a higher power mode, the pixel can return to a lower power mode. In some examples, a subset of sampled pixels can operate in lower power mode while the remaining pixels are turned off. A wake-up signal can be applied to the pixel region, resulting in additional power savings. Step changes in scene brightness can be detected by interleaving readouts of pixel rows, whether they occur during integration or readout.

[0025] Figure 1A-1C Several aspects of an example imaging system 100 are illustrated. The imaging system 100 may also be referred to herein as a camera. The term "camera" herein refers to any imaging component having at least one optical aperture and a sensor array configured to image a scene or object 102. The imaging system 100 includes a sensor array 104 of individually addressable pixels 106. In some implementations, the pixels may be complementary metal-oxide-semiconductor (CMOS) elements, but other suitable architectures are also contemplated. Each pixel responds to light over a wide wavelength range. For example, for silicon-based (e.g., CMOS) pixels, the wavelength response can be in the range of 300 to 1000 nm. For simplicity, the sensor array 104 is schematically shown as having only 25 pixels 106, but any other suitable number of pixels can be used.

[0026] In some implementations, the pixels 106 of the sensor array 104 may be differential pixels. Each differential pixel may include different acquisition terminals excited according to two different clock signals. In one example, for measuring modulated active illumination, the two clock signals may be substantially complementary (e.g., the two clock signals have a 50% duty cycle with a 180-degree phase difference). In other examples, the two different clock signals may have different relationships, such as for measuring ambient illumination or unmodulated active illumination.

[0027] This operation allows the use of the same sensor array to measure active light across a broad spectrum, including ultraviolet, visible, near-infrared, and infrared light. Furthermore, differential pixels reduce system noise because only a single readout operation is used to perform differential measurements of active spectral light and ambient light (i.e., the difference between measurements of active and ambient light and measurements of ambient light without active light). Specifically, each pixel readout / reset operation increases system noise each time it occurs. Therefore, instead of performing two separate measurements (i.e., active and passive) that require two readouts and then adding (or subtracting) each result to memory, a single differential measurement can be performed within a pixel, and the single differential measurement can be written to memory. This is discussed in this paper and referenced in [reference 1]. Figure 3 Additional aspects describing the differential pixel structure and operations.

[0028] In other camera implementations that do not include differential pixel sensor arrays, additional clock cycles can be used to perform differential measurements. While differential pixels offer the advantages described herein, it should be understood that other types of sensor arrays, including non-differential sensor arrays, can be used.

[0029] The microlens array 108 can optionally be arranged directly on top of the sensor array 104. The microlens array 108 includes a plurality of microlens elements 110. Each microlens element 110 of the microlens array 108 can be registered to a differential pixel 106 of the sensor array 104. When included, the microlens array 108 can provide a larger effective fill factor at each pixel to improve acquisition efficiency and reduce crosstalk between pixels.

[0030] Optical shutter 112 may optionally be arranged above sensor array 104 to optically cover the sensor array. When included, optical shutter 112 can be electronically switched between different filter states. In one example, optical shutter 112 may include two or more filter states. In one filter state, optical shutter 112 may transmit IR light and block light outside the IR band (e.g., visible light). In another filter state, optical shutter 112 may transmit visible light and block light outside the visible light subband (e.g., IR light). Optical shutter 112 may be configured to transmit light in any suitable selected subband of the electromagnetic spectrum and block light outside the selected subband. Optical shutter 112 may increase the signal-to-noise ratio of the IR image and the visible light image acquired by sensor array 104. Optical shutter 114 may include any suitable type of filter that transmits narrowband light without significantly reducing the intensity of the in-band signal received by the sensor array. In one example, optical shutter may include one or more liquid crystal layers. The optical shutter 112 can be configured as a global shutter, wherein accumulated charge is stored in the shaded area on a per-pixel or per-pixel group basis. The optical shutter 112 can be additionally or alternatively configured as a rolling shutter, wherein different portions of the image frame are read out at different times, such as on a sequential line-by-line basis.

[0031] In some implementations, the optical shutter can be omitted from the imaging system 100. For example, the optical shutter can be omitted to reduce the cost and / or complexity of the imaging optics stack. In some cases, omitting the optical shutter can allow the imaging optics stack to have an advantageously small total optical track length (TTL) because the imaging lens can be designed without increasing the complexity of having tunable filters in the imaging optics stack.

[0032] The tunable illumination source 114 can be configured to selectively emit active illumination to illuminate scene 102. The tunable illumination source 114 can emit any suitable active illumination. In some embodiments, the tunable illumination source 114 can be modulated. In other embodiments, the tunable illumination source 114 can be unmodulated. In some examples, the tunable illumination source 114 may include a solid-state laser or LED that operates in the near-infrared or infrared (IR) range (approximately 850 nm) to emit active IR light.

[0033] In other embodiments, the steerable illumination source 114 may be configured to emit active illumination light in the visible spectrum. In some examples, the steerable illumination source 114 may include a broadband illumination source, such as a white light source. In some examples, the steerable illumination source 114 may include multiple spectral illuminators (e.g., LEDs). In some such examples, the multiple spectral illuminators may be configured to emit active illumination in the same spectral band, but this is not required.

[0034] In some examples, the maneuverable illumination source 114 can be configured to selectively emit active illumination light with a narrow field, the size of which is determined to be an active illumination region in the scene 102. Furthermore, the maneuverable illumination source 114 includes a manipulation element 116 configured to manipulate the active illumination light emitted from the maneuverable illumination source 114 to individually and actively illuminate different illumination regions among a plurality of illumination regions 124 in the scene 102 viewed by the sensor array 104. Figure 1B and 1C (As shown in the diagram). Such a partitioned arrangement produces active illumination with a smaller angular range than the field of view of sensor array 104, and thus can provide a greater power density for active illumination with the same peak power compared to full-field imaging. The illumination area can be configured to be smaller than any suitable size of the entire field of view of the scene viewed by sensor array 104. In some examples, the size of each illumination area can be based on the characteristics of the manipulable illumination source 114. Non-limiting examples of these characteristics include size, power consumption, frequency, angle, position repeatability, and the drive voltage requirements of the manipulable illumination source 114.

[0035] In some implementations, a calibration process can be performed on the imaging system 100 to determine multiple illumination regions. The calibration process may also include determining the spatial mapping of the sensor array 104 to pixels 106 in each illumination region. In some examples, the determined illumination regions may collectively cover the entire field of view of the scene 102 viewed by the sensor array 104. In some examples, the different illumination regions may overlap each other. Any suitable number of illumination regions can be determined to collectively cover the field of view of the sensor array 104. Furthermore, any suitable number of pixels in the sensor array can be mapped to each illumination region.

[0036] Manipulation element 116 may include any suitable mechanical, electro-optic, microelectromechanical system (MEMS), electrowetting prism assembly, and / or other manipulation components configured to appropriately manipulate active illumination emitted from manipulatorable illumination source 116 to illuminate a designated illumination area. In some examples, manipulation element 116 may include a movable mirror providing mechanical manipulation components to manipulate the active illumination light to illuminate the illumination area in scene 102. In one example, at least one of a plurality of mirrors includes a movable mirror (e.g., a micromirror). In some examples, manipulation element 116 may include a refractive lens (e.g., a Fresnel lens, a prism, etc.) having a non-uniform surface that guides or manipulates the light in different directions based on the lateral position of the input light. In some examples, manipulation element 116 may include a switchable polarization grating to provide electro-optic manipulation components. In some examples, manipulation element 116 may include a liquid crystal lens system (e.g., a pair of liquid crystal lenses) providing manipulation via an electrowetting manipulation solution (electrowetting assembly). In some examples, a liquid crystal lens can be an array of microlenses suspended in a liquid crystal, which can be electrically tunable to manipulate light.

[0037] In some embodiments, the steerable illumination source 114 may optionally include one or more optical elements 118. For example, the optical elements 118 may include one or more of a collimating element, a diffuser element, and a focusing element. The collimating element is operable to collimate light emitted from the steerable illumination source 114 into collimated light. The diffuser element is operable to diffuse light emitted from the steerable illumination source 114, thereby converting the collimated light into diffuse light with a desired distribution (e.g., a uniform or Gaussian power distribution). The focusing element is operable to focus the diffused light at a specified focal length. Such a specified focal length may be selected based on the application or any other suitable factor. In some embodiments, any or all of these optical elements may be omitted from the steerable illumination source 114.

[0038] Controller 120 may include a logic machine and an associated storage machine. The storage machine may hold instructions that can be executed by the logic machine to perform any operations, algorithms, computations, or transformations disclosed herein. In some implementations, the logic machine may take the form of an application-specific integrated circuit (ASIC) or a system-on-a-chip (SoC), some or all of which are hardware- or firmware-coded.

[0039] The controller 120 can be configured to individually control the pixels 106 of the sensor array 104 and the manipulable illumination source 114 to selectively acquire different types of images (e.g., 2D, 3D) of at least a portion of the scene 102 viewed by the sensor array 104. Figure 1AIn the example shown, at time T0, controller 120 can be configured to address all pixels 106 of sensor array 104 while manipulable illumination source 114 is deactivated to acquire a 2D passively illuminated image (e.g., monochrome or color image) of scene 102. As used herein, time T0 is considered to include the entire duration from time T0 to time T1, thus allowing sequential addressing of individual pixels within the T0 time frame. The same convention applies to other times discussed herein. As used herein, the term "deactivated" means that manipulable illumination source 114 does not emit active illumination light to illuminate scene 102. When manipulable illumination source 114 is deactivated, manipulable illumination source 114 may consume minimal or no power. Instead, in this case, sensor array 104 relies on ambient illumination of scene 102 to acquire a 2D passively illuminated image 122. In an implementation including optical shutter 112, controller 120 can be configured to switch optical shutter 112 to allow light in the visible spectrum to pass through sensor array 104.

[0040] The controller 120 can be configured to control the sensor array 104 and the manipulable illumination source 114 to acquire image data based on active illumination of various illumination areas of scene 102. The controller 120 can use this image data based on active illumination to determine depth values ​​of objects located within the illumination areas, as described in further detail below. Figure 1B In the example shown, at time T1, the manipulable illumination source 114 is configured to emit active IR illumination light. The controller 120 can be configured to control (i.e., activate and manipulate) the manipulable illumination source 114 to illuminate a first illumination region 124a with active IR illumination light 128. The controller 120 can be configured to address pixels 106 of the sensor array 104 mapped to the first illumination region 124a to acquire IR image data of the first illumination region. Specifically, active IR illumination light 128' reflected from object trajectories in scene 102 and incident on each sensor mapped to the first illumination region 124a is measured for IR image acquisition. Note that the controller 120 may not address or "read out" pixels 106 of the sensor array 104 that are not mapped to the first illumination region 124a at time T1 to improve the power efficiency of the imaging system 100. When the optical shutter 112 is included, the controller 120 can be configured to switch the optical shutter 112 in sync with the activation of the steerable illumination source 114 for IR image acquisition of the first illumination region 124a, so as to block light outside the IR (or near-IR) photonic band.

[0041] In addition, such as Figure 1CAs shown, at time T2, controller 120 can be configured to control (i.e., activate and manipulate) the manipulable illumination source 114 to illuminate the second illumination region 124b with active IR illumination light 130. Controller 120 can be configured to address pixels 106 of sensor array 104 mapped to the second illumination region 124b to acquire IR image data of the second illumination region. Specifically, active IR illumination light 130' reflected from object trajectories in scene 102 and incident on each pixel mapped to the second illumination region 124b is measured for IR image acquisition. Note that controller 120 may not address or "read out" pixels 106 of sensor array 104 not mapped to the second illumination region 124b at time T2 to improve the power efficiency of imaging system 100. When the optical shutter 112 is included, the controller 120 can be configured to switch the optical shutter 112 in sync with the activation of the steerable illumination source 114 for IR image acquisition of the second illumination region 124b, so as to block light outside the IR (or near-IR) photonic band.

[0042] The controller 120 can be configured to repeat the imaging operation described above to acquire image data based on active illumination of any appropriate number of different illumination areas. For example, such image data can be acquired for 3D imaging of different illumination areas. In some examples, the entire field of view of the scene viewed by the sensor array 104 can be imaged by sequentially and individually imaging multiple illumination areas covering the field of view. In some examples, only the identified region of interest covered by one or more illumination areas can be imaged by individually imaging the illumination areas covering the identified region of interest.

[0043] Depending on the described imaging mode, the term "addressing" applied to pixel 106 of sensor array 104 may have a slightly different meaning. For planar imaging of spectral light including visible and IR light, addressing pixel 106 may include integrating the intensity of ambient light received at each pixel 106 and associating the integrated ambient light intensity in a sub-band with the portion of the 2D image corresponding to that pixel.

[0044] For depth and / or 3D imaging, pixel 106 can be addressed differently. Here, addressing a pixel can include resolving a phase offset from each pixel relative to the modulation intensity of the IR light. The phase offset, optionally converted to the depth domain, can be associated with a portion of the image corresponding to the addressed pixel. In other words, controller 120 can be configured to determine a depth value for each pixel 106 of the addressed sensor array 104. In some implementations, the phase offset can be obtained using a fast, sequential series of IR image acquisitions (e.g., 6-9). In combined 2D / 3D imaging applications, both addressing modes described above can be used in a time-multiplexed manner.

[0045] The aforementioned phase-discriminated time-of-flight (ToF) method is one of several depth imaging techniques covered by this disclosure. Typically, a depth imaging camera can be configured to acquire one or more depth maps of a scene or object. The term "depth map" refers to an array of pixels registered to corresponding regions (Xi, Yi) of the imaged scene, with a depth value (Zi) indicating the depth of the corresponding region for each pixel. In some examples, "depth" can be defined as a coordinate parallel to the camera's optical axis, increasing with distance from the camera. In other examples, depth can be defined as the radial distance from the camera. The term "depth video" herein refers to a time-resolved sequence of the depth map. In a ToF implementation, an IR illumination source (e.g., a manipulable illumination source 114) can project pulsed IR illumination or otherwise modulated IR illumination onto the scene. The sensor array of the depth imaging camera can be configured to detect the phase shift between the illumination reflected from the scene and the modulated emission. In some implementations, the phase shift of each pixel can be converted to the pixel-resolved time-of-flight of the pulsed illumination from the illumination source to the scene and back to the array. The ToF data can then be converted into depth data.

[0046] Controller 120 can be configured to output 2D and 3D image data (or depth data) in any suitable form. As an example, controller 120 can be configured to output a matrix of 126 pixels. Each pixel in the matrix includes a depth value (Zi) and one or more visible light values ​​(e.g., monochrome or color values). In some examples, controller 120 can output the matrix of 126 pixels as a data structure where each element of the matrix corresponds to a different pixel, and each element of the matrix includes an array of values ​​corresponding to the depth value and one or more visible light values ​​for that pixel. Controller 120 can be configured to output the matrix of 126 pixels (and / or any other suitable parameter values) to any suitable receiver, either inside or outside the imaging system 100. In one example, controller 120 can be configured to output the matrix of 126 pixels to another processing component for additional image processing (e.g., filtering, computer vision). In some examples, the processing component can be incorporated into the imaging system 100. In other examples, the processing component can be incorporated into a remote computing device communicating with the imaging system 100. In a further example, controller 120 can be configured to output pixel matrix 126 to an external display device for visual presentation as an image. Furthermore, in some implementations, 2D and 3D image data can be represented using different data structures (e.g., a first matrix for monochrome or color values ​​and a second matrix for depth values).

[0047] In some implementations, the pixels 106 of the sensor array 104 can be differential pixels, such as transmission gate ToF pixels and / or photoelectric gate ToF pixels. Figure 2 An exemplary schematic diagram of a transmission-gate ToF pixel 200 is shown. While this detailed description will primarily use a transmission-gate ToF pixel as an example, the systems and methods described herein are also applicable to opto-gate ToF pixels and other differential ToF pixel types. The ToF pixel 200 includes "A" circuit 201a and "B" circuit 201b, both of which, when activated, are exposed to a pixel voltage 205 (V). PIX The ToF pixel 200 includes a common drain gate 210 (DG) and a common photodiode 212 (PD).

[0048] Each of circuits 201a and 201b includes passive components, such as source followers 215a and 215b (SF6). A SF B ), floating diffusion capacitors 217a and 217b (FD) A FD B ) and storage diodes 220a and 220b (SD A SD B Each of circuits 201a and 201b includes components such as row select transistors 225a and 225b (SEL). A SEL B ), reset gates 227a and 227b (RG) A RG B ), transmission gates 230a and 230b (TG) A TG B ) and memory gates 232a and 232b (SG) A SG B Actuable elements such as ) are used. Selective operation of each gate under a clock signal determines the location of the charge integration over time within the ToF pixel 200, and when it occurs at acquisition nodes 235a and 235b (V OUT_A V OUT_B Read the accumulated charge at ().

[0049] During normal operation in ToF mode, circuits A 201a and B 201b are sequentially energized, and in some examples, they can alternate at frequencies on the order of hundreds of MHz. All electrons acquired during the period when circuit 201a is energized are read out at output node 235a, and all photons striking the pixel when circuit 201b is energized are read out at output node 235b.

[0050] Figure 3An example timing diagram 300 for operating a ToF pixel array under global shutter operation is depicted. Figure 300 represents an arbitrary number of N rows. In this example, conventional operation is depicted for two different modulation frequencies and phase delay captures. Although this detailed description will primarily use a ToF pixel under global shutter operation as an example, the systems and methods described herein are also applicable to ToF pixels operating under rolling shutter operation.

[0051] In this example, at 302, for the integration time, the A and B circuits of the pixels in rows 1-N are alternately excited using a first modulation frequency (F1) and a first phase delay (Φ1). At 304, rows 1-N are read out sequentially. At 306, for the integration time, the A and B circuits of the pixels in rows 1-N are alternately excited using a first modulation frequency (F1) and a second phase delay (Φ2). At 308, rows 1-N are read out sequentially. At 310, for the integration time, the A and B circuits of the pixels in rows 1-N are alternately excited using a second modulation frequency (F2) and a first phase delay (Φ1). At 312, rows 1-N are read out sequentially. At 314, for the integration time, the A and B circuits of the pixels in rows 1-N are alternately excited using a second modulation frequency (F2) and a second phase delay (Φ2). At 316, rows 1-N are read out sequentially. The combined readout from 304, 308, 312, and 316 is used to generate the depth image 318. Then at 320, the array is idle for a certain period of time.

[0052] At 325, highlight one frame of a line (dashed line). Figure 4 A single-line example timing diagram 400 depicts this TOF pixel operation under global shutter. Diagram 400 will describe the differential pixel 200 and its components. Figure 400 includes graphs 410 and 420, with graph 410 indicating memory gate A (SG). A As time is actuated, Figure 420 indicates storage gate B (SG) B Actuation over time. Figure 400 also includes graph 430, which indicates the actuation of the common drain gate (DG) over time. Figure 400 also includes graph 440, which indicates the actuation of transmission gates A and B (TG). A TG B As time is actuated, Figure 450 indicates the reset gates A and B (RG). A RG B Driven by time, Figure 460 indicates row selectors A and B (SEL). A SEL B The actuation of the signal over time. In some examples, correlated double sampling (CDS) can be used to perform the readout. This can include reading each acquisition node twice, thereby providing an improved signal compared to reading each acquisition point once.

[0053] From time t0 to time t1, the row undergoes a global reset of the photodiodes (PDs) of each pixel via the actuated drain gate, as shown in 430. At time t1, the drain gate closes, integration begins, and extends to time t2. From time t1 to time t2, memory gates A and B are actuated alternately, as shown in 410 and 420. At time t2, the drain gate reopens, and the row awaits its turn to be read. At time t3, the reset gate closes, and the row selector is actuated to initiate readout, as shown in 450 and 460, respectively. Charge transfer from the memory diodes to the floating diffusion is performed by actuating the transfer gate for a duration, as shown in 440. At time t4, the row selector is de-actuated, the reset gate is actuated, and readout for that row ends.

[0054] Figure 5 A flowchart of an example method 500 for manipulating differential time-of-flight pixels is shown. Time-of-flight imaging devices can use method 500 to reduce power consumption, for example, in a progressive wake-up operating mode.

[0055] In 510, method 500 includes operating a first differential pixel of the sensor array in a first lower power mode. The first lower power mode can be specified as a default mode. For example, the first lower power mode may include a lower frequency switching between a portion A and a portion B of the differential pixel during an integration period (e.g., 200 Hz), instead of a higher frequency switching (e.g., 200 MHz) associated with depth imaging.

[0056] Alternatively, the first lower power mode can be a passive mode, where the illumination device typically used to generate ToF data is not active. Instead, the differential pixels can be configured to integrate ambient light present in the environment. Furthermore, the voltages read from sections A and B can be performed without using an ADC, resulting in additional power savings.

[0057] In 520, method 500 includes supplying a first clock signal for a first duration to selectively activate a first acquisition terminal of a first differential pixel. For example, each of sections A and B can be activated or energized for a single duration (e.g., 5 ms), rather than switching back and forth on the order of MHz frequency (e.g., 10 ns).

[0058] In method 500, at 530, the method includes supplying a second clock signal for a second duration to selectively activate a second acquisition terminal of the first differential pixel. In some examples, the second clock signal may be substantially complementary to the first clock signal. The second duration may be equal in length to the first duration, but in some examples it may also be shorter or longer. However, in some examples, the first clock signal and the second clock signal may be configured such that the first acquisition terminal and the second acquisition terminal have overlapping activation periods or intervals of activation periods. (See also: Regarding...) Figure 4 As described, for ToF applications, the clock signal can be switched between sections A and B at a frequency of MHz. Each switch consumes power (CV^2F). Power savings are achieved by reducing the number of cycles, for example, from 1000 cycles to 1-2. Furthermore, the number of images captured can be reduced to 1 instead of the multiple images (e.g., 4-6) required to export accurate depth frames.

[0059] Examples of manipulating pixels in this way are in Figure 6 As shown in the image. Figure 6 A single-line example timing diagram 600 of TOF pixel operation under global shutter is shown. Diagram 600 will describe the differential pixel 200 and its components. Figure 600 includes graphs 610 and 620, with graph 610 indicating memory gate A (SG). A As time is actuated, Figure 620 indicates the storage gate B (SG) B Actuation over time. Figure 600 also includes graph 630, which indicates the actuation of the common drain gate (DG) over time. Figure 600 also includes graph 640, which indicates the actuation of transmission gates A and B (TG). A TG B Actuation over time, Figure 650, indicates the reset gates A and B (RG). A RG B As time progresses, Figure 660 shows the time-dependent actuation of row selectors A and B (SEL). A SEL B ).

[0060] From time t0 to time t1, the row undergoes a global reset of the photodiodes (PDs) of each pixel via the actuated drain gate, as shown in 630. At time t1, the drain gate closes, and integration begins. From time t1 to time t2, memory gate A is actuated, as shown in 610. From time t2 to time t3, memory gate B is actuated, as shown in 620. Figure 4 The timing diagram shown is different; here, each memory gate is open for a single extended duration.

[0061] return Figure 5At 540, method 500 includes reading out a first charge amount accumulated at a first acquisition terminal during a first duration. The first charge amount can be read out in the analog domain without being directed to the ADC. Continuing at 550, method 500 includes reading out a second charge amount accumulated at a second acquisition terminal during a second duration. The first and second charge amounts can be read out in the analog domain without being directed to the ADC. Optionally, the first and second charge amounts can be read out in a manner where a lower ADC bit depth is available for comparison.

[0062] Depending on the pixel configuration, the readout of the first charge accumulated at the first acquisition terminal during the first duration and the second charge accumulated at the second acquisition terminal during the second duration can occur simultaneously or sequentially. For example, returning... Figure 6 At time t3, the drain gate reopens. At time t4, the reset gate closes, and the row selector is actuated to initiate readout, as shown in 650 and 660, respectively. Charge transfer from the storage diode to the floating diffusion is performed by actuating transfer gates A and B for a period of time, as shown in 640. At time t5, the row selector is de-actuated, and the reset gate is actuated, ending readout for that row.

[0063] Return to Figure 5 Once both acquisition terminals have been integrated, method 500 can proceed to 560. In 560, method 500 includes comparing the first charge quantity and the second charge quantity in the analog domain. Figure 7 As shown, motion causes brightness to change over time, which can be determined by comparing the first and second charge amounts. This comparison can then be used to decide whether to push the pixel into a higher power mode.

[0064] Figure 7 Three example graphs, 700, 710, and 720, are shown. Graph 700 depicts a scene with static brightness. Graph 710 depicts a scene where brightness increases over time. Graph 720 depicts a scene where brightness decreases over time (e.g., dims). For a static scene, such as the one represented by graph 700, Sig_A - Sig_B = 0; therefore, abs(Sig_A - Sig_B) = 0. For a scene with increasing brightness, such as the one represented by graph 710, Sig_A - Sig_B < 0; and therefore, abs(Sig_A - Sig_B) > 0. For a scene with decreasing brightness, such as the one represented by graph 720, Sig_A - Sig_B > 0; abs(Sig_A - Sig_B) > 0. The value of abs(Sig_A - Sig_B) can be determined in the analog domain, bypassing the ADC. Any change in brightness within the pixel's field of view will result in an increase in abs(Sig_A-Sig_B).

[0065] return Figure 5 At 570, method 500 includes operating a first differential pixel of the sensor array in a second higher power mode in response to a first charge quantity differing from a second charge quantity by more than a threshold.

[0066] In some examples, the threshold can be based on a first charge and a second charge. In other words, the threshold can be based on the signal level so that fluctuations caused by noise do not cause the threshold to be exceeded. In this way, the threshold can be continuously updated so that it always exceeds the noise level.

[0067] When the signal S at a pixel is low, the noise N in the signal is also small (e.g., shot noise). Thus, a relatively small detection threshold T can be set to be greater than N. However, when the signal S at a pixel is high, the noise N in the signal is also high (e.g., due to shot noise). To avoid false positives, a relatively large detection threshold can be chosen. For example, in daylight, bright images are inherently noisy, and a threshold used for a relatively dark indoor environment may be ineffective.

[0068] Figure 8 An example comparator circuit 800 for determining whether first and second signals differ by a threshold is shown. The comparator circuit includes a first input 810 and a second input 820, the first input being configured to receive data based on a first acquisition terminal (V). OUT_A The second input 820 is configured to receive the signal read from the second acquisition terminal (V11). OUT_A The third input 830 is configured to receive a threshold signal 831, such as a hysteresis signal. The comparator circuit 800 also includes an output 840, which is configured to output a comparison signal 841 based on the result of the threshold comparison.

[0069] As the signal level increases, the noise increases proportionally to the square root of the signal. Therefore, a threshold signal can be set based on the square root of the signal. For example, the threshold signal can be set based on Sqrt(V). OUT_A +V OUT_B )+C, where C is a constant. This allows thresholds to be modulated by hysteresis based on signal values ​​811 and 821 without leaving the analog domain and without measuring these signals at the ADC.

[0070] As long as the threshold T is sufficiently higher than the level of noise N, the threshold T can be set using a threshold circuit that approximates the noise N of the signal S. Figure 9An example differential circuit 900 is schematically depicted. This differential circuit 900 can be used as a comparator circuit whose threshold varies based on the signal level. The differential circuit 900 can be configured to simultaneously detect changes in the signal and apply the signal difference to an automatically determined threshold. The differential circuit 900 includes a first input 910 and a second input 920. The first input 910 is configured to apply a threshold based on a first acquisition terminal (V) across a first resistor 911 (R1). OUT_A The second input 920 is configured to apply a signal read from the second acquisition terminal (V) across the second resistor 921 (R2). OUT_A The signal read out.

[0071] If there is a reasonably large change in V OUT_A -V OUT_B If the voltage across R3 is large, a relatively large voltage will appear across the third resistor 930 (R3). Otherwise, the voltage across R3 will be relatively small. Therefore, the differential circuit 900 can be used to approximate Sqrt(variable V) OUT_A -V OUT_B The threshold change is generated as a function of the common-mode value using a method of )+C. The transistor drive can be increased with increasing voltage, thus causing the third resistor 930 to be overpowered more quickly when the common-mode is high (e.g., a low signal). The output across the third resistor 930 can then be sent to another comparator configured with a fixed threshold to determine whether a sufficient signal change has occurred.

[0072] Although the differential circuit 900 has a large positive gain, in some examples, a certain amount of signal digitization and / or processing can be applied to remove signal polarity. In some examples, this circuit can be cascaded with a non-tunable version of the comparator 800.

[0073] return Figure 5 If the first charge amount is within a threshold of the second charge amount, the first differential pixel can remain in a first lower power mode. If the readout signal has a signal-to-noise ratio below a threshold, the first differential pixel can operate in a higher power mode for a period of time. For example, active illumination can be activated for the pixel region in the sensor array that includes the first differential pixel to obtain higher signal data.

[0074] If a threshold change in the charge accumulated between the first and second acquisition terminals is determined, the first differential pixel is then operated in a second higher power mode. In some examples, the second higher power mode may include an increased switching frequency between the acquisition terminals, some degree of digital processing and / or ADC activation, activation of active illumination via a manipulated illumination source, and / or entry into ToF mode.

[0075] In some examples, the second higher power mode is not a ToF mode. Instead, the second higher power mode can be used to acquire a higher level of information than that available through the first lower power mode. As an example, the second higher power mode could be a photographic mode where the illumination source is not activated, and the switching between the first and second acquisition terminals increases to a level higher than the first lower power mode, but not to the MHz level associated with ToF imaging. Image data can be shunted to an ADC for evaluation. The photographic information acquired in this second higher power mode can then be used to determine whether the differential pixels and / or the entire pixel array should proceed to an additional, higher power mode, such as ToF.

[0076] As an example, charge changes can be detected in two or more pixels within a region of a pixel array. The pixel region can be moved to a second, higher power mode, in which an image of the region is captured and evaluated against a set of criteria. The set of criteria can include parameters indicating whether the image indicates a significant brightness change in the region and / or whether the image indicates an ongoing brightness change in the region. If the image does not meet the set of criteria, the pixel region can return to a first, lower power mode. If the set of criteria is met, the pixel region can advance to a third power mode, which is higher than the second power mode, such as in Time-of-Flight (ToF) imaging. In this way, the power mode can be increased rapidly and incrementally based on signal data. The camera can then respond to actual changes in the scene while carefully examining noise and fluctuations without entering a full-power ToF mode.

[0077] Additional power savings can be achieved in lower power modes by selectively disabling or combining pixels. Figure 10A A pixel array 1000 operating at full resolution is schematically shown. The pixel array 1000 includes a region 1010, depicted as a 9x6 block of 54 pixels, coupled to an ADC block 1012. In this example, each of the 54 pixels within region 1010 is activated. Figure 10BA decimated (e.g., subsampling) pixel array 1050 is schematically illustrated. Pixel array 1055 includes region 1060, depicted as a 9x6 block of 54 pixels, coupled to ADC block 1062. In this example, region 1060 is divided into clusters of 9 pixels, such as cluster 1065. In this example, one of the 9 pixels within cluster 1065 is activated, thus 6 out of the 54 pixels within region 1060 are activated (e.g., by a factor of 1 / 9 decimation). Based on the signal generated by the activated pixels within the pixel array, inactive pixels can be activated, thereby increasing the array's resolution as a first stage of a progressive wake-up process. This process can be applied to regions within the pixel array, rather than to individual pixels or the entire array. This can reduce potentially unwanted pixel activation and progressive decimation through higher power modes. Specifically, the imaging device may include a manipulable illumination source configured to individually and actively illuminate different illumination regions within multiple illumination regions of a scene viewed by the sensor array. The manipulable illumination source may actively illuminate only a specific illumination region corresponding to a pixel array region indicating a change in brightness. Other pixel array areas can remain in passive detection mode.

[0078] Figure 11 An example method 1100 for operating a pixel array comprising multiple regions including differential time-of-flight pixels is described. As a non-limiting example, the pixel array may be divided into 16 regions, but more or fewer regions may be used. In some examples, each of the multiple regions may be equal in size and number of pixels included, while in other examples, these regions may be inconsistent, overlapping, or otherwise configured.

[0079] In 1110, method 1100 includes operating a first region of the pixel array in a reduced power mode. For example, the first region may include more than one active pixel, each active pixel being configured to operate in the first lower power mode, such as regarding... Figure 5 and Figure 6 The description refers to a lower power mode where pixels are operated in a passive low-frequency mode. In some examples, operation in the reduced power mode includes resampling of the region, as described regarding... Figure 10B Therefore, the first region can be considered as being extracted, having one or more non-active pixels. Two or more pixels can be grouped into a cluster, and one or more pixels can be read out to represent that cluster.

[0080] In some examples, operation in a reduced power mode involves combining the outputs of two or more pixels. Each pixel may operate in the described low-frequency, lower-power mode, but the output signals of two or more pixels can be combined for analysis. For example, many pixel arrays include amplifiers configured to accept voltages from multiple (e.g., four) pixels and perform resistor combining. Voltages from multiple first acquisition terminals can be combined at a common node via a first resistor, and voltages from multiple second acquisition terminals can be combined via a second resistor. The combined voltages can then be compared, for example, regarding... Figures 7 to 9 As described above, by reading out multiple pixels simultaneously, the power consumption of reading out a single pixel is maintained.

[0081] At 1120, method 1100 includes increasing the charge readout of a first region of the pixel array above a threshold, and operating a second region of the pixel array in a higher power mode. Thresholding for each pixel or pixel cluster can be as follows: Figure 8 and Figure 9 It is performed as described. In some examples, two or more pixels or clusters of pixels within a region may exhibit threshold changes before the first region of the pixel array is operated in a higher power mode. (See also: Regarding...) Figure 5 The higher power mode described can be camera mode, ToF mode, or other higher power modes.

[0082] In some examples, the higher power mode may include all pixels within a single active region, but the pixels operate in the first lower power mode. For example, such as Figure 10B The extraction region of region 1060 shown can be converted to, for example, Figure 10A The fully activated region 1010 shown is shown. However, each pixel can then operate in a low-frequency passive imaging mode. In some examples, based on the charge readout of a first region of the pixel array increasing above a threshold, the entire pixel array can operate in a higher power mode. For example, exceeding a first threshold can indicate operation of that region in a higher power mode, while exceeding a second higher threshold can indicate operation of the entire array in a higher power mode, even if each other region of the pixel array does not exhibit a comparable level of activity.

[0083] At 1130, method 1100 includes acquiring image data of the environment for a region of the pixel array in a higher power mode. (See also: Regarding...) Figure 5 As described, acquiring image data may include passive camera images of the environment using a region of a pixel array. In some examples, acquiring image data of the environment may include selectively illuminating an area of ​​the environment from an angle of the array region. Thus, the image data acquired for this area may be 2D or 3D image data.

[0084] At 1140, method 1100 includes adjusting the power mode of the pixel array based on the acquired image data. For example, if the image data indicates that the detected brightness change is noisy, transient, or otherwise does not indicate more power-intensive imaging, at least a first region of the pixel array may revert to a reduced power and / or low-resolution mode. If the image data indicates a detected brightness change in at least the first region, a steerable illumination source may be activated for at least the first region. Active imaging data of the actively illuminated region can then be acquired based on the pixels of the first region. In some examples, while other regions of the pixel array are operated in a higher power mode, the steerable illumination source may advance to sequentially illuminate one or more illumination regions.

[0085] Detectable step function changes in brightness can occur within a frame (intra-frame) or between frames (inter-frame). The system should detect both intra-frame and inter-frame changes so that the method described in this paper provides an effective detection scheme. However, detecting brightness changes occurring during readout may be more challenging than during integration.

[0086] Figure 12 An example method 1200 for detecting a step change in brightness is illustrated, for example, during a low-frequency, low-power mode as described herein. At 1210, method 1200 includes operating each active differential pixel of the sensor array in a first low-power mode. At 1220, method 1200 includes supplying a first clock signal for a first integration duration to selectively activate a first acquisition terminal of each active differential pixel. In some examples, these operations may be as described herein and referenced. Figure 5 To execute. For example, Figure 13 Example timing diagram 1300 is shown, illustrating the operation of 10 rows of differential pixels; however, the pixel array can have more or fewer pixel rows. A first clock signal can activate the "A" or first acquisition terminal for each differential pixel. In timing diagram 1300, the first clock signal selectively activates the A terminal during the integration periods of frame 1 and frame 3.

[0087] return Figure 12 At 1230, method 1200 includes, during a first readout duration following a first integration duration, reading out a first amount of charge accumulated at the first acquisition terminal for a first subset of the differential pixel rows during the first integration time, and maintaining the amount of charge accumulated at the first acquisition terminal for a second subset of the differential pixel rows during the first integration duration. For example, timing diagram 1300 shows odd-numbered rows (e.g., 1, 3, 5, 7, and 9) read out during the readout duration portions of frames 1 and 3. Even-numbered rows (e.g., 2, 4, 6, 8, and 10) are not read out during this readout duration. Therefore, when the array enters frame 2, the even-numbered rows retain their charge.

[0088] return Figure 12 In 1240, method 1200 includes supplying a second clock signal for a second integration duration to selectively activate a second acquisition terminal for each active differential pixel, the second clock signal being substantially complementary to the first clock signal. For example, as shown in timing diagram 1300, the second clock signal may activate either "B" or the second acquisition terminal or each differential pixel during the integration period of the second and fourth frames.

[0089] return Figure 12 At 1250, method 1200 includes, during a second readout duration following the second integration duration, reading out a first charge accumulated at the first acquisition terminal during the first integration duration for a second subset of the differential pixel rows, reading out a second charge accumulated at the second acquisition terminal during the second integration duration, and retaining the charge accumulated at the second acquisition terminal during the second accumulation duration for a first subset of the differential pixel rows. For example, timing diagram 1300 shows even-numbered rows (e.g., 2, 4, 6, 8, and 10) being read out during readout duration portions of frames 2 and 4. Odd-numbered rows (e.g., 1, 3, 5, 7, and 9) are not read out during this readout duration. Therefore, when the array enters frame 3, the odd-numbered rows retain their charge.

[0090] return Figure 12 At 1260, method 1200 includes, for each differential pixel, comparing a first charge quantity and a second charge quantity in the analog domain, for example, regarding Figure 8 and Figure 9 As described above. If a step change occurs during the integration duration (as shown in 1310), the step change will be detected in even-numbered rows in subsequent readout cycles. For example, for row 2, the integration at terminal A remains during the first frame until the readout during the second frame. During the integration period for terminal B in the second frame, when both terminal A and terminal B are read out simultaneously during the readout period of the second frame, the step change will cause the readout at terminal B to be higher than the readout at terminal A. If a step change occurs during the readout duration (as shown in 1320), when odd-numbered rows are read out during the third frame, the readout from terminal A for odd-numbered rows will be higher than the readout from terminal B. This assumes that the scene object causing this brightness change is large enough to span at least one odd-numbered row and one even-numbered row.

[0091] return Figure 12 Method 1200 includes, at 1270, for a first differential pixel, operating the first differential pixel of the sensor array in a second higher power mode in response to a first charge amount and a second charge amount differing by more than a threshold.

[0092] In some embodiments, the methods and processes described herein can be attached to a computing system of one or more computing devices. In particular, such methods and processes can be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0093] Figure 14 A non-limiting embodiment of a computing system 1400 that can implement one or more of the methods and processes described above is illustrated schematically. The computing system 1400 is shown in a simplified form. The computing system 1400 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.

[0094] The computing system 1400 includes a logic machine 1410 and a storage machine 1420. The computing system 1400 may optionally include a display subsystem 1430, an input subsystem 1440, a communication subsystem 1450, and / or... Figure 14 Other components not shown.

[0095] The logic machine 1410 includes one or more physical devices configured to execute instructions. For example, the logic machine may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transform the state of one or more components, achieve technical effects, or otherwise obtain desired results.

[0096] A logical machine may include one or more processors configured to execute software instructions. Additionally or alternatively, a logical machine may include one or more hardware or firmware logical machines configured to execute hardware or firmware instructions. The processor of the logical machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logical machine may optionally be distributed across two or more separate devices that can be remotely located and / or configured for coordinated processing. Aspects of the logical machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud computing configuration.

[0097] Storage machine 1420 includes one or more physical devices configured to store instructions executable by a logical machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine 1420 may be changed, for example, to maintain different data.

[0098] Storage machine 1420 may include removable and / or built-in devices. Storage machine 1420 may include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor storage (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic storage (e.g., hard disk drive, floppy disk drive, magnetic tape drive, MRAM, etc.). Storage machine 1420 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices.

[0099] It should be understood that the storage machine 1420 includes one or more physical devices. However, aspects of the instructions described herein may alternatively be propagated by a communication medium (e.g., electromagnetic signals, optical signals, etc.) that is not maintained by the physical device for a finite duration.

[0100] The aspects of logic machine 1410 and memory machine 1420 can be integrated together into one or more hardware logic components. For example, such hardware logic components may include field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), system-on-a-chip (SoCs), and complex programmable logic devices (CPLDs).

[0101] When included, the display subsystem 1430 can be used to present a visual representation of data stored by the storage machine 1420. This visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data held by the storage machine and thus change the state of the storage machine, the state of the display subsystem 1430 can also be transformed to visually represent changes in the underlying data. The display subsystem 1430 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with the logic machine 1410 and / or the storage machine 1420 in a shared enclosure, or such display devices may be peripheral display devices.

[0102] When included, the input subsystem 1440 may include or interface with one or more user input devices, such as a keyboard, mouse, touchscreen, or game controller. In some embodiments, the input subsystem may include or interface with selected Natural User Input (NUI) components. Such components may be integrated or peripheral, and the transduction and / or processing of input actions may be handled on-board or off-board. Example NUI components may include microphones for speech and / or speech recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for assessing brain activity.

[0103] When included, the communication subsystem 1450 can be configured to communicatively couple the computing system 1450 to one or more other computing devices. The communication subsystem 1450 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem may allow the computing system 1400 to send messages to and / or receive messages from other devices via a network such as the Internet.

[0104] In one example, the imaging system includes a sensor array comprising a plurality of differential pixels; and a controller configured to: operate a first differential pixel of the sensor array in a first low-power mode; supply a first clock signal for a first duration to selectively activate a first acquisition terminal of the first differential pixel; supply a second clock signal for a second duration to selectively activate a second acquisition terminal of the first differential pixel; read out a first charge accumulated at the first acquisition terminal during the first duration; read out a second charge accumulated at the second acquisition terminal during the second duration; compare the first charge and the second charge in an analog domain; and operate the first differential pixel of the sensor array in a second high-power mode in response to a difference between the first charge and the second charge exceeding a threshold. In such an example or any other example, the threshold is additionally or alternatively based on the first charge and the second charge. In any of the foregoing examples or any other examples, the first low-power mode additionally or alternatively includes low-frequency switching between activation of the first and second acquisition terminals. In any of the foregoing examples or any other examples, the second high-power mode additionally or alternatively includes time-of-flight imaging. In any of the foregoing examples or any other examples, the second higher power mode additionally or alternatively includes a passive camera mode that includes analog-to-digital conversion of image data acquired at pixels. In any of the foregoing examples or any other examples, the controller is additionally or alternatively further configured to: evaluate the digital image data acquired during the passive camera mode according to a set of criteria; and operate the first differential pixel of the sensor array in a third power mode higher than the second higher power mode based on the digital image data satisfying the set of criteria. In any of the foregoing examples or any other examples, the controller is additionally or alternatively further configured to: return the first differential pixel to the first lower power mode based on the digital image data not satisfying the set of criteria. In any of the foregoing examples or any other examples, the controller is additionally or alternatively configured to operate two or more differential pixels in the second higher power mode based on a difference between a first charge and a second charge exceeding a threshold. In any of the foregoing examples or any other examples, the sensor array is additionally or alternatively divided into a plurality of differential pixel regions, wherein a first differential pixel is additionally or alternatively located in a first differential pixel region, and wherein a controller is additionally or alternatively configured to operate each pixel in the first differential pixel region in a second higher power mode based on a first charge amount differing from a second charge amount by more than a threshold.

[0105] In another example, a method for operating a sensor array comprising a plurality of differential pixels includes: operating a first differential pixel of the sensor array in a first low-power mode; supplying a first clock signal for a first duration to selectively activate a first acquisition terminal of the first differential pixel; supplying a second clock signal for a second duration to selectively activate a second acquisition terminal of the first differential pixel; reading out a first charge amount accumulated at the first acquisition terminal during the first duration; reading out a second charge amount accumulated at the second acquisition terminal during the second duration; comparing the first charge amount and the second charge amount in an analog domain; and operating the first differential pixel of the sensor array in a second high-power mode in response to a difference between the first charge amount and the second charge amount exceeding a threshold. In such an example or any other example, the threshold is additionally or alternatively based on the first charge amount and the second charge amount. In any of the foregoing examples or any other examples, the first low-power mode additionally or alternatively includes low-frequency switching between activation of the first acquisition terminal and the second acquisition terminal. In any of the foregoing examples or any other examples, the second high-power mode additionally or alternatively includes time-of-flight imaging. In any of the foregoing examples or any other examples, the second higher power mode additionally or alternatively includes a passive camera mode that includes analog-to-digital conversion of image data acquired at pixels. In any of the foregoing examples or any other examples, the method additionally or alternatively includes evaluating digital image data acquired during the passive camera mode according to a set of criteria; and operating a first differential pixel of the sensor array in a third power mode higher than the second higher power mode based on the digital image data satisfying the set of criteria. In any of the foregoing examples or any other examples, the method additionally or alternatively includes returning the first differential pixel to a first lower power mode based on the digital image data not satisfying the set of criteria. In any of the foregoing examples or any other examples, the method additionally or alternatively includes operating two or more differential pixels in the second higher power mode based on a difference between a first charge and a second charge exceeding a threshold. In any of the foregoing examples or any other examples, the sensor array is additionally or alternatively divided into a plurality of differential pixel regions, wherein a first differential pixel is additionally or alternatively located in a first differential pixel region, and wherein the method additionally or alternatively includes operating each pixel in the first differential pixel region in a second higher power mode based on a first charge amount and a second charge amount differing by more than a threshold.

[0106] In another example, the imaging system includes a sensor array comprising a plurality of differential pixel rows; and a controller configured to: operate each active differential pixel of the sensor array in a first low-power mode; supply a first clock signal for a first integration duration to selectively activate a first acquisition terminal of each active differential pixel; during a first readout duration following the first integration duration, read out a first charge amount accumulated at the first acquisition terminal during the first integration duration for a first subset of the differential pixel rows, and maintain the charge amount accumulated at the first acquisition terminal during the first integration duration for a second subset of the differential pixel rows; and supply a second clock signal for a second integration duration to selectively activate each active differential pixel. The second acquisition terminal of the differential pixel has a second clock signal substantially complementary to the first clock signal; during a second readout duration following a second integration duration, for a second subset of the differential pixel rows, a first charge accumulated at the first acquisition terminal during the first integration duration is read out, a second charge accumulated at the second acquisition terminal during the second integration duration is read out, and for a first subset of the differential pixel rows, the charge accumulated at the second acquisition terminal is maintained during the second integration duration; for each differential pixel, the first charge and the second charge are compared in the analog domain; and for the first differential pixel, in response to a difference between the first charge and the second charge exceeding a threshold, the first differential pixel of the sensor array is operated in a second higher power mode. In such an example or any other example, in response to a difference between the first charge and the second charge exceeding a threshold, two or more differential pixels of the sensor array are additionally or alternatively operated in a second higher power mode.

[0107] It will be understood that the configurations and / or methods described herein are exemplary in nature, and these particular embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Therefore, the various actions shown and / or described may be performed in the order shown and / or described, in another order, in parallel, or omitted. Similarly, the order of the above processes may be changed.

[0108] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as any and all equivalents thereof.

Claims

1. An imaging system, comprising: A sensor array comprising multiple differential pixels; as well as The controller is configured as follows: The first differential pixel of the sensor array operates in a first lower power mode; A first clock signal is supplied for a first duration to selectively activate the first acquisition terminal of the first differential pixel; A second clock signal is supplied for a second duration to selectively activate the second acquisition terminal of the first differential pixel; Read the amount of first charge accumulated at the first acquisition terminal during the first duration; Read the amount of second charge accumulated at the second acquisition terminal during the second duration; Compare the first charge and the second charge in the simulation domain; as well as In response to the difference between the first charge and the second charge exceeding a threshold, the first differential pixel of the sensor array is operated in a second higher power mode. The threshold is based on the first charge and the second charge.

2. The imaging system of claim 1, wherein the first lower power mode includes low-frequency switching between activation of the first acquisition terminal and the second acquisition terminal.

3. The imaging system of claim 1, wherein the second higher power mode comprises time-of-flight imaging.

4. The imaging system of claim 1, wherein the second higher power mode includes a passive camera mode, the passive camera mode including analog-to-digital conversion of image data acquired at the pixel.

5. The imaging system of claim 4, wherein the controller is further configured to: The digital image data acquired during the passive camera mode was evaluated according to a set of criteria; and Based on the digital image data satisfying the set of criteria, the first differential pixel of the sensor array operates in a third power mode that is higher than the second higher power mode.

6. The imaging system of claim 5, wherein the controller is further configured to: Based on the fact that the digital image data does not meet the set of criteria, the first differential pixel is returned to the first lower power mode.

7. The imaging system of claim 1, further configured to operate two or more differential pixels in the second higher power mode based on the difference between the first charge amount and the second charge amount exceeding the threshold.

8. The imaging system of claim 7, wherein the sensor array is divided into a plurality of differential pixel regions, wherein the first differential pixel is located in the first differential pixel region, and wherein the controller is configured to operate each pixel in the first differential pixel region in the second higher power mode based on the first charge amount and the second charge amount differing from the threshold amount by more than the threshold.

9. A method for operating a sensor array, the sensor array comprising a plurality of differential pixels, the method comprising: The first differential pixel of the sensor array operates in a first lower power mode; A first clock signal is supplied for a first duration to selectively activate the first acquisition terminal of the first differential pixel; A second clock signal is supplied for a second duration to selectively activate the second acquisition terminal of the first differential pixel; Read the amount of first charge accumulated at the first acquisition terminal during the first duration; Read the amount of second charge accumulated at the second acquisition terminal during the second duration; Compare the first charge and the second charge in the simulation domain; as well as In response to the difference between the first charge and the second charge exceeding a threshold, the first differential pixel of the sensor array is operated in a second higher power mode. The threshold is based on the first charge and the second charge.

10. The method of claim 9, wherein the first lower power mode includes low-frequency switching between activation of the first acquisition terminal and the second acquisition terminal.

11. The method of claim 9, wherein the second higher power mode comprises time-of-flight imaging.

12. The method of claim 9, wherein the second higher power mode includes a passive camera mode, the passive camera mode including analog-to-digital conversion of image data acquired at the pixel.

13. The method of claim 12, further comprising: The digital image data acquired during the passive camera mode is evaluated according to a set of criteria. as well as Based on the digital image data satisfying the set of criteria, the first differential pixel of the sensor array operates in a third power mode that is higher than the second higher power mode.

14. The method of claim 13, further comprising: Based on the fact that the digital image data does not meet the set of criteria, the first differential pixel is returned to the first lower power mode.

15. The method of claim 9, further comprising: Based on the fact that the difference between the first charge and the second charge exceeds the threshold, two or more differential pixels are operated in the second higher power mode.

16. The method of claim 15, wherein the sensor array is divided into a plurality of differential pixel regions, wherein the first differential pixel is located in the first differential pixel region, and wherein the method further comprises: Based on the fact that the difference between the first charge and the second charge exceeds the threshold, each pixel in the first differential pixel region is operated in the second higher power mode.

17. An imaging system, comprising: The sensor array comprises multiple differential pixel rows; as well as The controller is configured as follows: Each active differential pixel of the sensor array operates in a first lower power mode; A first clock signal is supplied for the first integration duration to selectively activate the first acquisition terminal of each active differential pixel; During a first readout duration following the first integration duration, for a first subset of the differential pixel rows, the first charge accumulated at the first acquisition terminal during the first integration duration is read out, and for a second subset of the differential pixel rows, the charge accumulated at the first acquisition terminal during the first integration duration is maintained. A second clock signal is supplied for the second integration duration to selectively activate the second acquisition terminal of each active differential pixel, and the second clock signal is substantially complementary to the first clock signal; During a second readout duration following the second integration duration, for the second subset of the differential pixel rows, the first charge accumulated at the first acquisition terminal during the first integration duration is read out, the second charge accumulated at the second acquisition terminal during the second integration duration is read out, and for the first subset of the differential pixel rows, the charge accumulated at the second acquisition terminal is maintained during the second integration duration. For each differential pixel, the first charge and the second charge are compared in the analog domain; as well as For the first differential pixel, in response to the difference between the first charge and the second charge exceeding a threshold, the first differential pixel of the sensor array is operated in a second higher power mode.

18. The imaging system of claim 17, wherein in response to the difference between the first charge and the second charge exceeding the threshold, two or more differential pixels of the sensor array are operated in the second higher power mode.

Images