Distance measurement sensor having multiple memory nodes

The CMOS imager with multiple memory nodes optimizes pixel exposure and readout to enhance dynamic range and signal-to-noise ratio, addressing the challenges of wide dynamic range in indirect time-of-flight distance measurements.

JP7764411B2Active Publication Date: 2025-11-05TELEDYNE E2V SEMICON SAS
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2022578630
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-07-22
Filing Date
2021-07-21
Publication Date
2025-11-05
Estimated Expiration
2041-07-21

AI Technical Summary

Technical Problem

Existing CMOS imagers face challenges in maximizing dynamic range for indirect time-of-flight distance measurements due to the wide dynamic range of observed scenes, which affects signal-to-noise ratio and measurement repeatability, and current methods to increase dynamic range involve compromises that complicate data management and sensitivity.

Method used

A CMOS imager architecture with multiple memory nodes that optimizes exposure for each pixel, performs full-resolution conversion for one or two images, and uses non-destructive fast read operations to suppress exposure based on voltage criteria, ensuring maximum signal-to-noise ratio and linear response without saturation.

Benefits of technology

The solution allows for high dynamic range imaging with improved signal-to-noise ratio, reduced data transmission, and enhanced measurement repeatability by individually controlling pixel exposure and readout, optimizing the CMOS imager for indirect time-of-flight distance measurements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007764411000002
    Figure 0007764411000002
  • Figure 0007764411000003
    Figure 0007764411000003
  • Figure 0007764411000004
    Figure 0007764411000004
Patent Text Reader

Abstract

The CMOS image sensor (102) A set of Nk pixels with a particular global shutter architecture, where for each pixel Pix#k (k varies from 1 to N), various memory nodes MN k,m A set of Nk pixels, including a device (104) for simultaneously blocking exposure, common to all pixels (m varying from 1 to M); - M memory nodes MN for each pixel Pix#k k,m a subsystem for non-destructive high-speed readout of the pixels (m varies from 1 to M), which makes it possible to control the criteria for each pixel; a decision mechanism that allows controlling the devices that simultaneously block exposure based on the calculation of a criterion, located at pixel level, that is, the criterion is stored in the various memory nodes MN k,m A determination mechanism defined based on the results obtained by a non-destructive high-speed read operation at (m varies from 1 to M), - Conventional full-resolution image readout subsystem with analog-to-digital conversion (62) Includes:
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to a multi-node CMOS ("Complementary Metal Oxide Semiconductor") active pixel image sensor and a method for implementing said image sensor for indirect time-of-flight (IToF) distance measurement.

[0002] The invention also relates to a device for indirect time-of-flight IToF distance measurement in a scene of two-dimensional or three-dimensional objects and a method for implementing said IToF distance measurement device. [Background technology]

[0003] The present invention is concerned with problems associated with implementing a CMOS imager for measuring the distance between the imager system and various objects using what is called the "indirect time-of-flight" technique.

[0004] IToF (indirect time-of-flight) technology is applied to CMOS sensors and requires measuring the light information reflected by a target object or target at different phase offsets. This light information is emitted from a light source that is precisely synchronized with the image sensor. By combining the various phase offset samples obtained from one or more images, an algorithm can be used to determine the distance of the target object.

[0005] The measured distance does not depend on the received signal level, but on the signal deviation measured between multiple phase-offset samples taken under the same conditions (i.e., over the same integration time). However, the resulting signal-to-noise ratio depends on the measured signal level. To obtain the best accuracy for this distance measurement, it is necessary to maximize the signal-to-noise ratio on the phase-offset samples while preserving their respective deviations. However, this is difficult to achieve in a single image (i.e., one integration time) due to the very large dynamic range of the observed scene.

[0006] The minimum and maximum distance range between the object being observed and the measurement system, together with the reflectivity of the various materials that make up the target subject, contribute to the wide dynamic range required to properly image the scene being photographed in this type of application. Specifically, the light source of an IToF system is sized in terms of luminous power according to the maximum distance to be measured and the reflectivity of the target object.

[0007] Increasing the dynamic range that can be addressed by the image sensor is therefore essential to obtain the best repeatability (i.e., "precision") of indirect time-of-flight IToF distance measurements, but it should be noted that measurement repeatability should be distinguished from measurement accuracy.

[0008] To increase the dynamic range of a CMOS imager, the first type of method allows combining multiple images taken with different exposure times. However, obtaining each of these images requires a full-resolution analog-to-digital conversion. This first type of method has a number of drawbacks, namely: - Analog-to-digital data conversion at full resolution is a time-consuming task, With a large number of images captured, the amount of data that must be managed can become prohibitive for systems involved in utilizing the data from the imager. - When objects move, spatial coherence artifacts can occur This has the following drawbacks.

[0009] To increase the dynamic range of a CMOS imager, the second type of method involves increasing the electronic capacitance of the memory nodes at the pixel level. However, this quickly reaches a physical limit depending on the number of memory nodes and the pixel size. In addition, increasing the capacitance of the memory nodes can cause problems with the overall sensitivity of the CMOS imager and its readout noise. Therefore, the CMOS imager requires more light to obtain the same signal at the output.

[0010] To increase the dynamic range of a CMOS imager, a third type of method can combine multiple exposure time information at the CMOS sensor level and perform compression to preserve resolution at the imager output. This third type of method has the disadvantage that the resulting response is nonlinear, which complicates the use of the data at the system level.

[0011] In general, the problem of maximizing the dynamic range accessible to a CMOS imager requires various compromises between: the pitch of the pixel and the storage capacity of one or more memory nodes of the pixel, adapted to the overall sensitivity of the pixel; - the amount of data transmitted by the imager; - the maximum acquisition frequency of the imager, and - Linearity of the supplied response.

[0012] Applying this problem to the field of indirect time-of-flight IToF, the inherent dynamic range of the observed scene is then widened by the additional optical information required for the measurement, and the repeatability of indirect time-of-flight IToF measurements depends directly on the signal-to-noise ratio of the captured phase information.

[0013] The ideal solution is - maximizing the signal-to-noise ratio of each of the various memory nodes of the pixel (without saturation); - preserving good sensitivity at pixel level by limiting the storage capacity of the memory node at a given pixel pitch; - limiting the amount of data transmitted by the imager as much as possible; - Maximizing acquisition speed by reducing the number of (full resolution) data conversions and exposure times; - No compression to preserve linear response It should be possible to achieve all of the above simultaneously.

[0014] Although the document US2016 / 182847 teaches the principle of performing non-destructive measurements on the memory nodes of a matrix image sensor in order to suppress exposure before saturation is reached, it does not teach how to implement such a principle in practice without too much complexity. [Prior art documents] [Patent documents]

[0015] [Patent Document 1] US Patent Application Publication No. 2016 / 182847 Summary of the Invention [Problem to be solved by the invention]

[0016] The technical problem addressed by the present invention is to find a CMOS imager architecture suitable for distance measurement by the "indirect time-of-flight" technique, which cumulatively meets the requirement to maximize the dynamic range accessible to said CMOS imager. [Means for solving the problem]

[0017] The basic concept of the present invention is based on a CMOS imager architecture that optimizes the exposure for each pixel individually, maximizes the signal-to-noise ratio for each pixel, and performs full resolution conversion and transmission for only one image in the case of image acquisition without external correlated double sampling (CDS) or two images in the case of image acquisition with external correlated double sampling CDS.

[0018] To this end, one subject of the present invention is a CMOS active pixel image sensor with multiple memory nodes for indirect time-of-flight IToF distance measurements in a scene of reflective objects, which comprises: - each of which has a light receiving region PhDk and an integer M of 2 or more memory nodes MN k,mA set of Nk CMOS active pixels Pix#k (k varies from 1 to Nk), each of which is an integer equal to or greater than 1, where m varies from 1 to M; - M memory nodes MN for each pixel Pix#k (k varies from 1 to Nk) k,m an image readout subsystem for generating a pixel exposure image by a light receiving area PhDk (k varies from 1 to Nk) during a pixel exposure image duration equal to or less than a predetermined maximum exposure image duration Tmax_exp, and storing the pixel exposure image by M memory nodes MN k,m an image readout subsystem for determining, through analog-to-digital conversion, N×M digital values ​​representing the amount of charge integrated by an image pre-processing subsystem configured to apply digital processing operations to the digital values ​​provided at output by the image readout subsystem and to transmit pre-processed pixel-level image data to an application system that determines an indirect time-of-flight (IToF) of the observed scene; Includes:

[0019] The image sensor is - Nk × M memory nodes MN k,m and a subsystem for controlling a mechanism for suppressing exposure of Nk pixels Pix#k (k varies from 1 to Nk), During the current cycle Cycle_acq#k,s in which images are acquired from Nk pixels of the sensor, a predetermined integer Nr times of non-destructive fast read operations FCr (r varies from 0 to Nr-1) are performed on M memory nodes MN k,m (m varies from 1 to M) and is configured to determine a signal to suppress exposure of pixel Pix#k based on a criterion as to whether exposure of pixel Pix#k should continue or not after each fast read operation FCr, wherein the criterion is k,m a subsystem that depends on a voltage value non-destructively read at - in each pixel Pix#k, at least one transistor for inhibiting the exposure of the pixel and a memory point that can be charged or discharged by said signal for inhibiting pixel Pix#k in order to activate or deactivate the mechanism for inhibiting the pixel by driving said one or more exposure inhibition transistors; The present invention is characterized by comprising:

[0020] According to some specific embodiments, a CMOS active pixel image sensor with multiple memory nodes for indirect time-of-flight (IToF) distance measurement includes one or more of the following features: Each pixel Pix#k (k varies from 1 to Nk) is assigned to its M memory nodes MN k,m (m varies from 1 to M) is configured to integrate the charges generated by the light-receiving region PhDk during r basic integration intervals INTr (r varies from 0 to Nr-1), and following each basic integration interval INTr, M memory nodes MN k,m The high-speed reading operation FCr and the suppression criteria for The r integration intervals INTr distributed within the current cycle Cycle_acq#s for acquiring pixel images each have a basic duration Tr, so that the sum of the basic integration durations Tr (r varies from 0 to Nr-1) is equal to a predetermined maximum exposure image duration Tmax_exp; the basic durations Tr (r varies from 0 to Nr) are equal to the same value or at least two basic durations obtained from the set of values ​​Tr (r varies from 0 to Nr-1) are different, and the variance of the values ​​Tr (r varies from 0 to Nr-1) is selected so as to maximize the dynamic range of the sensor; The image sensor assigns commands for Nk pixels (k varies from 1 to Nk) to Nk×M memory nodes MN k,mand a subsystem for controlling a mechanism for suppressing exposure of Nk pixels Pix#k; a sequencer for activating the sequence by sending a signal to an image pre-processing subsystem; *For each pixel Pix#k (k varies from 1 to Nk), a light receiving area PhDk and M memory nodes MN k,m (m varies from 1 to M), a phase (Reset or Rst) of resetting the M memory nodes, a phase (FCr) of non-destructive fast readout of M memory nodes, a phase (exp0) of initially exposing pixel Pix#k, followed by one or more phases (Exp) of a first number Nr1(k,s) of holding the exposure of pixel Pix#k, and then one or more phases (Blocked) of a number Nr2(k,s) of suppressing the exposure of pixel Pix#k (where the sum of numbers Nr1(k,s) and Nr2(k,s) is equal to Nr-1), and a phase (Read) of reading the image from pixel Pix#k. Including, *Photosensitive region PhDk and M memory nodes MN k,m A reset phase (Reset or Rst) (where k varies from 1 to Nk and m varies from 1 to M) is performed simultaneously with each acquisition of an image s in "global shutter" mode of Nk pixels of the image sensor, For each pixel Pix#k (k varies from 1 to Nk), the image reading operation is performed using a light receiving area PhDk and M memory nodes MN k,m (m varies from 1 to M), and the image reading operation is performed after the reset phase, and is used to execute the external CDS; Each pixel Pix#k (k varies between 1 and Nk) is constructed with the same architecture and comprises a photosensitive area PhDk, a first transistor T1k that resets the photosensitive area PhDk at the beginning of each cycle of image acquisition and at the beginning or end of each microintegration cycle, the first transistor T1k being controlled by a global reset signal RPhk common to the Nk pixels of the set of pixels, and M storage channels VS for the charges generated by the photosensitive area PhDk and integrated in parallel during the exposure duration of the pixel Pix#k corresponding to the current cycle of image acquisition Cycle_acq#k,s. k,m M memory nodes MN for shared storage on k,m (m varies from 1 to M), a device for simultaneously blocking the exposure of pixel Pix#k common to various M memory nodes Mk,m (m varies from 1 to M), a subsystem for controlling the mechanism for fast readout of Nk×M memory nodes by pixel Pix#k and suppressing Nk pixels, and M independent read access conductors COL to an image readout subsystem for the M memory nodes of pixel Pix#k. k,m (m varies from 1 to M) and the suppression command access channel COL k,IM and a device for activating access via the - for each pixel Pix#k (k varies from 1 to Nk), each storage channel VS that stores the charge generated by the photosensitive area PhDk; k,m (m varies from 1 to M) is the memory node MN k,m configured to drain the charge generated during the exposure image duration of the pixel Pix#k, and k,m Charge transfer transistor T2 controlled by k,m and memory node MN at the beginning of the current cycle Cycle_acq#k,s of the image. k,m a transistor T3 configured to reset the potential of the pixel 100 and controlled by a reset signal RST common to the set of Nk pixels; k,m and, - connected in a follower configuration and connected from its gate to its source to a memory node MN k,m a memory node MN configured to return a potential level of k,m Read transistor T4 k,m and memory node MN k,m Read transistor T4 k,m and in read mode, the access conductor COL is connected to the source of the pixel Pix#k via the image readout subsystem for the M memory nodes of the pixel Pix#k. k,m memory node MN k,m a control signal SEL configured to return the charge potential of the pixel Pix#k and controlling access to the pixel Pix#k; k Select transistor T5 controlled by k,m and Includes For each pixel Pix#k (k varies from 1 to Nk), various M memory nodes MN k,m The device for simultaneously blocking the exposure of pixel Pix#k, which is common to all pixels Pix#k (m varies from 1 to M), is a control signal TRA for transferring the charge generated during the exposure image duration of pixel Pix#k. k,m (m varies from 1 to M) is a charge transfer transistor T2 that is activated when the subsystem for controlling the mechanism for high-speed readout of Nk×M memory nodes and suppressing Nk pixels decides not to continue exposing the pixel according to the value read in the previous non-destructive high-speed readout operation, and is maintained until the pixel is completely read out. k,m Each storage channel is configured to be modulated by a common modulation signal to suppress charge transfer due to the Vs. k,m (m varies from 1 to M), the charge transfer control signal TRA is used to store the charge generated by the light receiving region PhDk. k,m Transistor TI for suppressing k,m Including, For each pixel Pix#k (k varies from 1 to Nk), various M memory nodes MN k,mThe device for simultaneously blocking the exposure of pixel Pix#k, which is common to all pixel Pix#k (m varies from 1 to M), is a device for simultaneously blocking the exposure of pixel Pix#k by M charge transfer transistors T2 k,m a transistor TI for suppressing the discharge of charges generated by the light-receiving region PhDk to the common node, the transistor TI being connected in parallel with the transistor T1k for resetting the light-receiving region PhDk; k,m Including, Nk pixels (k varying from 1 to Nk) are arranged in a matrix form 54 with Nl rows, an integer greater than or equal to 2, and Nc columns, an integer greater than or equal to 2, where Nk is equal to the product of Nc and Nl, and the image sensor is configured such that, row by row and simultaneously over a given time frame, the pixels of the same row l are read out from the full resolution image readout subsystem and the M memory nodes MN k,m and control of the mechanism for suppressing exposure of Nc pixels in the same row l, and thus pooling the processing resources of the full resolution image readout subsystem and the subsystem for fast readout and control of the mechanism for suppressing exposure of pixels column by column.

[0021] Another subject of the invention is an apparatus for indirect time-of-flight IToF distance measurement in a two-dimensional or three-dimensional object scene, comprising an emitter light source emitting a series of light pulses, a CMOS active pixel image sensor having a plurality of memory nodes as defined above for converting the light pulses reflected by objects in the scene into pixel data characterizing one or more images, and a block for processing the pixel data from the one or more images to determine the depth of field of objects in the scene as seen from the location of the emitter light source and the image sensor.

[0022] Another subject of the invention is a method for implementing a CMOS active pixel image sensor with multiple memory nodes for indirect time-of-flight (IToF) distance measurements in an observed scene, comprising: The image sensor has a light receiving region PhDk and an integer M (greater than or equal to 2) of memory nodes MN k,m A set of Nk CMOS active pixels Pix#k (k varies from 1 to Nk), each of which is an integer greater than or equal to 1, and Nk×M memory nodes MN k,m a subsystem for controlling a mechanism for suppressing exposure of Nk pixels Pix#k (where k varies from 1 to Nk) and M memory nodes MN k,m and an image pre-processing subsystem for the image sensor, Nk × M memory nodes MN k,m and a subsystem for controlling a mechanism for suppressing exposure of Nk pixels Pix#k.

[0023] The image sensor is implemented during image acquisition by an image readout subsystem that stores M memory nodes MN for each pixel Pix#k (k varies from 1 to Nk). k,m, generated by the light-receiving regions PhDk (k varies from 1 to Nk) during a pixel exposure image duration equal to or less than a predetermined maximum exposure image duration Tmax_exp, and k,m and providing an M×Nk readout digital value at an output; and an image pre-processing subsystem applying digital processing operations to said digital values ​​provided at an output by the image readout subsystem and transmitting pre-processed pixel-level image data to an application system which determines an indirect time-of-flight (IToF) of the observed scene.

[0024] The image sensor is implemented using Nk × M memory nodes, MN k,m and a subsystem for controlling a mechanism for suppressing exposure of Nk pixels Pix#k, which performs a predetermined integer Nr times of non-destructive high-speed readout operations FCr (r varies from 0 to Nr-1) on M memory nodes MN during a current cycle Cycle_acq#k,s for acquiring an image s in each of the Nk pixels Pix#k (k varies from 1 to Nk). k,m (m varies from 1 to M), and after each fast read operation FCr, a signal for suppressing exposure of pixel Pix#k is determined based on a criterion for whether or not to continue exposure of pixel Pix#k, the criterion being: k,m The decision is based on a voltage value non-destructively read from the sensor and one or more decision thresholds.

[0025] The invention will be better understood on reading the following description of some embodiments, which description is given by way of example only and relates to the accompanying drawings, in which: [Brief explanation of the drawings]

[0026] [Figure 1]FIG. 1 illustrates the general operating principle of the IToF technology applied to a CMOS image sensor according to the present invention. [Figure 2] 1 is a diagram of an example of a two-phase algorithm for determining the distance separating a target object or target in a two-dimensional or three-dimensional scene under observation from a CMOS image sensor according to the present invention. [Figure 3] 1 is a diagram of a CMOS image sensor formed of Nk pixels arranged in a predetermined matrix embodiment of columns and rows, which embodiment generally applies to CMOS image sensors formed from a collection of Nk pixels according to the present invention. [Figure 4] 4 is a diagram of the image sensor of FIG. 3, and more generally of a pixel and its peripheral circuitry incorporated in an image sensor according to a first embodiment of the invention; [Figure 5] 4 is a flowchart of a method for implementing the inventive CMOS image sensor of FIG. 3. [Figure 6] FIG. 5 is a diagram of an example of principle ordering at the pixel level of the CMOS image sensor of FIG. 4. [Figure 7] FIG. 1 is a diagram of the sequence for acquiring a complete image at the level of one pixel. [Figure 8] 1 is a diagram of the operating principle of the criteria incorporated in each pixel for non-destructive high-speed readout operation and their effect on the mechanism for suppressing exposure. [Figure 9] 1A and 1B are comparative diagrams of the changes in frame and data flow as a function of multiple exposure time configurations required to provide good coverage of the scene dynamic range (i.e., HDR ("High Dynamic Range") management) for a conventional image sensor and a CMOS image sensor according to the present invention; [Figure 10] 1 is a comparative diagram of the change in frame and data flow according to the high-speed conversion duration in the case of a conventional image sensor and a CMOS image sensor according to the present invention; [Figure 11]FIG. 4 is a diagram of a second embodiment of a pixel and its peripheral circuitry to be incorporated in the image sensor of FIG. 3, and more generally in an image sensor according to the present invention, in which the suppression mechanism drives the reset of the light-receiving area of ​​the pixel Im rather than the transfer transistor. DETAILED DESCRIPTION OF THE INVENTION

[0027] FIG. 1 illustrates the basic principle of indirect time-of-flight distance measurements on two-dimensional or three-dimensional objects performed by a measuring device according to the invention.

[0028] The measurement device 2 according to the invention is configured to measure the indirect time-of-flight (IToF) distance of an object or target forming part of an observed two-dimensional or three-dimensional scene 4. In Figure 1, only one object 6 is shown to simplify the illustration.

[0029] The measuring device 2 according to the invention comprises a light source 8, an image sensor 12 according to the invention and an IToF processing block 14 for processing pixel image data.

[0030] The light source 8 is arranged to illuminate the scene 4 to be viewed and to emit a series or train of light pulses.

[0031] The image sensor 12 according to the present invention is a CMOS active pixel image sensor having multiple nodes configured to convert light pulses reflected by objects in a scene, in particular object 6, into pixel data that characterize one or more images.

[0032] The IToF processing block for processing pixel image data provided by the image sensor 12 is configured to determine the depth of field of objects in the scene as seen from the location of the light source 8 and the image sensor 12 according to the present invention.

[0033] The basic principle of IToF technology applied to CMOS sensors is based on measuring the light information reflected by a target object 6 or object at various phase offsets. This light information is generated by a light source 8 synchronized with an image sensor 12 with high precision. By combining the various phase offset samples, it is possible to determine the distance d of the target object through the use of an algorithm. The light pulse train emitted by the light source 8 is characterized by a unit pulse duration, denoted "T_pulse", and a pulse repetition duty cycle. As a variant, the light information can be generated based on a sinusoidal phase modulation of the light.

[0034] 2, for example, a two-phase algorithm is illustrated for determining the distance d separating a target object or target 6 from a CMOS image sensor 12 according to the present invention. The distance separating a reflective object under observation from the sensor is designated d and is given by:

number

[0035] In general, the architecture of the image sensor according to the present invention is a synchronous architecture, which makes it possible to capture precisely exposed images of a scene with a very high dynamic range by using a device that controls the exposure individually for each of the pixels, and each pixel contains multiple memory nodes, i.e., a number greater than or equal to 2. This architecture is particularly suitable for use with "indirect time-of-flight" techniques, as it allows for maximum repeatability of measurements for each pixel.

[0036] 3, the CMOS image sensor 52 according to the present invention includes a set of CMOS active pixels Pix#k (k varies from 1 to Nk) of 1 or more, each having a light receiving region PhDk formed for example by a photodiode, and a set of memory nodes MN k,m (m varies from 1 to M).

[0037] 3, in particular, Nk pixels (k varying from 1 to Nk) are arranged in the form of a matrix 54 with Nl rows 56, an integer greater than or equal to 2, and Nc columns 58, an integer greater than or equal to 2, where Nk is equal to the product of Nc and Nl. Each pixel Pix(c,l) of the matrix is ​​referenced within the matrix and uniquely identified by a pair of indices (c,l), where c and l respectively indicate the index c of the column and the index l of the row to which the pixel Pix(c,l) belongs.

[0038] A correspondence F between two unique items is defined between, on the one hand, a set of index pairs (c,l) where the column index c ranges from 1 to Nc and the row index ranges from 1 to Nl, and, on the other hand, a set of integers from 1 to Nk, and each pixel Pix(c,l) can be denoted as Pix#F(x,l) or Pix#k (k equals F(c,l)) according to this correspondence.

[0039] Each CMOS active pixel Pix#k (k varies from 1 to Nk) has a light receiving area PhDk and a memory node MN k,m (m varies from 1 to M).

[0040] The CMOS image sensor 52 is - M memory nodes MN for each pixel Pix#k (k varies from 1 to Nk) k,m a full-resolution image readout subsystem 62 for generating, through analog-to-digital conversion, pixel exposure image durations less than or equal to a predetermined maximum exposure image duration Tmax_exp by light-receiving regions PhDk (k varies from 1 to Nk) and storing the image data in M ​​memory nodes MN k,m a full resolution image readout subsystem 62 for determining N×M digital values ​​representing the amount of charge integrated by an image pre-processing subsystem 64 configured to apply digital processing operations to the digital values ​​provided at the output by the image readout subsystem 62 to transmit pre-processed pixel-level image data to an application system that determines the indirect time-of-flight (IToF) of the observed scene; Includes:

[0041] The CMOS image sensor 52 is - Nk × M memory nodes MN k,m and a subsystem 66 for controlling the suppression of exposure of Nk pixels Pix#k For each of Nk pixels Pix#k (k varies from 1 to Nk), during a current cycle Cycle_acq#k,s of acquiring an image from the Nk pixels of the sensor, a predetermined integer Nr times of non-destructive fast read operations FCr (r varies from 0 to Nr-1) are performed on M memory nodes MN k,m (m varies from 1 to M) and is configured to identify a signal to suppress exposure of pixel Pix#k based on a criterion for whether to continue exposure of pixel Pix#k after each fast read operation FCr, wherein the criterion is k,m It depends on the voltage value read non-destructively from

[0042] The CMOS image sensor 52, particularly in the case of the row / column matrix architecture of FIG. 3, allows pixels in the same row to be read by the full resolution image readout subsystem 62 and the Nk×M memory nodes MN during a given time frame. k,m and a row decoder 68 having exclusive access to the processing resources of a subsystem 66 for controlling the fast and non-destructive readout of N pixels in the same row, thereby enabling the full resolution image readout subsystem 62 and the subsystem 66 for controlling the fast readout and suppression of exposure of pixels to be pooled.

[0043] The CMOS image sensor 52 performs a phase in the current cycle Cycle_acq#s by transmitting commands for Nk pixels (k varies from 1 to Nk) to Nk×M memory nodes MN k,m and the suppression of exposure of Nk pixels Pix#k to the image pre-processing subsystem and in particular for the architecture of FIG.

[0044] In general, a CMOS active pixel image sensor according to the present invention having multiple memory nodes for indirect time-of-flight IToF distance measurement of reflective objects in a scene of interest comprises: - each of which has a light receiving region PhDk and an integer M of 2 or more memory nodes MN k,m A set of Nk CMOS active pixels Pix#k (k varies from 1 to Nk), each of which is an integer equal to or greater than 1, where m varies from 1 to M; - M memory nodes MN for each pixel Pix#k (k varies from 1 to Nk) k,m an image readout subsystem for generating, through analog-to-digital conversion, pixel exposure image durations that are shorter than or equal to a predetermined maximum exposure image duration Tmax_exp by light-receiving areas PhDk (k varies from 1 to Nk), and M memory nodes MN k,man image readout subsystem for determining N×M digital values ​​representing the amount of charge integrated by an image pre-processing subsystem configured to apply digital processing operations to the digital values ​​provided at the output by the image readout subsystem and to transmit the pre-processed pixel-level image data to an application system that determines an indirect time-of-flight (IToF) of the observed scene; Includes:

[0045] In general, an image sensor is - Nk × M memory nodes MN k,m and suppression of exposure of Nk pixels Pix#k, wherein for each of the Nk pixels Pix#k (k varies from 1 to Nk), a predetermined integer Nr number of non-destructive fast read operations FCr (r varies from 0 to Nr-1) are performed on M memory nodes MN k,m (m varies from 1 to M) and after each fast read operation FCr, a subsystem configured to determine a signal to suppress exposure of pixel Pix#k based on a criterion as to whether to continue exposure of pixel Pix#k, wherein the criterion is that after the fast read operation FCr, M memory nodes MN k,m Depends on the voltage value read non-destructively from It is characterized by:

[0046] Each pixel Pix#k (k varies from 1 to Nk) is connected to its M memory nodes MN k,m (m varies from 1 to M) is configured to integrate the charges generated by the light-receiving regions PhDk during r basic integration intervals INTr (r varies from 0 to Nr-1), and after each basic integration interval INTr, M memory nodes MN k,mThe high-speed read operation FCr and the re-evaluation of the suppression criteria (except for the final integration phase Nr-1) continue. The Nr integration intervals INTr distributed within the current cycle Cycle_acq#s for acquiring pixel images each have a basic duration Tr, so that the sum of the basic integration durations Tr (r varies from 0 to Nr-1) is equal to the predetermined maximum exposure image duration Tmax_exp.

[0047] The basic duration Tr (r is 0 to Nr) may be equal to the same value.

[0048] In a variant, at least two basic durations extracted from the set of values ​​Tr (r varies from 0 to Nr-1) are different, and the variance of the values ​​Tr (r varies from 0 to Nr-1) is selected to maximize the dynamic range of the sensor.

[0049] In general, the CMOS image sensor according to the present invention assigns a phase of a set of phases to be executed during a current cycle Cycle_acq#s to Nk pixels (k varies from 1 to Nk) and outputs commands to Nk×M memory nodes MN k,m and a subsystem for controlling the fast readout of Nk pixels Pix#k and the suppression of exposure of Nk pixels Pix#k, and a sequencer for activating the set of phases by sending a signal to the image pre-processing subsystem, For each pixel Pix#k (k varies from 1 to Nk), a light receiving area PhDk and M memory nodes MN k,m(m varies from 1 to M) a phase (Reset or Rst) of resetting the M memory nodes, a phase (FCr) of non-destructive fast readout of the M memory nodes, a phase (exp0) of initially exposing the pixel Pix#k, followed by one or more phases (Exp) of a first number Nr1(k,s) of phases that hold the exposure of the pixel Pix#k and one or more phases (Blocked) of a number Nr2(k,s) of phases that subsequently block the exposure of the pixel Pix#k (where the sum of the numbers Nr1(k,s) and Nr2(k,s) is equal to Nr-1), and a phase (Read) of reading the image from the pixel Px#k. Including, - light receiving regions PhDk and M memory nodes MN k,m A reset phase (Reset or Rst) (where k varies from 1 to Nk and m varies from 1 to M) is performed simultaneously with each acquisition of an image s in "global shutter" mode of the Nk pixels of the image sensor.

[0050] According to a first embodiment of the image sensors of Figures 4 and 3, and more generally of the pixel Pix#k and its peripheral circuits incorporated in an image sensor according to the invention, each pixel Pix#k (k varying from 1 to Nk) is incorporated into an image sensor architecture based on an analog-to-digital converter for full resolution image readout of a column, and each pixel Pix#k (k varying from 1 to Nk) has the same number M of memory nodes.

[0051] Each pixel Pix#k (where k varies from 1 to Nk) (one pixel 102 is shown generically in FIG. 4) is constructed according to the same overall architecture, - Photosensitive area PhDk, a transistor T1k for resetting the light-receiving area PhDk at the beginning of each cycle of image acquisition and at the beginning or end of each microintegration cycle or interval (i.e. at the beginning or end of each elementary integration interval INTr, with r varying from 0 to Nr-1), the transistor T1k being controlled by a global reset signal RPhk common to the Nk pixels of the set of pixels; M storage channels VS of the charges generated by the photosensitive area PhDk and integrated in parallel during the exposure duration of the pixel Pix#k corresponding to the current cycle Cycle_acq#k,s of image acquisition; k,m M memory nodes MN for shared storage on k,m (m varies from 1 to M) a device 104 for simultaneously blocking the exposure of pixels Pix#k common to various M memory nodes Mk,m (m varying from 1 to M); - a subsystem for controlling the fast readout of the Nk×M memory nodes by the pixel Pix#k and the exposure suppression of the Nk pixels, and M independent read access conductors COL to an image readout subsystem for the M memory nodes of the pixel Pix#k; k,m (m varies from 1 to M) and the suppression command access channel COL k,IM Device 106 for activating access via Includes:

[0052] Transistor T1k (k varies from 1 to Nk) is used to reset the photodetector region PhDk at the beginning or end of each microintegration window ("gating") (i.e., each basic integration interval INTr), as well as at the beginning of the acquisition sequence, to drain photogenerated charges outside the integration time interval that would be considered for the payload signal. Indeed, these stored parasitic charges must be drained because they form additional noise that should not be accumulated.

[0053] For each pixel Pix#k 102 (k varies from 1 to Nk), there is a storage channel VS that stores the charge generated by the light-receiving region PhDk. k,m (m varies from 1 to M) - Memory node MN k,m configured to drain the charge generated during the exposure image duration of the pixel Pix#k, and a charge transfer control signal TRA k,mCharge transfer transistor T2 controlled by k,m and, - memory node MN at the beginning of the current cycle Cycle_acq#k,s of the image k,m a transistor T3 configured to reset the potential of the pixel 100 and controlled by a reset signal RST common to the set of Nk pixels; k,m and, - connected in a follower configuration and connected from its gate to its source to a memory node MN k,m a memory node MN configured to return a potential level of k,m Read transistor T4 k,m and, - Memory node MN k,m Transistor T4 for reading out k,m , and in the read mode, the access conductor COL is connected to the M memory nodes of the pixel Pix#k via the image readout subsystem. k,m memory node MN k,m and a transistor T5 configured to return the charge potential of the pixel Pix#k and controlled by a control signal SELk that controls access to the pixel Pix#k. k,m and Includes:

[0054] According to FIG. 4 and the first embodiment of the pixel 102, for each pixel Pix#k 102 (k varies from 1 to Nk), various M memory nodes MN k,m (m varies from 1 to M) and a device 104 for simultaneously blocking exposure of pixel Pix#k, - a control signal TRA for transferring the charges generated during the exposure image duration of the pixel Pix#k; k,m (m varies from 1 to M) is activated when the subsystem for controlling the high-speed readout of Nk×M memory nodes and the exposure suppression of Nk pixels determines not to continue exposing the pixels according to the value read during the previous non-destructive high-speed readout operation, and is maintained until the pixels are completely read out. k,mconfigured to be modulated by a common modulation signal that suppresses charge transfer due to - Each storage channel VS that stores the charge generated by the photosensitive area PhDk k,m (m varies from 1 to M), the charge transfer control signal TRA k,m Transistor TI for suppressing k,m Includes:

[0055] The criteria are defined based on the application for which the sensor is used. This criteria may be based, for example, on the results of high-speed conversion of various memory nodes to evaluate whether or not it is necessary to continue exposing the pixel. This criteria may be, for example, a threshold setting for a configurable voltage common to all columns that becomes positive as soon as one of the memory nodes stores a signal that exceeds a reference voltage, and negative otherwise. This criteria may be different threshold settings for multiple configurable voltages depending on the column of memory nodes being read, as shown in FIG. 4.

[0056] According to FIG. 4, the device 104 for simultaneously blocking the exposure of the pixel Pix#k 104 or the mechanism for suppressing the exposure of the pixel is a signal that controls a transistor that transfers the charge accumulated in the light receiving area to a memory node, i.e., a signal TRA k,m (m varies from 1 to M) This filtering takes the form of an inhibitor memory point IMk (or "inhibitor memory") that filters or does not filter the transfer transistor TRA k,m (m varies from 1 to M) for controlling the exposure suppression transistor TI on the row k,m (i.e., "transfer inhibitors").

[0057] The memory point IMk of the suppression mechanism is connected to a row selection transistor SEL k The selected memory point is accessed through a transistor controlled by IM_SEL. k The exposure of the pixel in question is suppressed by the exposure suppression transistor TI k,m(m varies from 1 to M) can be charged or discharged to activate or deactivate the mechanism for suppression by driving the

[0058] In summary, the CMOS image sensor 102, according to the first embodiment of FIG. 4 and its variants, For each pixel Pix#k (k varies from 1 to Nk), various memory nodes MN k,m a collection of pixels of a particular electronic global shutter architecture including a device 104 for simultaneously blocking a common exposure of N pixels (where m varies from 1 to M); - Memory node MN k,m (m varies from 1 to M) M memory nodes for each pixel Pix#k are shared among k,m (m varies from 1 to M) or each memory node MN k,m a subsystem dedicated to the control of the criteria for each pixel; - A decision mechanism that allows the device to control the exposure suppression at pixel level based on the calculation of a criterion, which criterion is determined by the various memory nodes MN k,m (m varies from 1 to M) - Conventional full-resolution image readout chain with analog-to-digital conversion Includes:

[0059] Referring to FIG. 5, a method 202 of operating a CMOS image sensor according to the present invention includes a set of steps 204, 206, 206, 208, 210, 212, and 214.

[0060] In a first step 204, all of the pixels of the image sensor are reset, and the memory node MN of each pixel Pix#k (k varies from 1 to Nk) is reset. k,m (k varies from 1 to Nk, and m varies from 1 to M) and the charge stored in the light-receiving region PhDk is discharged.

[0061] Then, in a second step 206, an exposure phase for the pixel is performed, during which the suppression mechanism is inactive.

[0062] The second step 206 is followed by a third step 208, in which the memory node MN k,m A phase of non-destructive high-speed read operation at (m varies from 1 to M) is performed for each pixel Pix#k (k varies from 1 to Nk), and in the same third step 208, the result of the judgment criterion is determined.

[0063] If the result of the criterion is no, then in a fourth step 210 the mechanism that inhibits exposure remains deactivated, thereby allowing a signal, i.e., charge, to be stored in the memory node of the pixel in question during the next exposure.

[0064] If the result of the criterion is yes, then in a fifth step 212 the mechanism for suppressing exposure is activated or remains activated and remains active during the next exposure phase, so that the signal level (i.e., the amount of charge) stored in the node of the memory of the pixel in question during the next exposure phase remains constant.

[0065] The second, third, fourth and fifth steps 206, 208, 210, 212 are repeated a predetermined number Nr-1 times depending on the configuration of the sequence set in the image sensor. As the pixel allows a non-destructive read operation, the succession of different exposure phases allows the signal to accumulate in the memory node until it triggers or does not trigger the reference.

[0066] At the end of an image acquisition sequence, more or fewer pixels may trigger the reference, depending on the scene being observed and the configuration of the sequence.

[0067] At the end of the sequence, in a sixth step 214 below, an analog-to-digital conversion is performed over the full resolution of the sensor, after which the data is transmitted to a processing system to determine the distance of the object and its three-dimensional shape.

[0068] If the exposure time is long enough and distributed over a set of short enough exposure phases to avoid accumulating too many signals at once, all of the sensor's pixels can trigger the reference without otherwise reaching pixel saturation. Under these conditions, the signal obtained at the end of the sequence is optimal, since its signal-to-noise ratio is maximized.

[0069] FIG. 6 and the diagram showing an example of pixel-level ordering of the sensor illustrate the behavior of the various pixels of the sensor as they are viewed through a sequence that depends on the scene being observed.

[0070] The figure shows several pixels designated Pix(n), where n is the index of the level of photons received by that pixel. Index 0 represents the highest photon intensity, index Nn represents the lowest photon intensity. These different pixels trigger the reference at different times, and therefore different numbers of elementary exposure phases are required for this purpose.

[0071] The multiple phases of the CMOS image sensor sequencing are shown, namely, the reset phase (Reset or Rst), the exposure phase (Exposure), and the conversion and readout phase (Readout). For each pixel, there is a reset phase (Reset or Rst), an exposure phase (Exp), a fast conversion phase (FC), an exposure suppression phase (Blocked), and a full conversion and readout phase (Readout Black and Signal) for external CDS (Correlated Double Sampling).

[0072] The mechanism for inhibiting exposure of a pixel is an inhibit memory point IM (i.e., "inhibitor memory"), which is a signal that controls a transistor that transfers the charge stored in the light-receiving area to a memory node, i.e., a signal TRA k,m This filtering or this transmission suppression is performed via a filter transistor or transfer suppression transistor TI ("transfer inhibitor") located in the row for the signal controlling the transfer transistor.

[0073] Suppressed memory point IM k The row selection signal SEL k The selected memory point Im is accessed through a select transistor controlled by k The exposure suppression transistor TI k can be charged or discharged to deactivate or activate the mechanism for suppressing exposure of the pixel in question by driving

[0074] For example, FIG. 7 shows the complete image as seen from one pixel.

[0075] In the reset step, the mechanism is deactivated for all pixels. The conversion and readout of the reset levels is now performed to implement the external CDS.

[0076] An exposure phase is performed on the pixel, followed by a non-destructive fast read operation (FC Pix#k, Mn#x) of each memory node of the pixel, the result of which is used to calculate the reference (Crit Pix Col#k). In this fast conversion, the signal SEL k (Pix#k SELk) is activated.

[0077] When the reference is valid, the state of memory point "IM" (Pix#k IM) is updated by charging or discharging it. Discharging the inhibit memory point "IM" will block integration at that pixel.

[0078] By chaining the exposure and fast readout phases, it is possible to trigger the reference at different pixels.

[0079] At the end of the sequence, a full resolution analog-to-digital conversion is performed on each of the pixel's memory nodes and these data are transferred from the image sensor to an external image processing system.

[0080] The principle of operation of the criteria adapted to the fast read operation, together with their effect on the suppression mechanism incorporated in each pixel, is explained by Fig. 8 and the diagram of the sequence of the fast read operation phase and the exposure suppression control phase. The diagram shows the fast conversion sequence and its effect on the suppression mechanism in relation to pixels of the same column.

[0081] "FC Start Row #1" indicates the time when the high-speed read operation started, and "FC Done Row #1" indicates the completion of the high-speed conversion result for each memory node.

[0082] "FC Pix #k MN#" indicates the state of the high-speed conversion for memory nodes 0 to M of the pixel Pix#k currently being processed.

[0083] "Crit Pix Col#k" indicates the result of the exposure suppression criterion for the pixel currently being processed. "IM Pix Col#k" indicates the suppression memory point IM for the pixel currently being processed. k indicates the voltage level applied to

[0084] "TI Pix Col#k" indicates the logic state of the exposure suppression transistor currently being processed.

[0085] The fast conversion is based on the principle of a series of fast readouts of the various rows. For each row l (l between 1 and Nl), the signal SEL for the row to be converted is set, followed by the steps of fast readout, calculation of the exposure suppression criterion and its application. When a new row on the sensor is selected, the state of the signals from its pixels must be refreshed, and they therefore undergo unknown transition states.

[0086] For each selected row, a fast read operation is initiated, and then, upon completion, the new value of the reference is evaluated and the state of the inhibition memory point IM is refreshed, thereby updating the state of the current pixel exposure inhibition transistor TI for the next exposure phase.

[0087] The main advantage of this architecture is that it significantly reduces the time required for the various data conversions required to increase the dynamic range of the sensor, thus increasing the acquisition speed.

[0088] The amount of data transmitted by the sensor is significantly reduced, as the full analog-to-digital conversion is largely replaced by a high-speed or partial conversion without data transfer.

[0089] The data returned when the signal is read (final analog-to-digital conversion and transfer) directly contains the result of an HDR ("high dynamic range") processing operation that combines samples from multiple exposure time configurations and is suitable for indirect time-of-flight IToF distance calculations.

[0090] Conventional imaging seeks to measure the difference in intensity between different pixels, and does not aim for uniform response between different pixels of an image.

[0091] In contrast, indirect time-of-flight (IToF) seeks to equalize the maximum response of a pixel (as close as possible to saturation without reaching it) over multiple images with different phase offsets. Since the payload information is not in the absolute signal but in the signal difference between samples at different phase offsets, different exposure durations between pixels do not pose any problems. In contrast, to ensure coherence of the information measured within the same pixel, the same integration time should be applied to the various memory nodes of the pixel.

[0092] FIG. 9 shows the change in the acquisition speed by the image sensor and the data rate transmitted thereby depending on the number of images (i.e., the number of exposure time configurations) used for HDR processing, comparing a conventional image sensor with an image sensor according to the present invention.

[0093] The two image sensors whose performance is being compared share the following characteristics: - 600 lines, - number of columns 800, - 3 memory nodes per pixel, - Analog-to-digital conversion full resolution 10bit, - Use of external CDS (Correlated Double Sampling), - gating pulse duration 20.00ns, - 20% throttle duty cycle, - Numerical aperture 10000.

[0094] The results for certain performance parameters of a conventional image sensor without the architecture of the present invention (but incorporating a non-destructive readout mechanism) are as follows: - total exposure time length 10.00ms, - 10 exposure intervals for HDR (i.e. the number of exposure time configurations to manage the dynamic range of the scene), - No high speed conversion phase, - 11 complete read operations, - Memory node reset time 1.00 μs, - Complete conversion time per line: 3.50μs, - Total data conversion time 23.10ms, - Total acquisition time 33.10ms, - Maximum frame rate 30.21 frames per second (fps), - 30.21 frames per second equivalent data rate of 4.79 Gbit / s.

[0095] The results for specific performance parameters for an image sensor having an architecture according to the present invention are as follows: - total exposure time length 10.00ms, - 10 exposure intervals for HDR (i.e. the number of exposure time configurations to manage the dynamic range of the scene), - 9 high-speed conversion phases, - 2 full read operations (to perform external CDS), - Memory node reset time 1.00 μs, - Fast conversion time of 0.40 μs per row (including evaluation of criteria and refreshing of the suppression mechanism); - Complete conversion time per line: 3.50μs, - High speed conversion time 2.16ms, - Total data conversion time 4.20ms, - Total acquisition time 16.36ms, - Maximum frame rate 61.12 frames per second (fps), - Maximum data rate 1.76Gbits / s, - 30.21 frames per second equivalent data rate of 0.87Gbits / s - 82% data rate gain between solutions with and without the invention (for an acquisition rate of 30.21 frames / s in both cases).

[0096] FIG. 10 shows the variation in the acquisition speed by the image sensor and thus the transmitted data rate as a function of the duration of the fast conversion phase (evaluation of the reference and refreshing of the suppression mechanism) of the 10 images used, comparing a conventional image sensor with an image sensor according to the invention, the two sensors being the same as those used to determine the performance shown in FIG. 9.

[0097] 9 and 10 therefore clearly show that increasing the number of images captured to perform HDR processing has a much smaller impact on the image sensor according to the invention. Similarly, when the high-speed transform is faster than the conventional transform, the gains in acquisition speed and data rate are significant. Similarly, even when the high-speed transform is not significantly faster than the standard transform, the advantages of this solution remain, since the data exchange with the outside world already incorporates HDR processing, greatly simplifying the management of the processing operations that must be performed at the system level.

[0098] According to FIG. 11 and a second embodiment of pixels 302 and their peripheral circuitry incorporated in the image sensor of FIG. 3, and more generally in a CMOS image sensor according to the present invention, each pixel 302 differs from pixel 102 of FIG. 4 in that its suppression mechanism involves resetting the photosensitive area of ​​the pixel rather than the transfer transistor.

[0099] The principle of operation of this variant is slightly different: the suppression of pixel exposure is performed by resetting the light-receiving area rather than using a transfer transistor. The logic that applies here is reversed: - By applying a voltage to the suppression memory point IMk, the light receiving area is k , and thus prevents the collection of charges during the transfer phase, since they are collected by VDDpix. - Suppression Memory Point IM k When no voltage is applied to the exposure suppression transistor TI kis in the off state and nothing is done to the light-receiving region, so the charge collection in the transfer phase is maintained.

[0100] At the start of the acquisition sequence, for each pixel Pix#k (k varies from 1 to Nk), the light receiving area PhDk, memory node MNk,m (m varies from 1 to M) and the inhibition memory point are reset. Then, the blocking mechanism is deactivated for all pixels Pix#k (k varies from 1 to Nk).

[0101] After each exposure phase, a fast conversion phase is performed, which allows evaluating an exposure inhibition criterion for each pixel Pix# (k varies from 1 to Nk). If the result of the evaluation of the exposure inhibition criterion is no, the inhibition memory point IMk of the pixel in question remains discharged (deactivating the charge collection inhibition and turning off the transistor TIk). If the result of the evaluation of the exposure inhibition criterion is yes, the inhibition memory point IMk of the pixel in question remains charged (activating the charge collection inhibition and turning on the transistor TIk).

[0102] Photosensitive area PhD regardless of the state of the exposure suppression device k can be reset at any time.

Claims

1. 1. A CMOS active pixel image sensor having multiple memory nodes for indirect time-of-flight (IToF) distance measurements in a scene of reflective objects, comprising: - light receiving regions PhDk and memory nodes MN, each of which is an integer M greater than or equal to 2; k,m A set of CMOS active pixels (102, 302) Pix#k (k varies from 1 to Nk), each of which is an integer greater than or equal to 1, and has a pixel size of Nk (m varies from 1 to M). the M memory nodes MN of each pixel Pix#k (k varies from 1 to Nk) k,m an image readout subsystem (62) for the M memory nodes MN generated by the light receiving areas PhDk (k varies from 1 to Nk) during a pixel exposure image duration equal to or less than a predetermined maximum exposure image duration Tmax_exp, k,m an image readout subsystem (62) for determining, through analog-to-digital conversion, Nk×M digital values ​​representing the amount of charge integrated by an image pre-processing subsystem (64) configured to apply digital processing operations to the digital values ​​provided at output by the image readout subsystem and to transmit pre-processed pixel-level image data to an application system that determines the indirect time of flight (IToF) of the observed scene; 1. A CMOS active pixel image sensor comprising: The Nk×M memory nodes MN k,m and a subsystem (66) for controlling a mechanism for suppressing exposure of the Nk pixels Pix#k, wherein in each of the Nk pixels Pix#k (k varies from 1 to Nk), a predetermined integer Nr number of non-destructive fast read operations FCr (r varies from 0 to Nr-1) are performed on the M memory nodes MN during a current cycle Cycle_acq#k,s for acquiring images from the Nk pixels of the CMOS active pixel image sensor. k,m (m varies from 1 to M), and after each fast read operation FCr, a signal for suppressing the exposure of the pixel Pix#k is determined based on a criterion as to whether the exposure of the pixel Pix#k should be continued or not, and the criterion is k,m a subsystem (66) that depends on the voltage value non-destructively read at In each pixel Pix#k, at least one transistor (TI k , TIk,m), and a memory point (IM) that can be charged or discharged by the signal that inhibits the pixel Pix#k in order to activate or deactivate the mechanism that inhibits the pixel by driving its one or more exposure inhibition transistors. k )and 1. A CMOS active pixel image sensor comprising:

2. Each pixel (102, 302) Pix#k (k varies from 1 to Nk) has M memory nodes MN k,m (m varies from 1 to M) is configured to integrate the charge generated by the light-receiving region PhDk during r basic integration intervals INTr (r varies from 0 to Nr-1), and following each basic integration interval INTr, k,m A fast read operation FCr for and a re-evaluation of the criteria are performed; and 2. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement as described in claim 1, wherein the r integration intervals INTr distributed within the current cycle Cycle_acq#s for acquiring pixel images each have a basic duration Tr, whereby the sum of the basic integration durations Tr (r varies from 0 to Nr-1) is equal to the predetermined maximum exposure image duration Tmax_exp.

3. The basic durations Tr (r varies from 0 to Nr) are equal to the same value, or 3. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement of claim 2, wherein at least two elementary durations obtained from a set of values ​​Tr (r varies from 0 to Nr-1) are different, and a variance of the values ​​Tr (r varies from 0 to Nr-1) is selected to maximize a dynamic range of the CMOS active pixel image sensor.

4. The phases of the set of phases to be executed during the current cycle Cycle_acq#s are commands for the Nk pixels (k varies from 1 to Nk) to the Nk×M memory nodes MN k,m and the mechanism for suppressing the exposure of the Nk pixels Pix#k, and the set of phases includes a sequencer (72) that activates the set of phases by sending a signal to the image pre-processing subsystem. For each pixel (102, 302) Pix#k (k varies from 1 to Nk), the light receiving area PhDk and the M memory nodes MN k,m (m varies from 1 to M) a phase (Reset or Rst) of resetting the M memory nodes, a phase (FCr) of non-destructive fast readout of the M memory nodes, a phase (exp0) of initially exposing the pixel Pix#k, followed by one or more phases (Exp) of a first number Nr1(k,s) of holding the exposure of the pixel Pix#k and then one or more phases (Blocked) of a number Nr2(k,s) of suppressing the exposure of the pixel Pix#k (where the sum of the numbers Nr1(k,s) and Nr2(k,s) is equal to Nr-1), and a phase (Read) of reading an image from the pixel Pix#k. and the light receiving regions PhDk and the M memory nodes MN k,m 4. A CMOS active pixel image sensor with multiple nodes for indirect time-of-flight IToF distance measurement according to claim 1, wherein the phase (Reset or Rst) of resetting (k varying from 1 to Nk and m varying from 1 to M) is performed simultaneously with each acquisition of an image s in "global shutter" mode of the Nk pixels of the CMOS active pixel image sensor.

5. For each pixel (102, 302) Pix#k (k varies from 1 to Nk), the image reading operation is performed by the light receiving region PhDk and the M memory nodes MN k,m 5. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement of claim 4, wherein the image reading operation is performed after a phase of resetting (m varies from 1 to M), and the image reading operation is used to perform external CDS.

6. Each pixel (102, 302) Pix#k (k varies from 1 to Nk) is configured with the same architecture, said light receiving areas PhDk, a transistor T1k for resetting said light-sensitive area PhDk at the beginning of each cycle of image acquisition and at the beginning or end of each microintegration cycle, said transistor T1k being controlled by a global reset signal RPhk common to said Nk pixels of said set of pixels; M storage channels VS of the charges generated by the light-receiving area PhDk and integrated in parallel during the exposure duration of the pixel Pix#k corresponding to the current cycle Cycle_acq#k,s of image acquisition; k,m M memory nodes MN for shared storage on k,m (m varies from 1 to M) a device (104, 304) for simultaneously blocking the exposure of said pixel Pix#k common to said various M memory nodes Mk,m (m varying from 1 to M); M independent read access conductors COL to the subsystem for the fast readout of the Nk×M memory nodes by the pixel Pix#k and for the control of the mechanism for suppressing the Nk pixels, and to the image readout subsystem for the M memory nodes of the pixel Pix#k; k,m (m varies from 1 to M) and the suppression command access channel COL k,IM Device (106, 306) for activating access via 5. A CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement according to claim 1, comprising:

7. For each pixel (102, 302) Pix#k (k varies from 1 to Nk), there is a storage channel VS that stores the charge generated by the light receiving region PhDk. k,m (m varies from 1 to M) - said memory node MN k,m a charge transfer control signal TRA configured to drain the charge generated during the exposure duration of the pixel Pix#k; k,m Charge transfer transistor T2 controlled by k,m and, at the beginning of the current cycle Cycle_acq#k,s of an image k,m and controlled by a reset signal by a signal RST common to the set of Nk pixels. k,m and, - connected in a follower configuration and connected from its gate to its source to the memory node MN k,m the memory node MN k,m Transistor T4 reads k,m and, - said memory node MN k,m The transistor T4 reads k,m and in a read mode, the access conductor COL is connected to the source of the M memory nodes of the pixel Pix#k through the image readout subsystem. k,m The memory node MN k,m a control signal SEL configured to return a charge potential of k Select transistor T5 controlled by k,m and 7. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement according to claim 6, comprising:

8. For each pixel (102) Pix#k (k varies from 1 to Nk), the various M memory nodes MN k,m The device (104) for simultaneously blocking the exposure of the pixel Pix#k, which is common to all pixels (m varies from 1 to M), comprises: the charge transfer control signal TRA, which transfers the charge generated during the exposure duration of the pixel Pix#k; k,m (m varies from 1 to M) to the charge transfer transistor T2, which is activated when the subsystem (66) for controlling the mechanism for fast readout of the Nk×M memory nodes and suppressing the Nk pixels determines not to continue the exposure of the pixel according to a value read in a previous non-destructive fast read operation, and is maintained until the pixel is completely read. k,m configured to modulate the charge transfer by a common modulation signal; and - each storage channel VS k,m (m varies from 1 to M), the charge transfer control signal TRA is used to store the charge generated by the light receiving region PhDk. k,m The transistor TI k,m 8. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement according to claim 7, comprising:

9. For each pixel (302) Pix#k (k varies from 1 to Nk), the various M memory nodes MN k,m The device (304) for simultaneously blocking the exposure of the pixel Pix#k, which is common to all pixels (m varies from 1 to M), comprises: the M charge transfer transistors T2 of the charges generated by the light-receiving area PhDk during the exposure duration of the pixel Pix#k k,m (m varies from 1 to M) by a common modulation signal that is activated when the subsystem (66) for controlling the mechanism for fast readout of the Nk×M memory nodes and suppressing the Nk pixels decides not to continue the exposure of the pixel (302) Pix#k according to a value read during a previous non-destructive fast read operation, and that suppresses the discharge of the charge to the common connection node that is maintained until the pixel is completely read; and the transistor TI connected in parallel with the transistor T1k resetting the light-receiving region PhDk and suppressing the discharge of the charge generated by the light-receiving region PhDk to the common connection node; k 8. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement according to claim 7, comprising:

10. The memory point (IMk) of the mechanism for suppressing exposure of the Nk pixels Pix#k is controlled by the control signal SEL k 10. A CMOS active pixel image sensor for indirect time-of-flight IToF distance measurement according to any one of claims 7 to 9, having a plurality of nodes accessed via transistors controlled by:

11. said Nk pixels (k varying from 1 to Nk) are arranged in the form of a matrix (54) with Nl rows, an integer greater than or equal to 2, and Nc columns, an integer greater than or equal to 2, Nk being equal to the product of Nc and Nl; row by row and simultaneously over a given time frame, pixels of the same row l are read from the image readout subsystem (62) and from the Nc×M memory nodes MN k,m 5. The CMOS active pixel image sensor having multiple nodes for indirect time-of-flight IToF distance measurement according to claim 1, further comprising a row decoder (68) configured to enable access to processing resources of the subsystem (66) for the fast readout and control of the mechanism for suppressing the exposure of the Nc pixels in the same row l, and thus pooling the processing resources of the image readout subsystem (62) and the subsystem (66) for fast readout and control of the mechanism for suppressing the exposure of the pixels column by column.

12. 1. An apparatus for indirect time-of-flight (IToF) distance measurement in a two-dimensional or three-dimensional object scene, comprising: an emitter light source (8) that emits a series of light pulses; a CMOS active pixel image sensor (12, 52) with a plurality of memory nodes according to any one of claims 1 to 10 for converting the light pulses reflected by objects in the two-dimensional or three-dimensional object scene (6) into pixel data characterizing one or more images; a block (14) for processing the pixel data from the one or more images to determine the depth of field of the object in the scene (6) as seen from the location of the emitter light source (8) and the CMOS active pixel image sensor (12, 52); An apparatus comprising:

13. 1. A method for implementing a CMOS active pixel image sensor having multiple memory nodes for indirect time-of-flight (IToF) distance measurement of a scene to be observed, the CMOS active pixel image sensor (52) comprising: - light receiving regions PhDk and memory nodes MN, each of which is an integer M greater than or equal to 2; k,m A set of Nk CMOS active pixels (102, 302) Pix#k (k varies from 1 to Nk), where Nk is an integer greater than or equal to 1, having m (m varies from 1 to M); said Nk×M memory nodes MN executed during the acquisition of the image with said Nk pixels; k,m a subsystem (66) for non-destructive high-speed readout of the Nk pixels Pix#k and for controlling a mechanism for suppressing exposure of the Nk pixels Pix#k; the M memory nodes MN of each pixel Pix#k (k varies from 1 to Nk) k,m an image readout subsystem (62) for - Image Pre-Processing Subsystem (64) wherein the method of implementing the image sensor during image acquisition comprises: The image readout subsystem (62) reads the M memory nodes MN of each pixel (102, 302) Pix#k (k varies from 1 to Nk). k,m , generated by the light-receiving regions PhDk (k varies from 1 to Nk) and stored in the M memory nodes MN during a pixel exposure image duration equal to or less than a predetermined maximum exposure image duration Tmax_exp. k,m determining N×M digital values ​​representing the amount of charge integrated by - the image pre-processing subsystem (64) applies digital processing operations to the digital values ​​provided at output by the image readout subsystem (62) and transmits pre-processed pixel-level image data to an application system that determines the indirect time-of-flight (IToF) of the observed scene. A method comprising: The Nk×M memory nodes MN k,m The subsystem (66) for controlling the mechanism for suppressing the non-destructive fast readout and the exposure of the Nk pixels Pix#k performs a predetermined integer Nr number of non-destructive fast read operations FCr (r varies from 0 to Nr-1) on the M memory nodes MN during a current cycle Cycle_acq#k,s for acquiring an image s in each of the Nk pixels Pix#k (k varies from 1 to Nk). k,m (m varies from 1 to M), and after each fast read operation FCr, a signal for suppressing the exposure of the pixel Pix#k is determined based on a criterion for whether or not to continue the exposure of the pixel Pix#k, and the criterion is k,m and one or more decision thresholds; and The signal that inhibits each pixel Pix#k is transmitted through at least one transistor (TI k , TIk,m) of the pixel Pix#k to activate or deactivate the mechanism for inhibiting the pixel. k ) charging or discharging.

Citation Information

Patent Citations

  • Conditional reset multi-bit readout image sensor

    JP2015530855A

  • Image sensor having an extended dynamic range upper limit

    US20160182847A1

  • Time-of-flight (TOF) image sensor using amplitude modulation for range measurement

    US20180059224A1

  • Pixel-level background light subtraction

    WO2019049685A1