Feedback control sensor array and sensor unit

Abstract

The present application relates to systems, devices, and methods for feedback control of a sensor array, the device including an imaging sensor unit including a sensor module including a plurality of optical sensors, a memory module coupled to the sensor module and configured to receive a sensor output from the sensor module, and a local control module coupled to the sensor module and configured to receive a local feedback signal from the sensor module, receive a global feedback signal, control the memory module using the local feedback signal and the global feedback signal, and store the sensor output.

Description

[Technical field]

[0001] The present application relates to sensor arrays, particularly image sensor arrays, systems, apparatus and methods for feedback control of individual sensor units of an image sensor array, and methods for providing local and global feedback control of image sensor arrays and sensor units to optimize sensing dynamics. [Background technology]

[0002] A sensor array is a group of sensors arranged, usually in a regular geometric pattern, to detect and process some type of signal. Examples of sensor arrays include optical arrays used in image sensors and cameras, electromagnetic radiation sensor arrays used in deep space telescopes and ground-based antennas, chemical sensor arrays, pressure sensor arrays, sonar arrays, and geophone arrays.

[0003] An image sensor or image sensor array or imager is a sensor that detects the spatial distribution of electromagnetic radiation (visible or invisible) that strikes the sensor and creates an image that conveys relevant information. A complementary metal-oxide semiconductor (CMOS) sensor is an example of an active pixel image sensor used to capture images in modern cameras. These sensors typically contain a regular geometric pattern of collectively controlled pixels.

[0004] However, CMOS image sensors (CIS) have certain drawbacks under certain conditions. Because CMOS sensors may capture one row of pixels at a time, the resulting image may suffer from a rolling shutter effect that distorts the image. Even CMOS sensors that use a global shutter may require post-processing to form clear image information of moving objects. However, the way pixel output is captured in a CMOS sensor may result in image artifacts from the output of pixels at a previous moment. Furthermore, when a scene contains simultaneous bright and dark areas, many details in the image may be lost due to overexposure / underexposure of pixels that contain bright and dark areas.

[0005] Vision, or the technology that deals with images, is the most information-dense sensing technology, providing a better representation of the world around us. Interpreting a scene in real-time machine vision applications typically requires intensive digital processing, which must be performed on high-performance computing platforms, especially when artificial intelligence techniques are used. Machine vision applications typically use cameras that incorporate automatic exposure control, pixel correction, analog-to-digital conversion, and standard output video formats. Although these offer high performance, one of their main drawbacks is that they have a fixed dynamic range. This can severely limit their use in many outdoor scenarios with uncontrolled light conditions and large dynamics within the scene, such as automotive, surveillance, and industrial process monitoring applications.

[0006] The most common approach to acquiring a scene containing both bright and dark areas is to take two or more frames acquired with different exposure times and merge them into one high dynamic range (HDR) image using software-based techniques of post-processing. However, this approach can introduce artifacts, especially in scenarios with fast moving objects, since the time lag between the acquisition of the frames leads to misalignment between images taken at different moments. On the other hand, using multiple sensors in parallel operating with different exposure times to capture the images to be merged can reduce the problems caused by moving objects, but cannot completely solve them. It also requires post-processing to correct potential misalignments in the images (the so-called "parallax" problem), which comes at a significant cost in terms of space and energy.

[0007] Other approaches have been employed to obtain single-exposure HDR images with a low probability of artifacts. Sensor arrays based on pixels with compressed dynamic response of current or voltage (e.g. log-log or linear-log): information is lost in the compressed part of the dynamic range and / or significant levels of noise are present when the signal is weak. Sensor arrays based on pixels containing two photodiodes: one with high sensitivity and one with low sensitivity, whose signals are fused after the exposure time. However, this solution requires redundant pixels and careful design to reduce crosstalk between pixels and obtain uniform optical performance. Sensor arrays based on pixels with lateral overflow can be very effective at providing single-exposure HDR images, but the added complexity can lead to signal-to-noise (S / N) issues.

[0008] In many cases, photo quality and image resolution are less important if poor photo quality is compensated for by the sensor's ability to capture all the relevant details of a scene. When a sensor can capture all the relevant details, the system can interpret the scene with early access to visual features that can be efficiently extracted by the image, allowing the system to make fast and reliable decisions. For example, images acquired by a sensor that uses different levels of sensitivity to different areas of the scene may appear distorted, flat, or have the wrong colors, but if this is due to mechanisms that enhance the visibility of the relevant details contained in the scene, the perception ability of a vision system using such images will be improved.

[0009] In this respect, custom vision sensors designed for machine vision allow for a tight and effective integration of mixed analog and digital image processing layers with optical sensing elements, providing high-speed, low-power visual processing devices. On the other hand, their main drawback is their larger pixel size compared to commercial imagers designed for photography, which means a larger chip area is required for a sensor with the same pixel resolution, resulting in higher silicon costs. Nevertheless, there are many applications relying on vision technology that do not require high resolution, but high dynamic range and low power consumption, such as people and object monitoring, tracking, detection, surveillance scenarios, and human-machine interfaces.

[0010] The above considerations also apply when the physical quantity to be acquired is distributed in space and its spatial distribution is important and when that distribution is detected using sensors other than those based on visible light, which is not the case, for example, for electromagnetic waves not in the visible spectrum (e.g., infrared, ultraviolet, terahertz, etc.) or other physical sources (e.g., sound, pressure, etc.).

[0011] In contrast to CMOS image sensors and the artificial sensor arrays mentioned above, biological vision, as typified by the human eye, is not based on a globally controlled, evenly spaced array of identical light-sensitive elements, but rather uses an irregular distribution of different types of photoreceptors, mainly cones, rods and ganglion cells, to detect the incoming light in an autonomous manner (i.e., without the need for a global control system). The rods and cones are connected in a biological feedback system involving the brain via bipolar cells, ganglion cells and finally the optic nerve. Retinal ganglion cells, as shown by recent results, also show the ability to detect light (i.e., their behavior is influenced by the light that falls on them) and receive input from about 100 rods and cones on average. Retinal ganglion cells collectively transmit image-forming and non-image-forming visual information in the form of action potentials. Excitation of retinal ganglion cells increases the firing rate of action potentials, whereas inhibition decreases the firing rate of action potentials. This biological system is able to adaptively use simple inputs from rod and cone cells to discern complex visual information in the brain's visual cortex. Different cells are excited depending on the local and global properties of the scene presented to the human eye. Furthermore, the human visual system does not introduce artifacts into the image. Also, as mentioned before, biological vision does not require a regular geometric arrangement of cells and does not rely entirely on global control. Finally, images acquired by the biological visual system will look poor (usually blurry, distorted, and sometimes monochromatic) when compared to typical images acquired by standard CMOS imagers. The quality of the image and its information content are perfectly restored by the brain through a feedback-driven process that optimizes the retinal response.

[0012] It has been recognized that sensor arrays inspired by biological visual systems can overcome the problems associated with artificial sensor arrays. Summary of the Invention

[0013] This summary is provided to introduce some concepts in a simplified form that are further described in the Detailed Description.

[0014] In a first aspect, the present disclosure provides an imaging sensor unit including a sensor module including a plurality of optical sensors, a memory module coupled to the sensor module and configured to receive a sensor output from the sensor module, and a local control module coupled to the sensor module and configured to receive a local feedback signal from the sensor module, receive a global feedback signal, control the memory module using the local feedback signal and the global feedback signal, and store the sensor output. The sensor module may be referred to as a photoreceptor or photoreceptor unit.

[0015] In this manner, the local control module controls the function and operation of the multiple light sensors, allowing the sensor units to operate at least partially independently from the other sensor units and to store the sensor outputs according to optimal times or conditions established based on the global and local feedback signals. The light sensors may be photodiodes or any light-sensitive elements.

[0016] Preferably, the global feedback signal includes a threshold value, and the local control module is configured to compare the local feedback signal with the global feedback signal threshold value and to control the memory module to store the sensor output when the local feedback signal meets the threshold value.

[0017] The global feedback signal may be adjustable and may be provided from an external source such as a global control module or a computer. The optimal threshold may be determined prior to using various optimization techniques in the computer.

[0018] Preferably, the plurality of photosensors include a first photosensor sensitive to a first portion of the visible spectrum and configured to output a first sensor output component, a second photosensor sensitive to a second portion of the visible spectrum and configured to output a second photosensor output component, and a third photosensor sensitive to a third portion of the visible spectrum and configured to output a third photosensor output component, wherein the sensor outputs include the first, second and third photosensor output components.

[0019] Preferably, the imaging sensor unit further includes a summation node configured to sum the first, second, and third photosensor output components to generate a summation value, the local feedback signal including the summation value, the local control module configured to compare the summation value with a threshold value of the global feedback signal, and when the summation value meets the threshold value, the local control module configured to cause the memory module to store the first, second, and third photosensor output components.

[0020] Instead of a summing node, the imaging sensor unit preferably further includes a winner-take-all node configured to receive the first, second, and third photosensor output components and determine a winning photosensor output component by identifying which of the first, second, and third photosensor output components exhibits a largest magnitude difference relative to initial values ​​of the respective first, second, and third output components, wherein the local feedback signal includes the winning photosensor output component, and the local control module is configured to compare the winning photosensor output component with a threshold value of the global feedback signal, and when the winning photosensor output component meets the threshold value, the local control module is configured to cause the memory module to store the first, second, and third photosensor output components.

[0021] Preferably, the first light sensor is sensitive to red light, the second light sensor is sensitive to green light, and the third light sensor is sensitive to blue light.

[0022] Preferably, the plurality of light sensors includes a broad spectrum light sensor, the broad spectrum light sensor configured to detect a brightness of a viewed scene and generate a brightness output, the local feedback signal includes the brightness output, and when the brightness output satisfies a global feedback signal threshold, the local control module is configured to cause the memory module to store the sensor output.

[0023] This broad spectrum light sensor may be used in combination with the first, second, and third light sensors, a summing node, or the winner-take-all node described above. When used in conjunction with the first, second, and third light sensors, the broad spectrum light sensor output may be used as a local feedback signal, eliminating the need to sample or interfere with the outputs from the first, second, and third light sensors before storing them.

[0024] Preferably, the plurality of light sensors includes a broad spectrum light sensor, the broad spectrum light sensor configured to detect brightness of a viewed scene to generate a brightness output, and the local control module is further configured to receive a second local feedback signal and a second global feedback signal, the second local feedback signal including the brightness output, the second global feedback signal including a second threshold, the local control module configured to compare the second local feedback signal with the second threshold, and when the second local feedback signal satisfies the second threshold, the local control module is configured to cause the memory module to store the brightness output. The imaging sensor further includes a readout module, the readout module configured to output one or more of the brightness output, the first, second and third light sensor element outputs from the imaging sensor based on a result of, or a function of, the comparison with a third global feedback signal received at the readout module.

[0025] The use of both the broad-spectrum light sensor and either the sum node or the winner-all node allows for two contributions to the local feedback signal, each of which may be used to trigger storage of the sensor output. The broad-spectrum light sensor may be a highly sensitive light sensor configured to detect the brightness of the observed scene and generate a brightness output even when the intensity of the incident light is low or very low, where the broad-spectrum light sensor is more sensitive to the incident light than the first, second, and third light sensors.

[0026] Preferably, the broad spectrum light sensor is configured to detect white light.

[0027] Preferably, the broad spectrum light sensor is configured to detect infrared radiation.

[0028] Preferably, the imaging sensor unit is subject to a maximum exposure time, and the memory module is configured to store the sensor output if the local feedback signal does not meet the threshold within the maximum exposure time.

[0029] The imaging sensor unit may form part of an array of imaging sensor units. Each of these units may be globally controlled in a synchronous manner, such that each unit is activated and starts sensing at the same time, and the maximum exposure period of each unit is the same. However, based on the local control module of each imaging sensor unit, the sensor outputs stored by each unit may be different and stored at different times. Each of these sensor outputs and the relative times they were stored may be read out in a synchronous manner at the end of the maximum exposure period.

[0030] Preferably, the local control module is further configured to store timing information in the memory module indicating the time the sensor output was stored.

[0031] Preferably, the memory module is configured to receive a timing signal as part of the global feedback signal, and the timing information is a record of the timing signal indicating the time the sensor output was stored.

[0032] Storing timing information in this manner means that the imaging sensor unit stores both the sensor output according to an optimal condition that depends on the local and global feedback signals, and the time at which the sensor output is stored that indicates when the optimal condition was achieved. Having both these parameters can be useful in post-processing tasks to obtain absolute sensor values.

[0033] According to a second aspect, there is provided a sensor array comprising a plurality of imaging sensors according to the first aspect, the sensor array further comprising a global control module configured to globally control the sensor array, the global control module configured to provide a global feedback signal to each of the plurality of imaging sensors.

[0034] Preferably, the global feedback signal is adjustable to adjust characteristics of the image captured by the sensor array. The global feedback signal may include a number of different signals, such as a timing signal, a first, a second and a third global feedback signal.

[0035] Preferably, the sensor array further includes readout circuitry configured to read out the stored sensor outputs from the plurality of imaging sensors.

[0036] Preferably, the readout circuitry is further configured to read out timing information for each of the plurality of imaging sensors, the timing information indicative of a time at which the stored sensor output for each of the plurality of imaging sensors is stored.

[0037] As previously mentioned, the sensor array may be globally synchronously controlled such that each sensor unit is activated at the same time and the readout process of the sensor units occurs simultaneously after the global maximum exposure time, but each sensor unit is also locally controlled, i.e., each unit stores sensor output values ​​and timing information that may be obtained before the end of the maximum exposure period according to optimal conditions achieved based on the use of a global feedback signal and the local feedback signal of each particular sensor unit.

[0038] Preferably, the sensor array is communicatively coupled to a computing device, and the readout circuitry is configured to transmit information relating to at least the sensor output of each of the plurality of image sensors to the computing device.

[0039] Preferably, the sensor array further includes one or more motion sensor units, the one or more motion sensor units including a light sensor, a motion sensor memory module configured to receive a timing signal, and a motion sensor control module, the motion sensor control module configured to receive and compare a sensor output of the light sensor with a global motion threshold, and store timing information from the timing signal in the memory module when the sensor output of the light sensor meets the global motion threshold, the timing information indicating a time when the global motion threshold is met, the motion sensor memory module includes a switch configured to change a storage location of the timing information, during a first active motion sensor phase of the light sensor corresponding to a first frame of the observed scene, the switch is placed in a first position corresponding to the first storage location, and and during a subsequent second active motion sensor phase of the optical sensor corresponding to a second frame of the observed scene, the switch is placed in a second position corresponding to the second storage location, and the local control module causes the memory module to store first timing information indicative of a time when a global motion threshold for the first active motion sensor phase is met in the first storage location and causes the memory module to store second timing information indicative of a time when a global motion threshold for the second active motion sensor phase is met in the second storage location, the local control module is configured to calculate a difference value corresponding to a difference between the first timing information and the second timing information, compare the difference value to a second global motion threshold, and output a signal from the motion sensor indicating that motion has been detected between the first and second frames of the observed scene when the difference value satisfies the second global motion threshold.

[0040] Preferably, the first global motion threshold and the second global motion threshold are adjustable.

[0041] Preferably, the first global motion threshold is a photosensor output threshold and the second global motion threshold is a time threshold.

[0042] Preferably, a signal indicating that motion has been detected between the first and second frames of the observed scene is configured to activate the plurality of imaging sensor units.

[0043] Preferably, the sensor array includes a plurality of motion sensor units.

[0044] Preferably, the plurality of motion sensor units are arranged at least around the periphery of the sensor array.

[0045] Using such a motion detector, which effectively detects brightness changes between frames at a local level, allows the sensor array to be "motion activated," where activation of one or more sensor units depends on the excitation of the motion sensor unit. Based on which motion sensor units are excited, sensor units of the sensor array may be activated only in areas of the sensor array where the excited motion sensor is localized.

[0046] According to a third aspect, there is provided a method of controlling an imaging sensor unit including a sensor module including a plurality of light sensors, a local control module coupled to the sensor module, and a memory module coupled to the sensor module, the method including receiving a sensor output from the sensor module at the memory module, receiving a local feedback signal from the sensor module at the local control module, receiving a global feedback signal at the local control module, and controlling the memory module using the local feedback signal and the global feedback signal to store the sensor output.

[0047] Preferably, the method includes a step of operating the imaging sensor unit according to a cycle, the cycle being defined by an active sensor phase initiated by activating the sensor module from an initial state so that the sensor module begins to detect and generate a sensor output, and an inactive sensor phase initiated by resetting the sensor module to the initial state.

[0048] Preferably, the method further comprises the steps of entering an active sensor phase and, after storing the sensor output, entering an inactive sensor phase.

[0049] Preferably, the duration of the active sensor phase is limited by a maximum exposure period, and the method further includes the steps of causing the memory module to store the sensor output at the maximum exposure period, and entering an inactive sensor phase such that the inactive sensor phase is entered after the maximum exposure period has elapsed.

[0050] Preferably, the method further comprises, when storing the sensor output, storing timing information indicating the time the sensor output was stored.

[0051] Preferably, the method further comprises the step of controlling the plurality of imaging sensor units, including the step of synchronously entering an active phase for each imaging sensor unit.

[0052] According to a fourth aspect, there is provided a motion sensor including a light sensor, a motion sensor memory module configured to receive a timing signal, and a motion sensor control module configured to receive and compare a sensor output of the light sensor to a global motion threshold, and to store timing information from the timing signal in the memory module when the sensor output of the light sensor satisfies the global motion threshold, the timing information indicating a time when the global motion threshold was met. the motion sensor memory module includes a switch configured to change a storage location of the timing information, wherein during a first active motion sensor phase of the light sensor corresponding to a first frame of the observed scene, the switch is placed in a first position corresponding to the first storage location, and during a subsequent second active motion sensor phase of the light sensor corresponding to a second frame of the observed scene, the switch is placed in a second position corresponding to the second storage location; the local control module is configured to cause the memory module to store first timing information indicative of a time when a global motion threshold for the first active motion sensor phase is met in the first storage location and to cause the memory module to store second timing information indicative of a time when a global motion threshold for the second active motion sensor phase is met in the second storage location; the local control module is configured to calculate a difference value corresponding to a difference between the first timing information and the second timing information, compare the difference value to a second global motion threshold, and output a signal from the motion sensor indicating that motion has been detected between the first and second frames of the observed scene when the difference value satisfies the second global motion threshold.

[0053] According to a fifth aspect, a system includes a sensor element for detecting a physical property, a local control module coupled to the sensor element and configured to locally control the sensor element, and a memory module coupled to the sensor element and configured to receive a sensor output from the sensor element, the sensor output being time-variable, the local control module configured to receive and compare a local feedback signal and a global feedback signal, the local feedback signal being based on the sensor output, the global feedback signal including a threshold criterion, and when the local feedback signal meets the threshold criterion the local control module is configured to store timing information in the memory module indicating a time at which the threshold criterion was met.

[0054] Preferably, the timing information is a measurement of a variable electrical signal that varies according to a predetermined relationship with time, the variable electrical signal being input to the memory module as part of a global feedback signal.

[0055] Preferably, the variable electrical signal is initialized at the start of the active sensor phase and reset at the end of the active sensor phase, and a measurement of the variable electrical signal indicates the time that the threshold criterion is met since the time the active sensor phase was activated.

[0056] Preferably, if the sensor output does not meet the threshold criterion during a predetermined maximum exposure period, the local control module is configured to store timing information in the memory module at the end of the maximum exposure period indicating the time when the exposure period will end.

[0057] Preferably, the sensor element includes one or more optical sensor elements configured to transduce / detect electromagnetic radiation.

[0058] Preferably, the one or more light sensor elements are configured to detect visible light.

[0059] Preferably, the one or more photosensor elements further include a first photosensor element sensitive to a first portion of the visible spectrum (configured to output a first photosensor element output), a second photosensor element sensitive to a second portion of the visible spectrum (configured to output a second photosensor element output), and a third photosensor element sensitive to a third portion of the visible spectrum (configured to output a third photosensor element output), and the sensor output includes the first, second, and third photosensor element outputs.

[0060] Preferably, the sensor unit further includes a summation node configured to sum the first, second and third photosensor element outputs to generate a summation value, wherein the local feedback signal includes the summation value, and the local control module is configured to compare the summation value with a threshold criterion of the global feedback signal, and when the summation value satisfies the threshold criterion, the local control module is configured to cause the memory module to store the first, second and third photosensor element outputs.

[0061] Preferably, and instead of the summing node, the imaging sensor unit further includes a winner-take-all node configured to receive the first, second, and third photosensor element outputs and determine a winning photosensor element output by identifying which of the first, second, and third photosensor element outputs exhibits a largest magnitude difference relative to initial values ​​of the respective first, second, and third photosensor element outputs, wherein the winning photosensor element output is a local feedback signal, and the local control module is configured to compare the winning photosensor element output with a threshold criterion of the global feedback signal, and when the winning photosensor element output satisfies the threshold criterion, the local control module is configured to cause the memory module to store the first, second, and third photosensor element outputs.

[0062] Preferably, the one or more light sensor elements include a broad spectrum light sensor element, the broad spectrum light sensor element configured to detect a brightness of a viewed scene and generate a brightness output, the sensor output being the brightness output. The broad spectrum light sensor element cannot be used in conjunction with the first, second, and third light sensor elements, a summation node, a winner-take-all node, or any of these functions. The broad spectrum light sensor element may be the only light sensor element in the sensor unit.

[0063] Preferably, the local feedback signal includes a brightness output of the broad spectrum photosensor element, and when the brightness output meets a threshold criterion, the local control module is configured to store timing information in the memory module indicative of the time when the threshold criterion was met.

[0064] Preferably, when using a brightness output as set forth in the paragraph above, the one or more photosensor elements include a first photosensor element sensitive to a first portion of the visible spectrum (configured to output a first photosensor element output), a second photosensor element sensitive to a second portion of the visible spectrum (configured to output a second photosensor element output), and a third photosensor element sensitive to a third portion of the visible spectrum (configured to output a third photosensor element output), and when the brightness output meets a threshold criterion, the local control module is configured to cause the memory module to store the first, second, and third photosensor element outputs.

[0065] Preferably, the memory module further includes a switch configured to change a storage location of timing information indicative of the time when the threshold criterion is met, where during a first active sensor phase of the broad spectrum light sensor corresponding to a first frame of the observed scene, the switch is placed in a first position corresponding to the first storage location and during a subsequent second active sensor phase of the broad spectrum light sensor corresponding to a second frame of the observed scene, the switch is placed in a second position corresponding to the second storage location; the local control module is configured to cause the memory module to store in the first storage location the first timing information indicative of the time when the threshold criterion is met for the first active sensor phase and to store in the second storage location second timing information indicative of the time when the threshold criterion is met for the second active sensor phase; the local control module is configured to calculate a difference value corresponding to a difference between the first timing information and the second timing information and compare the difference value to a second global feedback signal, the second global feedback signal including a second threshold criterion; and when the difference value satisfies the second threshold criterion, output an indication from the image sensor indicating that the difference value has satisfied the second threshold criterion.

[0066] Preferably, when using a summing node or a winner-take-all node, the one or more photosensor elements further include a high sensitivity broad spectrum photosensor element, the broad spectrum photosensor element configured to detect brightness of the observed scene and generate a brightness output even when the intensity of incident light is low or very low, and the local control module is further configured to receive and compare a second local feedback signal and a second global feedback signal, the second local feedback signal being a brightness output, the second global feedback signal including a second threshold criterion, and when the second local feedback signal meets the second threshold criterion, the local control module is configured to cause the memory module to store second timing information indicative of a time when the second threshold criterion is met and cause the memory module to store the brightness output. Wherein the imaging sensor further includes a readout module, the readout module is configured to output one or more of the brightness output, the first, second and third photosensor element outputs, the timing information and the second timing information from the imaging sensor based on a result of, or a function of, the comparison with a third global feedback signal received at the readout module.

[0067] Preferably, the readout module is configured to compare the second timing information to a third global feedback signal, the third global feedback signal including a third threshold criterion, and the readout module is configured to output from the imaging sensor either the first, second and third photosensor element outputs and the first timing information, or a brightness output and the second timing information.

[0068] Preferably, the imaging sensor unit is configured to output the stored sensor output and the stored timing information from the imaging sensor unit.

[0069] Preferably, the first, second and third photosensor elements include photosensor elements sensitive to red, green and blue light.

[0070] According to a sixth aspect, there is provided a sensor array comprising a plurality of imaging sensors according to the fifth aspect, the sensor array further comprising a global control module configured to globally control the sensor array, the global control module configured to provide a global feedback signal to each of the plurality of imaging sensors.

[0071] Preferably, the sensor array further includes one or more first type image sensors as defined in claim 16, and the global control module is configured to receive one or more instructions from the first type image sensors to determine whether motion is present in the observed scene between the first frame and the second frame.

[0072] Preferably, the sensor array further includes one or more second type image sensors as defined in claim 9, 10, 15, 17 or 18, and the global control module is configured to activate / turn on the second type image sensors when it determines that motion is present between the first frame and the second frame.

[0073] According to a seventh aspect, there is provided a method of controlling an imaging sensor unit, the sensor unit including a sensor element for detecting a physical property, a local control module coupled to the sensor element, and a memory module coupled to the sensor element, the memory module configured to receive a sensor output from the sensor element, the sensor output being time-variable, the method including the steps of receiving a local feedback signal and a global feedback signal at the local control module, the local feedback signal being based on the sensor output, the global feedback signal including a threshold criterion, comparing the local feedback signal to the threshold criterion, and, if the local feedback signal satisfies the threshold criterion, storing timing information in the memory module indicative of a time when the threshold criterion was met.

[0074] Preferably, the method includes a step of operating the sensor unit according to a cycle, the cycle being defined by an active sensor phase initiated by activating the sensor element from an initial state so that the sensor element begins to detect the physical property and generate a sensor output, and an inactive sensor phase initiated by resetting the sensor element to the initial state.

[0075] Preferably, the method includes the steps of entering an active sensor phase, causing a memory module to store timing information indicating when a threshold criterion was met, and then entering an inactive sensor phase.

[0076] Preferably, the duration of the active sensor phase is limited by a maximum exposure time, and the method further includes, at the maximum exposure time, causing the memory module to store timing information indicative of the maximum exposure time, and entering an inactive sensor phase such that the inactive sensor phase is entered after the maximum exposure time has elapsed.

[0077] Preferably, the method further comprises the step of storing the sensor output when the local feedback signal meets a threshold criterion.

[0078] Preferably, the method further comprises the step of controlling the plurality of imaging sensor units, including the step of synchronously entering an active phase for each imaging sensor unit.

[0079] Some of the methods described herein may be performed by software in machine-readable form on a tangible storage medium, for example in the form of a computer program including computer program code means adapted to execute all the steps of any of the methods described herein when the program can be executed on a computer and the computer program is embodied on a computer readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., but do not include propagated signals. The software is suitable for execution on a parallel or serial processor, and the steps of the method may be performed in any suitable order or simultaneously. The method may be performed using devices that exploit the concept of "programmable logic", such as, for example, programmable logic arrays (PLAs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), and simple programmable logic devices (SPLDs).

[0080] Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0081] [Figure 1] 1 is a schematic diagram of a sensor array including multiple sensor units according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic diagram of a photoreceptor unit according to one embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram illustrating a light intensity invariance function of a photoreceptor unit according to one embodiment of the present invention. [Figure 4] FIG. 4 is a timing diagram illustrating the operating principle of the circuit shown in FIG. [Diagram 5] FIG. 2 is a schematic diagram illustrating a color intensity function of a photoreceptor unit according to one embodiment of the present invention. [Figure 6] FIG. 6 is a timing diagram illustrating the operating principle of the circuit shown in FIG. 5. [Figure 6A]2A-2C are timing diagrams illustrating the operation of circuits according to various embodiments at different times. [Figure 7] FIG. 2 is a schematic diagram illustrating the light intensity invariant function of a photoreceptor unit according to one embodiment of the present invention. [Figure 8] FIG. 8 is a timing diagram illustrating the operating principle of the circuit shown in FIG. [Figure 9] FIG. 4 is a schematic diagram of a further exemplary circuit according to an embodiment of the present invention. [Figure 10] 4 is a further schematic diagram of a further exemplary circuit according to an embodiment of the present invention; [Figure 11] FIG. 4 is a schematic diagram of a further exemplary circuit according to an embodiment of the present invention. [Figure 12] 1 is a schematic diagram of a sensor array according to an embodiment of the present invention; [Figure 13] FIG. 13 is a further schematic diagram of the sensor array of FIG. 12 in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0082] Common reference numbers are used throughout the figures to denote similar features.

[0083] In the following description, the terms "sensor," "pixel," "photodiode," "photosensitive element," and "group of photodiodes" are used interchangeably in applications where light is detected. It should be understood that references to any of these terms may be used in place of any other of these terms.

[0084] FIG. 1 shows a schematic diagram of a number of sensor units 100a-100n that define a sensor array 200. Each sensor unit 100 of the sensor array 200 includes a local control module 102, a number of sensor elements 104, and a memory module 106. The local control module 102 is provided with three inputs: a global feedback signal 108, a local feedback signal 110, and a global clock signal 112. The local control module 102 is configured to control the function of the sensor elements 104. The sensor elements 104 are configured to detect or sense a particular physical signal and provide an output to the memory module 106. The memory module 106 is configured to provide a sensor unit output 114, which may then be stored in a computer or memory of the sensor array 200.

[0085] The sensor array 200 is typically constructed on a substrate composed of a bulk material suitable for supporting all of the sensor's electronic elements, including all necessary signal and power lines. The specific characteristics of the substrate will vary depending on the manufacturing technology.

[0086] In operation, the global control module 202 of the sensor array 200 is configured to provide global control of the sensor units 100a through 100n. This global level of control is complemented by a local level of control at each sensor unit 100a-100n provided by the local control modules 102. The global control module 202 is configured to provide a global feedback signal 108 to each sensor unit 100, and the local control modules 102 are configured to locally control each sensor unit 100 using local feedback 110.

[0087] The global control module 202 is configured to periodically control / adjust the operation of each sensor unit 100a-100n. Each cycle is initiated by a first pulse signal generated by the global control module 202. The first pulse signal is transmitted on the same line as the global clock signal 112 and sent to each local control module 102 of each sensor unit 100a-100n to initiate a sensor capture phase. When this first pulse is received by the local control module 102, the local control module 102 transmits a first control signal to the sensor element 104. This first control signal causes the sensor element 104 to exit the reset / standby / low power state and transition to a detection / active state, initiating the sensor capture phase.

[0088] Once the end of the sensor capture phase is reached, the cycle is complete. This is signaled to the sensor units 100a-100n by the global control module 202, which writes a second pulse to each local control module 102 on the same line carrying the global clock signal 112. Data is then collected from the sensor elements 104 via the sensor unit outputs 114 during the sensor read phase. The second pulse causes the local control modules 102 to send a reset / stop signal to the sensor elements 104, forcing them into a low power / non-operational state until a new cycle begins.

[0089] In some embodiments, upon reaching an optimal sensor state, as determined by the satisfaction of some condition / criterion in each individual sensor unit 100, the memory module 106 of the sensor unit 100 is configured to send a termination signal to the control module 102 of the sensor unit 100 using the same line used to send the local feedback signal 110. This termination signal triggers the local control module 102 to send a reset / stop signal to the sensor element 104, forcing the sensor element into a low power / non-operating state until a new cycle begins. The achievement of the condition / criterion is described in more detail below.

[0090] Each of the multiple sensor elements 104 may be the same type of sensor element. For example, each of the multiple sensor elements 104 may be an optical sensor element, each sensitive to a different wavelength of visible light, designed for use when strong or weak optical signals are present, providing a fast or slow light-to-voltage conversion scheme optimized to reduce electromagnetic noise, thermal noise, and the like. In another example, one or more of the multiple sensor elements 104 may be an optical sensor element sensitive to wavelengths in the visible wavelength range, and / or one or more of the multiple sensor elements 104 may be a sensor element sensitive to wavelengths in a different wavelength range within the electromagnetic spectrum, such as the infrared range of the electromagnetic spectrum. It should be understood that sensor elements sensitive to gamma ray, x-ray, ultraviolet, microwave, and radio wave ranges of the electromagnetic spectrum may also be used if such sensors are technically compatible and can be implemented on the same substrate.

[0091] Alternatively, each of the plurality of sensor elements 104 may include a group of different sensor elements configured to detect a different type of signal or different type of information extracted from the input signal.

[0092] In the multiple sensor elements 104, a combination of light / electromagnetic radiation sensors, thermal sensors, pressure sensors, chemical sensors, etc. may be used together if they can be implemented on the same substrate.

[0093] 1 may be the same sensor unit 100 including the same plurality of sensor elements 104. Alternatively, one or more of the sensor units 100a-100n may include different sensor units including different combinations of sensor elements 104, as described above. This allows the different sensor units 100a-100n in the sensor array 200 to perform different functions and tasks. Each of the sensor units 100a-100n is controlled at least in part locally by the local control module 102 depending on the particular function of each of the sensor units 100a-100n.

[0094] The sensor units 100a-100n are at least partially controlled locally by the control module 102 of each sensor unit 100, such that the sensor units 100a-100n are configured to function asynchronously depending on the functionality of each sensor unit 100a-100n.

[0095] The function of the sensor units 100a to 100n will now be described. As mentioned above, the operation of the sensor unit 100 is based on the repetition of a series of actions arranged in cycles. Each cycle is initiated by a first pulse signal transmitted on the same line as the global clock signal 112, initiating a sensor capture phase. When this first pulse is received by the local control module 102, the local control module 102 transmits a first control signal to the sensor element 104. This first control signal transitions the sensor element 104 from a reset / standby / low power state to a detection / active state.

[0096] In the detect state, sensing is initiated and the sensor elements 104 are configured to provide a sensed output to the memory module 106. The sensor elements 104 are configured to detect a physical signal, for example, by converting the physical signal into an electrical signal. How each sensor element of the sensor elements 104 reacts to the physical signal varies depending on its respective characteristics. The electrical signals generated by each of the sensor elements 104 together form a sensed output, which is continuously monitored by the memory module 106. The electrical signal can be, for example, a voltage or a current.

[0097] The memory module 106 is configured to provide a local feedback signal 110 to the local control module 102, where the local feedback signal 110 is based on a sensing output from the sensor element 104. The local feedback signal 110 may be equivalent to a sensing output, and the memory module 106 simply returns the sensing output from the sensor element 104 to the local control module 102 as the local feedback signal 110. The memory module 106 uses the electrical signal from the sensor element 104 to transmit the local feedback signal 110 to the control module 102. Thus, the local feedback signal 110 varies depending on the type, strength, or characteristics of the physical signal that is converted into the electrical signal.

[0098] The global feedback signal 108 is generated by the global control module 202 and is also provided to the local control module 102. The control module 102 combines the local feedback signal 110 generated by the memory module 106 with the global feedback signal 108 generated by the global control module 202 to generate a second control signal. The second control signal is used to adjust the operating parameters and / or states (e.g., threshold voltages, control voltages, configuration signals, etc.) of the sensor elements 104 and to indicate to the memory module 106 when an optimal sensor state has been achieved or the end of the sensor capture phase has been reached and data from the sensor elements 104 has been captured and used to generate the sensor unit output 114. As previously mentioned, the end of the sensor capture phase is indicated by the global control module 202 with a second pulse on the line carrying the global clock signal 112. The end of the sensor capture phase may correspond to a maximum sensor exposure time, which is a programmable time interval.

[0099] How and when the second control signal is generated varies depending on the implementation of the local control module 102. In particular, the local control module 102 uses the global feedback signal 108 and the local feedback signal 110 in a calculation, comparison, or other computation to determine how and when to control the sensing element 104 and / or the memory module 106 with the second control signal. The local control module 102 generates the second control signal when the global feedback signal 108 and the local feedback signal 110 meet a particular condition or criterion according to the comparison or computation. Satisfying the condition or criterion indicates that an optimal sensor state has been achieved.

[0100] When a condition or criterion is met, the second control signal is configured to cause the sensor element 104 and / or the memory module 106 to perform an action. For example, the second control signal may cause the memory module 106 to generate a sensor unit output 114 for storing the sensing data generated by the sensing element 104. The sensor unit output 114 may then be transmitted to, for example, a global memory or an external computer.

[0101] Although the global clock signal 112 is described above as being transmitted to the local control module 102 over a single line, it should be understood that the global clock signal 112 may be implemented by multiple lines. For example, the global clock signal 112 may include a first line (i.e., a wire) that carries first and second pulses used to switch the sensor unit 100 between a low power / non-operating state and an active / detecting state, and one or more additional lines that define the parameters or types of activities to be initiated or performed by the sensor unit 100. References to the global clock signal 112 include both of these possibilities.

[0102] At the same time that the sensor data is stored or captured by the memory module 106 and the sensor unit output 114 is generated, the global clock signal 112, or a subset thereof, is also stored or captured by the memory module 106. In particular, the global clock signal 112 includes a timing signal V(t) that is dependent on time or a function thereof. Capturing or recording a value of the timing signal V(t) at the same time that the data from the sensor elements 104 is captured can indicate the time at which the data is captured. This time data and the sensor data are used to generate the sensor unit output 114. Thus, the external computer or global memory to which the sensor unit output 114 is sent can have access to the data captured from the sensor elements 104 and the time of capture. In effect, the timing signal of the global clock signal 112 is used by each sensor unit 100 to indicate the local time at which the data capture criteria / conditions were met in the local control module 102 of the sensor unit 100, i.e., the time at which the data from the sensor elements 104 was captured and stored. This mechanism allows sensing to be optimized in terms of signal-to-noise ratio or other quality metrics used in the application, based on local characteristics of the incident light, rather than based on global parameters / characteristics (e.g. integration / exposure time) as in most CMOS sensors.

[0103] When the end of the sensor capture phase is reached, the cycle is complete, which is signaled to the sensor unit 100 by the global control module 202, which writes a second pulse on the line carrying the global clock signal 112 to the local control module 102, as described above.

[0104] In one example, the memory module 106 includes a sample and hold circuit that reads the electrical signals generated by each sensor element of the sensor elements 104 and stores and makes them available via the sensor unit output 114 when the sensor elements 104 reach an optimal sensor state or the end of a sensor capture phase.

[0105] FIG. 2 shows a specific implementation of the sensor unit 100 according to FIG. 1. In FIG. 2, the sensor unit 100 is a photoreceptor unit 100. The photoreceptor unit 100 is a type of visual sensor unit or imaging sensor unit. The photoreceptor unit 100 includes a local control module 102, a number of sensor elements (the sensor elements are photosensitive elements 104a), and a memory module 106. The local control module 102 is provided with three inputs: a global feedback 108, a local feedback 110, and a global clock 112. The local control module 102 is configured to control the function of the photosensitive elements 104a. The photosensitive elements 104a are configured to detect light or its components and provide a sensed output to the memory module 106. The memory module 106 is configured to provide a photoreceptor unit output 114, for example, for storage in a global memory or for use in a computer. Thus, the components of the photoreceptor unit 100 in FIG. 2 are the same as those in FIG. 1, and the sensor elements 104 are photosensitive elements 104a. However, it is understood that different types of sensors may be used in place of the photosensitive element 104a, and therefore the following description pertains to all types of sensors, not just the photosensitive element 104a (sensor) specified below.

[0106] The function of the photoreceptor unit 100 of FIG. 2 will now be described. The local control module 102 uses a global feedback signal 108, a global clock 112, and a local feedback 110 to control the operation of the photosensitive elements 104a and the memory module 106. The sensed output from the photosensitive elements 104a is a collection of voltages (PixOut) corresponding to a single voltage from each photosensitive element 104a. These voltages are sent to the memory module 106. The memory module 106 (which may be a sample and hold circuit) is configured to store the voltages when the conditions / criteria are met, as outlined above. The memory module 106 makes the stored voltages available as the photoreceptor unit output 114. This output can be read and sent to an output interface of a device such as a computer or external memory for eventual use.

[0107] The operation of the photoreceptor unit 100 is based on the same repetition of the cycle described above with reference to FIG. 1. Each cycle is initiated by a first pulse to initiate an image acquisition phase. When this pulse is received by the local control module 102, the local control module 102 sends a first control signal to the photosensitive element 104a. This causes the photosensitive element 104a to exit the reset / standby / low power state and enter the detection state. In the detection state, the photosensitive element 104a is configured to detect light by converting the incident light beam into a voltage that changes over time. For a photosensitive sensor such as the photosensitive element 104a, the changing voltage is often represented by some discharge curve caused by the incident photons. The voltage generated by the photosensitive element 104a forms the sensed output and is continuously monitored by the memory module 106. The memory module 106 uses the voltage from the photosensitive element 104a to send a local feedback signal 110 to the control module 102. In some implementations, the local feedback signal 110 is directly proportional to or equal to the voltage generated by the photosensitive element 104a. The control module 102 is provided with the local feedback signal 110 and the global feedback signal 108 generated by the global control module 202. The local feedback signal 110 and the global feedback signal 112 are then used in a comparison or calculation depending on the particular implementation of the photoreceptor unit 100 to determine if a particular condition / criterion has been met that indicates that an optimal sensor state has been achieved. If this condition / criterion is met, the local control module 102 generates a second control signal configured to control the photosensitive element 104a by outputting the sensor unit output 114 and / or controls the memory module 106 to store data from the photosensitive element 104a.The memory module 106 may be a sample and hold circuit, in which case, when a condition / criterion is met, the local control module 102 controls the sample and hold circuit to store a voltage corresponding to the sensed output of the photosensitive element 104a and generate the sensor unit output 114.

[0108] The sensor unit output 114 is sent to, for example, a global memory or a computer. At the same time that the voltage values ​​are stored by the memory module 106, the global clock signal 112, or a subset thereof, is also stored or captured by the memory module 106. In particular, the global clock signal 112 includes a timing signal V(t) that is time-dependent or a function of time. Capturing or recording the value of the timing signal V(t) at the same time that the data from the photosensitive element 104a is captured can indicate the time that the data was captured. This time data and the voltage data are used to generate the sensor unit output 114. Thus, the computer or global memory to which the sensor unit output 114 is sent can access the voltage data (sensor data) and an indication of the time that it was sampled.

[0109] The photoreceptor unit 100 may be one of a plurality of photoreceptor units 100 forming part of an array 200, as shown in FIG. 1. The respective photoreceptor unit outputs 114 from each of the plurality of photoreceptor units 100 together form an acquired image / frame. This acquired image / frame is transferred from the array 200 via output ports Line1 and Line2. The global control module 202 of the array 200 is configured to control each photoreceptor unit 100 in a series of cycles. These cycles include image acquisition phases, as explained above. At the end of the image acquisition phase for the entire array 200, an image readout operation is performed. The duration of the image acquisition phase is defined by a programmable time interval, which fixes the maximum exposure time of each photoreceptor unit 100 and each photosensitive element 104a. The time interval is stored in a register of the global control module 202. The time required for the readout phase depends on the size of the array, the speed of the data transfer lines, and other operating parameters. At the beginning of each cycle, the global control module 202 activates an image acquisition phase by generating a first pulse on the line of the global clock signal 112, as described above. Once the first pulse is detected, each photoreceptor unit 100 begins to convert the incident light independently and autonomously, so that the global control module 202 and the other photoreceptor units 100a-100n do not need to be directly involved in acquiring data from a particular photoreceptor unit 100. To inform all photoreceptor units 100a-100n that the image acquisition phase has ended (and the maximum exposure time has elapsed), the global control module 202 generates another second pulse on the line used for the global clock signal 112. The global control module 202 also generates a global feedback signal 108 that is used by each photoreceptor unit 100 at the beginning of an image acquisition phase. The global feedback signal 108 may differ depending on the type of photoreceptor unit 100. Notably, there may be multiple types of photoreceptor units 100 in the same array 200.Once the image acquisition phase is complete, the global control module 202 initiates the readout phase. During the readout phase, the voltage values ​​stored by each memory module 106 of each photoreceptor unit 100 are used to generate the respective sensor unit outputs 114 and send them to the output ports (via Line 1 and Line 2). As previously mentioned, the voltage values ​​stored by each memory module 106 reflect the data captured at the optimal state of the respective photoreceptor unit 100. By adding appropriate readout logic and circuitry, the image acquisition and readout phases can be arranged according to a pipeline architecture and run in parallel to increase the number of images / frames generated per unit time.

[0110] Additionally, if the global control module 202 is equipped with appropriate processing circuitry, it can read or access the output values ​​generated by each photoreceptor unit 100 to regulate the operation of the photoreceptor units 100a through 100n by generating a global feedback signal 108 at the start of each cycle. The global feedback signal 108 can be a static signal that does not change during an image acquisition phase, or it can be a variable signal that can change during an image acquisition phase. An example of a static global feedback signal is represented by the global threshold voltages used by the photoreceptor units in FIGS. 3 and 5. These static signals are determined from the average illumination of the scene to be captured to minimize noise and improve resolution. The global control module 202 can also operate in conjunction with an external computer, microcontroller, programmable logic, microprocessor, or the like to generate an optimal global feedback signal. In this case, the external computer unit monitors / reads / accesses the stream of images / frames generated by the device including the photosensitive array 200 and processes them to generate an optimal global feedback signal, which is then stored in an internal memory unit of the global control module 202 and transmitted to the photoreceptor 100 during the image acquisition phase according to a programmable time scale.

[0111] The timing signal V(t), included as a component of the global clock signal 112, may also be considered as a type of global feedback signal. Here, the timing signal V(t) will be described in more detail. The timing signal V(t) is a signal that can be used to measure the time of events occurring in the circuit (as explained above). The timing signal V(t) can also be considered as a global feedback signal because it is generated and adjusted by the global control module 202 to set an appropriate time resolution in the measurement data of the photoreceptor units 100 that is appropriate for the characteristics of the observed scene. This will be explained in more detail later with reference to a specific example. However, in general, the timing signal V(t) included in the global clock signal 112 is an electrical signal that exhibits a known relationship with the time within an image acquisition phase (the integration time between the start and end of each acquisition cycle), so that by sampling the timing signal V(t), a simple inversion operation can be performed to calculate the time t at which data is recorded by the memory module 106 of a particular photoreceptor unit 100. In the following example, the timing signal V(t) is implemented with a rising linear ramp that starts at a low value at the start of the image acquisition phase and reaches a maximum value at the end of the image acquisition phase. Such a shape of the timing signal V(t) is used for illustrative purposes only, and in general, any signal shape (linear or nonlinear, increasing or decreasing, etc.) that allows the calculation of the value of the time at which the timing signal V(t) is sampled can be used. Thus, the timing signal V(t) is a signal that can be used as a source for measuring the time of an event. At the same time, the shape of the function of the timing signal V(t) can also be changed to adjust the resolution of the time measurement. The shape of the timing signal V(t) is selected such that at a particular time t, a large change Δt in t corresponds to a small change ΔV in the timing signal V(t), but the time resolution at that particular time is low because a small increase in the measured electrical signal timing signal V(t) is equivalent to a large elapsed time Δt. This means that a small noise on the timing signal V(t) is amplified when the timing signal V(t) is inverted to calculate t.Conversely, if the shape of the timing signal V(t) is selected such that a small change in time Δt corresponds to a large change ΔV in the timing signal V(t), the time resolution is high because the large change ΔV is equal to a small period Δt. In this case, noise on the timing signal V(t) is attenuated when the timing signal V(t) is inverted to calculate t. In an exemplary implementation, the timing signal V(t) included in the global clock signal 112 is described by a linear shape / function, meaning that the time resolution is constant over the integration period, while in other implementations, the function describing the timing signal V(t) is non-linear, meaning that the time resolution can vary over the integration period and may be more or less precise at certain times than at other times.

[0112] Other global feedback signals can be generated to improve the overall performance of the device including the photosensitive array 200 based on overall operating conditions such as sensor temperature and power consumption. For example, the voltage sources used to implement components such as the comparators of FIG. 3 and the winner-take-all (WTA) circuit of FIG. 5 can be modified based on such feedback signals to improve speed, accuracy, or power consumption. These circuits and diagrams are described in more detail below. The local feedback signals generated by the memory module 106 of FIG. 1 are combined with the global feedback signals to adjust the performance of each photoreceptor unit 100.

[0113] The photosensitive array 200 can provide improved signal-to-noise ratio and energy saving advantages, as will be described below.

[0114] Specific examples of photosensitive arrays 200 and photoreceptor units 100 will now be described with reference to FIGS.

[0115] 3 is a schematic diagram illustrating a first exemplary implementation of a photoreceptor unit 100-1. In this implementation, the local control module 102 includes a comparison circuit module 102-1, and the photosensitive element 104 is a group of photodiodes 104-1 including a red light sensitive diode 104-1a, a green light sensitive diode 104-1b, and a blue light sensitive diode 104-1c. The memory module 106 is a sample and hold circuit 106-1, the global feedback signal 108 is a global threshold 108-1, the local feedback signal 110 is a sum 110-1 of the sensor outputs from the photodiodes 104-1, and the global clock signal is a voltage signal V(t) 112-1 that is a function of time. The sensor unit output 114 is not shown, but will also be described below. The comparison circuit module 102-1 is a component of the local control module 102. It should be understood that other circuitry not shown in Figure 3 may also be included in the local control module to control local operation of the photoreceptor 100-1, for example using the global clock signal 112. The local control module 102 may be configured to locally generate a reset signal from the global clock signal to initiate a reset of the photodiode 104-1. The local control module 102 may also generate a "save" signal to send an instruction to the sample and hold circuit 106-1 to capture and store data when the integration phase ends (as indicated by the global clock signal 112).

[0116] Also shown in Figure 3 is circuitry 150-1 that may be used during a readout phase. In particular, when a particular photoreceptor unit 100 at location (I,j) of the sensor array 200 needs to be read, the select SEL signal corresponding to that photoreceptor unit 100 is asserted and the Rth, Gth, Bth, and Vth signals from the corresponding line buffers, shown as transistors in Figure 3 and connected to a power supply voltage VDDR, are copied to the corresponding bit lines blr, big, bib, and blv. These bit lines are grouped with the bit lines of other photoreceptor units of the same array 200 and are typically arranged in columns.

[0117] As shown in FIG. 3, in operation of the first exemplary implementation of the photoreceptor unit 100, during an image acquisition phase, the voltages of the red 104-1a, green 104-1b, and blue 104-1c photodiodes are sampled in the sample and hold circuit 106-1 and summed at a summing node to form a sum 110-1 that at least partially represents a local feedback signal. The sum 110-1 of the three voltages from the photodiodes 104-1 is input to the comparison circuit module 102-1. A global threshold 108-1 is also input to the comparison circuit module 102-1. The comparison circuit module 102-1 compares the sum 110-1 to the global threshold 108-1 using a comparator. When the sum 110-1 reaches or exceeds the global threshold 108-1, the comparison circuit module 102-1 transmits the second control signal described above along line 116-1, as shown in FIG. 3. This second control signal controls the sample and hold circuit 106-1 to store the current values ​​of the voltages of the red 104-1a, green 104-1b, and blue 104-1c photodiodes, as well as the voltage value of the timing signal V(t) 112-1. This is done, for example, by opening the switches of the sample and hold circuit 106-1 shown in FIG. 3 to store the voltage value of each corresponding signal. Thus, the operation of the comparator module 102-1 when the global threshold 108-1 is exceeded serves to initiate the capture of sensor data. In one example, the local feedback signal (sum 110-1 in the case of FIG. 3) is a discharge curve of a voltage that decreases during the integration phase. Thus, when this discharge curve is equal to or below the global threshold 108-1, the comparator changes its output.

[0118] In relation to the general description above, the condition / criterion is that the sum 110-1 passes the global threshold 108-1. As mentioned above, the sample and hold circuit 106-1 is also controlled by a global signal generated by the global control module 202 or a local signal generated when a second pulse indicating the end of the image acquisition phase (i.e., the integration phase) is received over the line carrying the global clock signal 112 from the global control module 202. This local signal is generated to override the output of the comparator in the comparison circuit module 116-1 to save the voltages of the three photodiodes 104-1 and the timing signal V(t) at the end of the image acquisition phase, even if the sum 110-1 does not exceed the global threshold 108-1 during the image acquisition phase.

[0119] Thus, the sample and hold circuit 106-1 is instructed to save and capture the voltage value of each photodiode 104-1 when the sum 110-1 crosses the global threshold 108-1 or when the end of the image acquisition phase is reached.

[0120] Upon capturing the voltage data and timing signal V(t) 112-1 of each photodiode 104-1, the sample and hold circuit 106-1 may be configured to send a termination signal to the comparison circuit module 102-1 and associated circuitry of the sensor unit 100 using the same lines used to send the local feedback signal 110-1. This termination signal triggers the local control module 102, including the comparison circuit module 102-1 and associated circuitry, to send a reset / stop signal to the photodiode 104-1, forcing it into a low power / non-operating state until a new cycle is initiated in response to a first pulse sent by the global control module 202.

[0121] The operation of the photoreceptor unit 100 shown in Figure 3 will now be described in more detail. First, during the low power / off state, i.e. the reset phase (RES=H), the photodiode 104-1 is precharged to a voltage V r . After reset, when a first control signal is provided to photodiodes 104-1 and the image acquisition phase begins, photodiodes 104-1 begin to integrate the incident light and the voltage across each photodiode 104-1 decreases approximately linearly with the intensity of the light falling on it, as described by the following equation: V(k)=(l K / C K )t, and i=R,G,B (1) Here, C k is the capacitance of the reverse-biased photodiode, I k is the photogenerated current and t is the exposure / integration time of the photodiode.

[0122] After the reset phase, i.e. at the start of the image acquisition phase, a timing signal V(t), which forms part of the global clock signal 112-1, becomes active and tracks the time elapsed since the start of the image acquisition phase. The voltages across the three photodiodes 104-1 are monitored by a comparison circuit module 102-1 which calculates a sum 110-1 (S) of the three voltages received from the red 104-1a, green 104-1b and blue 104-1c photodiodes and compares it to a global threshold value 108-1 (TH), as shown in (2). s = (V r -V R )+(V r -v G )+(V r -v B ) (2) Here, V R , V G , and V Bare the sampled voltages of the red 104-1a, green 104-1b, and blue 104-1c photodiodes, respectively. During the exposure / integration time, the voltage of each photodiode 104-1 decreases over time as it is discharged, so the value of S starts to increase from zero. At time T0, when the sum 110-1 (S) reaches the global threshold 108-1 (TH), the sample and hold circuit 106-1 controlled by the comparison circuit module 102-1 stores the voltage signals of the red 104-1a, green 104-1b, and blue 104-1c photodiodes along with the voltage signal V(t) 112-1. At the end of operation, the photoreceptor unit 100 transmits three signals Rth, Gth, and Bth, associated with the red 104-1a, green 104-1b, and blue 104-1c photodiodes, respectively. These values ​​are normalized to the global threshold 108-1 TH. R(To)+G(To)+B(To)=Rth+Gth+Bth=TH (3)

[0123] The fourth voltage signal Vth is related to the time (To) at which the signal was stored by the sample and hold circuit 106-1. In this exemplary implementation, the global threshold 108-1 (TH) serves as a global signal that sets the local exposure time of the photodiodes and thus the pixels of the array 200, the local parameter being represented by the sum 110-1 (S). The second global signal is the timing signal V(t), the value of which is stored in the sample and hold circuit 106-1 as T, a voltage ramp / curve that represents the time stamp of the pixel.

[0124] Once the image acquisition phase is over, the values ​​stored in the sample and hold circuit 106-1 are output via the sensor unit output 114 and read out in the readout phase. In the readout phase, all sensor unit outputs 114 are acquired from each photoreceptor unit 100a-100n, making the RGB values ​​of each pixel in the array 200 across the scene available for post processing or other end use. In addition to the RGB values, a value of V(T0) is acquired which indicates the time T0 at which each photoreceptor unit 100 crossed the global threshold 108-1(TH). Knowing V(T0), the absolute RGB values ​​for each photoreceptor unit 100 can be calculated using each set of RGB values ​​corresponding to the photoreceptor unit 100. The absolute RGB values ​​can be calculated using the following formula: △V x *T i / T 0、 Here, △V x =(V r -V x ) (4) where x=R, G, or B and V x corresponds to the RGB voltage value obtained from the sensor unit output 114, and T iis the global integration / exposure time. T0 can be calculated by inverting the timing signal V(T0) as described above. This allows the array 200, and any device incorporating it, to discern useful color information in both bright and dark areas of a scene. For example, in a dark area of ​​a scene captured by the first photoreceptor unit 100, the value of the timing signal V(T0) may be relatively large, meaning that a longer time T0 was required for the sum 110-1(S) to reach the global threshold 108-1(TH). In a bright area of ​​the same scene captured by the second photoreceptor 100, the value of the timing signal V(T0) may be relatively small, meaning that a shorter time T0 was required for the sum 110-1(S) to reach the global threshold 108-1(TH). By normalizing these values ​​using the different values ​​of T0 returned by each photoreceptor in the sensor array, an image can be obtained with RGB values ​​that faithfully represent the full dynamic range of light conditions present in a scene, even when very bright and very dark areas are present at the same time.

[0125] FIG. 4 shows a timing graph illustrating the process described above with reference to FIG. 3. In particular, after a reset phase, each of the red 104-1a, green 104-1b, and blue 104-1c photodiodes is set to an initial voltage V(r). When the image acquisition phase begins, the timing signal V(t) carried by / on the global clock signal 112-1 line is initiated, and the voltage of each of the photodiodes 104-1 begins to discharge approximately linearly according to Equation 1 above, as shown by the R, G, and B lines in FIG. 4. At the same time, the sum 110-1(S), calculated according to Equation 2 above, increases according to the sum of the voltage decreases of the RGB voltage values. At time T0, the sum 110-1S reaches the global threshold 108-1 TH. At this time (T0), the sample and hold circuit 106-1 is instructed by the comparator module 102-1 to capture and store the R, G, and B voltage values ​​from the red 104-1a, green 104-1b, and blue 104-1c photodiodes, respectively. These values ​​are shown as the Rth, Gth, and Bth lines in FIG. 4. At the same time, the sample and hold circuit 106-1 also stores the value of the timing signal V(t) at t=T0, shown as the Vth line in FIG. 4. The Rth, Gth, Bth, and Vth values ​​are then output as part of the sensor unit output 114 and read in a readout phase following the image capture phase.

[0126] 5 is a schematic diagram illustrating a second exemplary implementation of the photoreceptor unit 100-2. In this implementation, the local control module 102 is a comparison circuit module 102-2, the photosensitive element 104 is a group of photodiodes 104-2 including a red light sensitive diode 104-2a, a green light sensitive diode 104-2b, and a blue light sensitive diode 104-2c. The memory module 106 is a sample and hold circuit 106-2, the global feedback signal 108 is a global threshold 108-2, the local feedback signal 110 is a winner-take-all signal 110-2 of the sensor output from the photodiodes 104-1, and the timing signal V(t) 112-2 is a signal carried by a global clock signal line and is a function of time. The sensor unit output 114 is not shown, but will also be described below.

[0127] Also shown in Figure 5 is circuitry 150-2 that may be used during a readout phase. In particular, when a particular photoreceptor unit 100 at location (I,j) of the sensor array 200 needs to be read, the select SEL signal corresponding to that photoreceptor unit 100 is asserted and the Rth, Gth, Bth, and Vth signals from the corresponding line buffers, shown as transistors in Figure 5 and connected to a power supply voltage VDDR, are copied to the corresponding bit lines blr, big, bib, and blv. These bit lines are grouped with the bit lines of other photoreceptor units of the same array 200 and are typically arranged in columns.

[0128] As shown in FIG. 5, in operation of the second exemplary implementation of the photoreceptor unit 100, during an image acquisition phase, the voltages of the red 104-2a, green 104-2b, and blue 104-2c photodiodes are sampled by the sample and hold circuit 106-2, and a winner-take-all operation is performed at the winner-take-all node (WTA) to pass the signal from the photodiode 104-2 with the largest voltage change from the initial reset voltage V(r) to the comparison circuit module 102-2. The "winner" voltage signal forms the local feedback signal 110-2. Thus, the local feedback signal 110-2 is substantially equal to the voltage signal of the one of the red 104-2a, green 104-2b, and blue 104-2c photodiodes that exhibits the largest change since reset. This local feedback signal 110-2 is sent to the comparison circuit module 102-1. A global threshold value 108-2 is also sent to the comparison circuit module 102-2. The comparison circuit module 102-2 uses a comparator to compare the winner-take-all signal 110-2 with the global threshold 108-2. As previously mentioned, the WTA node is a circuit that compares the voltage signals of the three photodiodes 104-2 and transmits or copies the voltage signal of the photodiode with the greatest change (i.e., the voltage of the photodiode that discharges faster) to the output of the WTA node to form the local feedback signal 110-2. The combination of the WTA node and the comparator 102-2 is equivalent to a circuit that simultaneously compares all input signals with the global threshold 108-2.

[0129] When the output of the winner-take-all 110-2 reaches or exceeds the global threshold 108-2, the comparison circuit module 102-2 transmits the aforementioned second control signal along the line 116-2, as shown in FIG. 5. This second control signal controls the sample-and-hold circuit 106-2 to store the current values ​​of the voltages of the red 104-2a, green 104-2b, and blue 104-2c photodiodes and the voltage value of the voltage signal V(t) 112-2. This is done, for example, by opening the switches of the sample-and-hold circuit 106-2 shown in FIG. 5 to store the voltage value of each corresponding signal. Thus, the operation of the comparison circuit module 102-2 when the global threshold 108-2 is exceeded serves to initiate the capture of sensor data. In relation to the general description above, the condition / criterion is that the winner-take-all local feedback signal 110-2 passes the global threshold 108-2. Apart from this, the features of the second exemplary implementation shown in FIG. 5 are similar or equivalent to the features of the first exemplary implementation shown in FIG. 3. This is despite the fact that the sample and hold circuit 106-2 also forms part of the local control module 102. The local control module 102 is controlled by a global signal generated by the global control module 202 of the array 200, or a local signal generated when a second pulse generated by the global control module 202 is received locally over the global clock line, to override the output of the comparator of the comparison circuit module 102-2 if the "win" signal 110-1 does not exceed the threshold 108-2 during the image acquisition phase, and to store the voltages of the three photodiodes and the timing signal V(t) at the end of the image acquisition phase.

[0130] Upon capturing the voltage data and voltage signal V(t) 112-2 of each photodiode 104-2, the sample and hold circuit 106-2 may be configured to send a termination signal to the local control module 102, including the comparison circuit module 102-2 and associated circuitry of the sensor unit 100, using the same lines used to send the local feedback signal 110-2. This termination signal triggers the comparison circuit module 102-2 and associated circuitry of the local control module 102 to send a reset / stop signal to the photodiode 104-2, forcing the photodiode into a low power / non-operating state until a new cycle begins.

[0131] The operation of the photoreceptor unit 100 shown in FIG. 5 will now be described in more detail. First, in the low power / off state, i.e., in the reset phase (RES=H), the three photodiodes 104-2 are precharged to a voltage Vr. After the reset is released, the image acquisition phase begins with a first control signal. This causes the three photodiodes 104-2 to start integrating light as described above with reference to FIGS. 4 and 5. The voltage across each photodiode 104-2 starts to decrease approximately linearly with the intensity of the light falling on it. After the reset, the voltage signal V(t) 112-2 becomes active and tracks the time elapsed from the end of the reset phase, i.e., to the start of the image acquisition phase. The comparator circuit module 102-2 monitors the voltages of the three photodiodes during the exposure time. As soon as any of the three signals (R, G, B voltages) reaches the global threshold 108-2 (Vth), the voltage signals R, G, B of the red 104-2a, green 104-2b, and blue 104-2c photodiodes are stored by the sample-and-hold circuit 106-2 along with the voltage signal 112-2 V(t). The photoreceptor unit 100 shown in FIG. 5 performs a winner-take-all (WTA) operation between the three photodiodes 104-2. That is, the brightest photodiode is the one that reaches the global threshold 108-2 (Vth) first. The other two non-dominant voltage signals avoid saturation because their voltage levels are always greater than TH. The three signals (Rth, Gth, Bth) from the photodiode 104-2 are stored and used to form the sensor unit output 114. These signals are not normalized. In particular, if we assume that R (red) is the brightest signal, then Rth is equal to the global threshold 108-2 (TH). The timestamp of a pixel is denoted by Vth. Similar to the first exemplary implementation of FIG. 3, if V(T0), i.e., the timing signal V(t) when the WTA output signal (local feedback signal 110-2) reaches the global threshold 108-2, is known, then the absolute RGB values ​​of each photoreceptor unit 100 can be calculated using each set of RGB values ​​corresponding to the photoreceptor unit 100. In fact, the absolute RGB values ​​can be calculated using Equation 4 as described above.

[0132] FIG. 6 shows a timing diagram illustrating the process described above with reference to FIG. 5. In particular, after a reset phase, the red 104-2a, green 104-2b, and blue 104-2c photodiodes are set to an initial voltage VI. When the image acquisition phase begins, the global clock signal, represented by the timing signal V(t), is started and the voltage of each photodiode 104-2 starts to discharge approximately linearly according to Equation 1 above, as shown by the R, G, and B lines in FIG. 6. At time T0, the winning signal reaches the global threshold 108-2TH. In the example of FIG. 6, the winning signal is the red signal R. At this moment (T0), the sample and hold circuit 106-2 receives instructions from the comparison circuit module 102-2 to read the values ​​from the red 104-2a, green 104-2b, and blue 104-2c photodiodes, as shown by the Rth, Gth, and Bth lines in FIG. 6. At the same time, the sample and hold circuit 106-1 also stores the value of V(t) at V(t)att=T0, shown as the Vth line in Figure 6. The values ​​of Rth, Gth, Bth, and Vth are obtained from the portion of the sensor unit output 114 that is read out during the read phase.

[0133] The second exemplary implementation of the photoreceptor unit 200 shown in Figures 5 and 6 is advantageous compared to standard techniques because it prevents the photodiode 104-2 from saturating due to local mechanisms, regardless of the illumination level. This property is illustrated in Figure 6A, which shows an example of two photoreceptors 100-2 of the same type with the same color components but different intensity conditions. In particular, the first of the two photoreceptors is brighter than the second photoreceptor. The first photoreceptor is associated with values ​​R1th, G1th, and B1th, and the second photoreceptor is associated with values ​​R2th, G2th, and B2th. In both the first and second photoreceptors, R is the dominant color component, and therefore the R values ​​of each of the first and second photoreceptors reach the threshold value TH first due to a winner-take-all approach. This triggers a sample and hold circuit to store the following signals: R1th, G1th, B1th, and timing signal V(T0) for the first photoreceptor, and R2th, G2th, B2th, and timing signal V(T1) for the second photoreceptor. In this case, the time it takes for the RGB values ​​of the second photoreceptor to reach the threshold TH is longer than that of the first photoreceptor, T1>T0. In other words, the first photoreceptor is exposed to brighter light than the second photoreceptor. The ratio of T1 / T○ is directly proportional to the ratio of the light intensities falling on the two photoreceptors. Furthermore, we found that the relationship between the two triplets R1th, G1th, B1th and R2th, G2th, B2th is the same. This means that the color information of both the first and second photoreceptors is preserved regardless of the different light intensities encoded by V(T0) and V(T1). Therefore, even if the first photoreceptor is exposed to much brighter light than the second photoreceptor, we can easily obtain the RGB information of both photoreceptors, and therefore easily compare the true color information of both photoreceptors. Therefore, only four signals, R, G, B, and V(T), are needed to obtain the absolute information of each photoreceptor over a wide dynamic range.

[0134] 7 is a schematic diagram illustrating a third exemplary implementation of the photoreceptor unit 100-3. In this implementation, the local control module 102 includes a comparison circuit module 102-3, and the photosensitive element 104 is a group of photodiodes 104-3, including a photodiode 104-3a sensitive to red light, a diode 104-3b sensitive to green light, a diode 104-3c sensitive to blue light, and another photodiode 104-3d sensitive to white light. The memory module 106 is a sample and hold circuit 106-3, the global feedback signal 108 is a global threshold 108-3, and the global clock signal is a voltage signal V(t) 112-3 that is a function of time. The sensor unit output 114 is not shown, but will also be described below. This third exemplary implementation differs from the first and second exemplary implementations described above in that the local feedback signal 110 is not sampled or obtained directly from any of the red light sensitive photodiode 104-3a, the green light sensitive photodiode 104-3b, or the blue light sensitive photodiode 104-3c. Instead, an additional photodiode 104-3d is used. The additional photodiode 104-3d is sensitive to white light, and its voltage is sampled to form the local feedback signal 110-3.

[0135] Also shown in Figure 7 is circuitry 150-3 that may be used during a readout phase. In particular, when a particular photoreceptor unit 100 at location (I,j) of the sensor array 200 needs to be read, the select SEL signal corresponding to that photoreceptor unit 100 is asserted and the Rth, Gth, Bth, and Vth signals from the corresponding line buffers, shown as transistors in Figure 7 and connected to a power supply voltage VDDR, are copied to the corresponding bit lines blr, blg, blb, and blv. These bit lines are grouped with the bit lines of other photoreceptor units of the same array 200 and are typically arranged in columns.

[0136] The operation of the third exemplary implementation of the photoreceptor unit 100-3 is the same as that of the first and second exemplary implementations described above, except for the presence of the white light sensitive diode 104-3d. During the image acquisition phase, the voltages of the red 104-3a, green 104-3b, and blue 104-3c photodiodes are not directly sampled, but only the white light sensitive diode 104-3d is sampled to provide a local feedback signal. The voltage 110-3 of the white light sensitive diode 104-3d is provided to the comparison circuit module 102-3. The global threshold 108-3 is also provided to the comparison circuit module 102-3. The comparison circuit module 102-3 uses a comparator to compare the white light sensitive diode 104-3d signal 110-3 to the global threshold 108-3. When the white light photosensitive diode 104-3d signal 110-3 reaches or exceeds the global threshold 108-3, the comparison circuit module 102-3 transmits the second control signal described above along the line 116-3, as shown in FIG. 3. This second control signal controls the sample and hold circuit 106-3 to store the current values ​​of the voltages of the red 104-3a, green 104-3b and blue 104-3c photodiodes and the voltage value of the voltage signal V(t) 112-3. Optionally, the voltage value of the white light photodiode 104-3d is also stored. This is done, for example, by opening the switches of the sample and hold circuit 106-3 shown in FIG. 7 to store the voltage value of each corresponding signal. The operation of the comparison circuit module 102-3 when the global threshold 108-3 is exceeded is therefore responsible for starting the capture of sensor data. In relation to the general description above, the condition / criterion is that the white light photosensitive diode 104-3d signal 110-3 passes the global threshold 108-3. Similar to the first and second exemplary implementations above, the sample and hold circuit 106-3 is also controlled by a global control signal generated by the global control module 202 or by a local signal generated at the end of the image acquisition phase (at the end of the integration phase) in response to a second pulse generated by the global control module 202.As previously described, a second pulse is received locally via the global clock line and overrides the output of comparator 116-3, preserving the voltages of the three photodiodes and the timing signal V(t) at the end of the image acquisition phase, even if the white signal 110-3 does not exceed threshold 108-2 during the image acquisition phase.

[0137] Upon capturing the voltage data and voltage signal V(t) 112-3 of each photodiode 104-3, the sample and hold circuit 106-3 may be configured to send a termination signal to the comparison circuit module 102-3 and associated circuitry of the sensor unit 100 using the same lines used to send the local feedback signal 110-3. This termination signal triggers the comparison circuit module 102-3 and associated circuitry to send a reset / stop signal to the photodiode 104-3, forcing it into a low power / non-operating state until a new cycle begins.

[0138] During the reset phase (RES=H) of the photoreceptor 100-3, the four photodiodes 104-3, including the white light sensitive diode 104-3d, are precharged to a voltage Vr. After the reset is released (RES=L), the four photodiodes 104-3 start to integrate light, and the voltage of each photodiode starts to drop approximately linearly with the intensity of the light falling on it, as explained above. The spectral response of the white light sensitive diode 104-3d can be approximated as the integral of the spectral responses of the red 104-3a, green 104-3b, and blue 104-3c photodiodes. Therefore, it can be assumed that the voltage of the white light sensitive diode 104-3d reaches the global threshold 108-3TH when the sum of the signals of the red 104-3a, green 104-3b, and blue 104-3c photosensitive diodes also reaches the global threshold 108-3TH. Thus, the third exemplary implementation is similar to the first exemplary implementation described above with reference to Figure 3, but this third implementation seeks to replicate the operation of the first embodiment by not directly sampling the red 104-3a, green 104-3b, and blue 104-3c photodiodes, but instead using white light sensitive diodes as an approximate "physical adder" of the red, green, and blue components of the incident light. Finally, similar to the first and second exemplary implementations, if V(T0), i.e., V(t), is known when the white signal 110-3 reaches the threshold 108-3, then each set of RGB values ​​corresponding to photoreceptor unit 100-3 can be used to calculate the absolute RGB values ​​of each photoreceptor unit 100 according to Equation 4.

[0139] FIG. 8 shows a timing diagram illustrating the third exemplary implementation and the process described above with reference to FIG. 7. In particular, after a reset phase, the red 104-3a, green 104-3b, blue 104-3c, and white 104-3d photodiodes are set to an initial voltage V(r). When the image acquisition phase begins, the global clock signal, represented by a voltage signal V(t), is initiated and the voltage of each photodiode 104-3 begins to discharge approximately linearly according to Equation 1 above, as shown by the R, G, B, and W lines in FIG. 8. At time T0, the white 104-3d photodiode signal 110-3 reaches the global threshold 108-3TH. At this time (T0), the sample and hold circuit 106-3 is instructed by the comparison circuit module 102-3 to capture and store the R, G, and B voltage values ​​from the red 104-3a, green 104-3b, and blue 104-3c photodiodes, respectively. These values ​​are shown by the Rth, Gth, and Bth lines in Figure 8. At the same time, the sample and hold circuit 106-1 also stores the value of V(t) at V(t)att=T0, shown by the Vth line in Figure 8. The Rth, Gth, Bth, and Vth values ​​are then output from the portion of the sensor unit output 114 that is read out during the read phase.

[0140] Using the white light sensitive diode 104-3d instead of the red 104-3a, green 104-3b and blue 104-3c photodiodes is advantageous because the red 104-3a, green 104-3b and blue 104-3c photodiodes are not manipulated in a local feedback loop before being stored by the sample and hold circuit 106-3 to form part of the sensor unit output signal 114. In fact, the red 104-3a, green 104-3b and blue 104-3c photodiodes are not disturbed by the sampling activity of the WTA module of FIG. 5 or by the sampling activity of the summation node of FIG. 3. As a result, the accuracy of the stored values ​​of the signals generated by the red 104-3a, green 104-3b and blue 104-3c photodiodes is improved, resulting in a more accurate representation of the RGB values ​​of the scene.

[0141] FIG. 9 is a schematic diagram illustrating a fourth exemplary implementation of the photoreceptor unit 100-4. This fourth implementation substantially combines features of the first, second, and third exemplary implementations described above. In the fourth exemplary implementation, the local control module 102 includes a first comparison circuit module 102-4a and a second comparison circuit module 102-4b. Although depicted separately, the first comparison circuit module 102-4a and the second comparison circuit module 102-4b may be implemented in the same circuit block portion of the local control module 102. The photosensitive element 104 is a group of photodiodes 104-4 including a red light sensitive diode 104-4a, a green light sensitive diode 104-4b, a blue light sensitive diode 104-4c, and a high sensitivity white light sensitive diode 104-4d. The white light sensitive diode 104-4d is a broad spectrum light sensor. The memory module 106 includes a sample and hold circuit 106-4.

[0142] The first comparison circuit module 102-4a of the local control module 102 functions in the same manner as described with reference to the first and second exemplary implementations above. In particular, the first local feedback signal 110-4a from the sample and hold circuit 106-4 includes either the sum of the RGB voltage values ​​from the photodiode 104-4 (in the first exemplary implementation) or a winner-take-all WTA signal of the RGB values ​​(in the second exemplary implementation). This first local feedback signal 110-4a is compared to a first global threshold 108-4a in a comparator in the first comparison circuit module 102-4a. When the first global threshold 108-4a is met, the first comparison circuit module 102-4a of the local control module 102 is configured to instruct the sample and hold circuit of 106-4 to store the R, G, B signals and the first value of the global timing signal 112-4a. In FIG. 9, these values ​​are indicated by the stored RGB values ​​Rth, Gth, Bth, and the stored first timing signal value Vth1.

[0143] The second comparison circuit module 102-4b functions in a similar manner as described with reference to the third embodiment. In particular, the second local feedback signal 110-4b is the value of the white light sensitive diode 104-4d. This value is compared to the second global threshold 108-4b in a comparator included in the second comparison circuit module 102-4b portion of the local control module 102. The white light sensitive diode may be a highly sensitive "white" photodiode, such as a pinned photodiode, or a single photon detector, or other CMOS-compatible highly sensitive photodetector. When the second global threshold 108-4b is met, the second comparison circuit module 102-4b of the local control module 102 is configured to instruct the sample and hold circuit 106-4 to store a W value and a second value of the timing signal 112-4a. In FIG. 9, these values ​​are indicated as the stored W value Wth and the stored second timing signal value Vth2.

[0144] Thus, it will be appreciated that multiple global thresholds can be used to compare the data for each photoreceptor unit 100-4.

[0145] The operation of the fourth exemplary implementation is the same as the first, second and third exemplary implementations described above. At the end of the image acquisition phase (integration phase), the sample and hold circuit 106-4 stores or will store the values ​​Rth, Bth, Gth of the red, green and blue photodiode voltages together with the first value Vth1 of the timing signal V(t) 112-4a indicating the time when the first local feedback signal 110-4a (sum or subject to WTA) corresponding to the combination of the R, G, B signals crosses the first global threshold 108-4a. Alternatively, the first value Vth1 of the timing signal 112-4a indicates the time of the end of the image acquisition phase (end of the integration phase) if the first global threshold 108-4a is not crossed during the maximum exposure period allocated. The sample and hold circuit 106-4 stores or has stored the value Wth of the voltage across the white light sensitive photodiode 104-4d together with the second value Vth2 of the timing signal V(t) 112-4a at the time the second local feedback signal 110-4b corresponding to the voltage across the white light sensitive photodiode crosses the second global threshold 108-4b, or at the end of the integration phase if the second global threshold 108-4b is not crossed before the end of image acquisition.

[0146] The sample and hold circuit ultimately stores six values ​​(the R, G, B signals Rth, Gth, Bth, and W signal Wth of the four photodiodes 104-4 and 104-4d, the first value Vth1 of the timing signal 112-4a, and the second value Vth2 of the timing signal 112-4a) that are transferred from the photoreceptor unit 100-4 as photoreceptor unit outputs during the readout phase and are made available, for example, to an external computer or memory.

[0147] In general, the readout circuitry is configured to transfer the photoreceptor output to an external unit / device / application, reading out all photoreceptor units 100-4 of the sensor array 200 sequentially or with some degree of parallelism, and transmitting the data to the external application using appropriate I / O channels connected to the photoreceptor units 100-4, and in particular the sample and hold circuits 106-4.

[0148] A first example of a read block 300 is shown in FIG. 9. The read block 300 includes a logic circuit with six inputs corresponding to the six values ​​Rth, Bth, Gth, Wth, Vth1, and Vth2 acquired during an image acquisition phase. The read block 300 may be implemented by one or more circuits designed and implemented to filter the six values ​​Rth, Bth, Gth, Wth, Vth1, and Vth2 generated by each photoreceptor unit 100-4 to reduce the bandwidth required to transfer the data to an external computer, memory, or application. Depending on the result of the comparison operation, the logic implemented in the read block 300 may determine whether to transmit only the R, G, and B photodiode values ​​Rth, Gth, and Bth along with the corresponding first value Vth1 of the timing signal 112-4aV(t) or to transmit only the W photodiode value Wth along with the corresponding second value Vth2 of the timing signal 112-4aV(t).

[0149] The read block 300 is activated during the read phase after the end of the image acquisition / integration phase when all six values ​​stored by the sample and hold circuit 106-4 are available. In this read phase, the following processes occur in the read block 300: When a second value Vth2 of the timing signal 112-4a, which defines the time when the second local feedback signal 110-4b corresponding to the voltage W of the white light photosensitive diode 104-4d reaches the second global threshold 108-4b or the time when the white light photosensitive diode 104-4d reaches the end of the integration phase without crossing the second global threshold 108-4b, is lower than a third global threshold 108-4c, a plurality of switches 302 in the readout block 300 are configured to connect respective output lines OutR, OutG, OutB, and OutV to input lines 304 corresponding to the R, G, B photodiode voltage values ​​Rth, Bth, Gth stored by the sample and hold circuit 106-4 and the value Vth1 of the first timing signal 112-4a. Otherwise, if the second value Vth2 of the timing signal 112-4a is greater than the third global threshold 108-4c, the switches are configured to connect the output lines OutR, OutG, OutB to the input line 304 corresponding to the value Wth of the white light photosensitive diode 104-4d stored by the sample and hold circuit 106-4, and the output line OutV is connected to the second value Vth2 of the timing signal 112-4a. The input line 304 corresponding to the value Wth is connected to three points in the read block 300, and a number of switches can respectively switch between the value Wth and any of the values ​​Rth, Gth, and Bth.

[0150] Since the white light photosensitive diode 104-4d is more sensitive than the red, green and blue light photosensitive diodes 104-4a, 104-4b and 104-4c, the second local feedback signal 110-4b is more likely to reach and meet the second global threshold 108-4b before the first local feedback signal 110-4a corresponding to the WTA or sum value of the RGB photodiodes 104-4 reaches the first global threshold 108-4a. Therefore, the second value Vth2 of the timing signal 112-4a is selected to be compared with the third global threshold 108-4c using the comparator, rather than the first value Vth1 of the timing signal 112-4a.

[0151] If the intensity of light incident on the photoreceptor unit 100-4 is very low, for example when a dark area of ​​the scene is being observed, the red, green and blue light photosensitive diodes 104-4a, 104-4b, 104-4c will discharge slower and will likely not reach the first global threshold 108-4a within an image acquisition phase and therefore within the maximum allocated exposure time. However, the white light photosensitive diode 104-4d will always discharge faster, since this diode is more sensitive and therefore more likely to reach the second global threshold 108-4b within an image acquisition phase.

[0152] The third global threshold 108-4c used in the readout block 300 helps to determine the optimal set of output signals OutR, OutB, and OutG in terms of quality or signal-to-noise ratio. If the second value Vth2 of the timing signal 112-4ab stored by the sample-and-hold circuit 106-3 passes the third global threshold 108-4c, this means that the intensity of the incident light is very low, and the readout block 300 transmits OutR, OutB, and OutG using the Wth value collected by the more sensitive white photosensitive diode. Therefore, under low light conditions, the values ​​of OutR, OutG, and OutB are all equal to the value Wth. This means that color information is lost and the final image becomes a gray level image. Conversely, if the intensity of the incident light is not low, the readout block 300 transmits OutR, OutB, and OutG collected by the colorful set of R, G, and B photodiodes 104-4a, 104-4b, and 104-4c. In either case, the OutV value generated by the read block 300 includes the timing signal V(t) 112-4a sampled when the corresponding signal set was sampled. In other words, when there is bright incident light, OutR, OutG, and OutB include colorful representations of the incident light corresponding to the stored values ​​Rth, Gth, and Bth, and OutV provides a first value Vth1 of the timing signal 112-4a sampled when the output of the sum or WTA circuit 110-4a reaches the first global threshold 108-4a. When there is dark incident light, OutR, OutG, and OutB provide a gray level representation of the incident light corresponding to the stored value Wth, and OutV provides a second value Vth2 of the timing signal 112-4a sampled when the output of the white photosensitive diode reaches the second global threshold 108-4b.

[0153] As shown in FIG. 10, in a further variation of the fifth exemplary implementation, the alternative readout block 400 is controlled by a more complex function of the first stored value Vth1 of the timing signal 112-4a, the second stored value Vth2 of the first timing value 112-4a, and the third global threshold 108-4c, resulting in a more general management of the output signal. Using these value functions, a sensor suitable for both very low light conditions and very bright light conditions can be used to optimize the signal-to-noise ratio. Thus, although the comparison of the second value Vth2 of the timing signal 112-4a to the third global threshold 108-4c is performed according to the fifth exemplary implementation, this is not required and it should be understood that another function may be used to determine which of the six stored values ​​Rth, Gth, Bth, Wth, VTh1, and Vth2 to filter and / or send to the readout circuitry of the sensor array 200. In some examples, the functions may be optimized for a particular application domain and may be generated by a neural network trained according to the particular application domain. The function may use spatial information, such as the position of the photoreceptor units in the array, to modify the operation of the readout block. For example, in industrial or video surveillance applications where the sensor array is installed in a fixed location, different criteria may be applied by the function to switch between less sensitive color vision and more sensitive gray level vision depending on the particular characteristics of the scene. The function may be implemented by an external composite device, such as a computer. The computer may replace the readout block, and the stored signals input to the readout block may instead be read out from each photoreceptor by the computer. In this way, the computer has access to the values ​​of Rth, Bth, Gth, Wth, Vth1, and Vth2.

[0154] It is understood that photodiodes with high sensitivity to light capture, such as those capable of detecting single photons, are devices that occupy more silicon area than standard photodiodes, and thus the fifth implementation described herein minimizes the complexity and area requirements of a sensor that can achieve a good signal-to-noise ratio in normal lighting conditions as well as in very dark scenes.

[0155] The operation of this fifth implementation is very similar to that found in the human retina, where red, blue, and green cones are active when light intensity is at a sufficient level, but are disabled when the intensity of the incoming light is low. On the other hand, the rods, which are very sensitive photoreceptors, are only active when there is a dark scene in which the cones are disabled. The fifth implementation described above uses readout logic circuitry in readout blocks 300 and 400 to effectively mimic this operation.

[0156] Finally, with respect to the general architecture 100, the shape / function of the global feedback signals 108-4a, 108-4b, and 108-4c and the timing signal 112-4a can be adjusted by the global control block 202 based on the characteristics of the observed scene. As previously mentioned, the first and second timing signals 108-4a and 108-4b are transmitted on the line carrying the global clock signal 112, along with other signals (not shown in FIGS. 9 and 10) used to synchronize and coordinate operations during an image acquisition cycle. > The sample and hold circuit 106-4 may be implemented as one or two circuits and represents the memory module 106 or part thereof. The comparators and WTA / sum circuits that form part of the first and second comparison circuit modules 102-4a and 102-4b are part of the local control module 102. Also part of the local control module 102 is a circuit for synchronizing operations and information received via the global clock line, while the local feedback line 102 includes the signal generated by the photosensitive element 104a.

[0157] It should be appreciated that the above-described read blocks 300 and 400 may be used in any or all of the above-described exemplary implementations.

[0158] FIG. 11 is a schematic diagram showing an example of how additional sensors / circuits can be used to complement the examples above. In particular, the circuit 500 can detect temporal changes in the incident light, which can be used to activate blocks of the architecture of FIG. 1 so that the architecture of FIG. 1 is only used when the light intensity changes. This technique can be used to effectively add motion detection capabilities to the device, or to improve the energy efficiency of the overall architecture shown in FIG. 1. Instead of continuously processing the numerous signals provided by the blocks of the architecture of FIG. 1, it is even more convenient to process them only when some light change is detected in the scene. The circuits described in FIG. 11 can be used to build blocks or units that are added to the blocks and photoreceptor units described in the general architecture shown in FIG. 11. These can also be included in the global control module 202 to provide additional functionality that can be used to modify the global operation of the sensor device. FIG. 11 shows a circuit 500 that includes a first comparison circuit module 102-5a, an additional sensor 502, a first global feedback signal 108-5a, a first local feedback signal 110-5a, and a bidirectional switch included in the sample and hold circuit 106-5.

[0159] The circuit 500 also includes a second comparison circuit module 102-5b, a second global feedback value 108-5b, a second local feedback signal 110-5b, and an output circuit 504. The operation of these components is described below.

[0160] The general cycles and phases described above also apply here. The additional sensor 502 is for example an infrared (IR) sensor or a visible white light sensitive diode. This additional sensor is used in the same way as described above with reference to the white light sensitive diodes 104-3d and 104-4d in Fig. 7 and Figs. 9 and 10. After a reset phase (RES), the additional sensor 502 starts integrating the light or other physical signal it is designed to detect, and its output voltage 110-5a starts to decrease linearly over time. This signal forms the first local feedback signal 110-5a and is fed to the first comparison circuit module 102-5a. The first global threshold value 108-5a is also fed to the first comparison circuit module 102-5a. When the first local feedback signal 110-5a from the additional sensor 502 reaches the first global threshold 108-5aTH, the first comparison circuit module 102-5a uses a comparator to change its output voltage from logic LOW to HIGH, instructing the sample and hold module 106-5 to store the timing signal V(t) 112-5 in one of the two memory modules S&H1, S&H2 at time T0, and updating the output values ​​V1th and V2th according to the state of the bidirectional switch controlled by the global signal SD and generating the signals SW1 and SW2. During the next image acquisition cycle, the process is repeated for the next frame recorded in the array 200, and the value of the timing signal V(t) 112-5 corresponding to time T1 is stored in the other memory module 106-5 S&H1 or S&H2 depending on the value of the global control signal SD. Thus, for example, if the value of timing signal 112-5 is stored in S&H1 memory module in the first cycle, it is stored in S&H2 memory module in the next cycle, and again in line memory module S&H1 in the next cycle. A global signal SD is used to select the outputs SW1 and SW2 of the bidirectional switches included in memory module 106-5 in each image acquisition cycle.Thus, in each image acquisition cycle, the values ​​V1th and V2th stored at the output of the sample and hold module 106-5 alternately represent the current value of the timing signal 112-5 and the value of the previous image acquisition cycle when the output 110-5a of the additional sensor 502 crossed the global threshold TH 108-5a. The absolute value of the difference between the two analog voltages V1th and V2th is then used as a second local feedback signal 110-5b and input to the second comparison circuit module 102-5b. The second comparison circuit module 102-5b compares the second local feedback signal 110-5b with the second global threshold 108-5b and generates an output signal DIFF on the output line 504. The DIFF signal on the output line 504 is LOW if the second global threshold 108-5b has not been crossed and is HIGH if the second global threshold 108-5b has been crossed.

[0161] In this way, the two switches Sx and Sy can be controlled to activate the two current sources IY and IX to inject a certain amount of current into the respective output lines, as described below. For example, in FIG. 11, the two current sources IX and IY of a common pixel at coordinates (i,j) in the sensor array 200 are shorted to the row bit line (IXi) and column bit line (lYj), respectively, only if the comparator of the second comparison circuit module 102-5b detects a large change in light intensity (DIFF=HIGH). In this case, the two switches Sx and Sy connect the two current sources IX and IY to the respective bit lines IXi and lYj, otherwise, if DIFF=LOW, the two current sources are disconnected from the bit lines and do not contribute to the total current flowing through the associated bit lines. Thus, the circuit 500 is equivalent to a photosensitive element that generates a LOW voltage on the output line 504 if the difference in the intensity of the incident signal (e.g., light) between two successive acquisition cycles does not reach a predetermined threshold (in this case, less than the value TH1 of the second global threshold 108-5b), and generates a HIGH voltage on the output line 504 otherwise. For imaging applications, a sudden change in the intensity of the light detected by the additional sensor 502 is often caused by a moving object moving across the scene. In this way, the circuit 500 can be considered a "motion / event detector" pixel or "motion / event pixel," and such a pixel can be added / coupled to the sensor array 200 or global control module 202 to build a sensor that can detect or react to the presence of a moving object in a scene.

[0162] For example, in FIG. 11, if the sensor array 200 were comprised solely of motion / event pixels 500 instead of sensor units 100, or if the sensor array 200 were comprised of an array of motion / event pixels 500 and an array of sensor unit 100 sensors, the current flowing through the row bit lines and column bit lines connected to the motion / event pixels 500 would be proportional to the amount of motion generated by a moving object in the scene during an image acquisition phase. More generally, a large change in intensity between frames recorded by the circuit 500 can be used to infer motion detected by the circuit 500. Multiple such circuits 500 can be placed across an array 200 to track motion across the array 200. Furthermore, the image acquisition phase of the photoreceptor units 100 can be made dependent on the detection of such motion within the field of view of the array 200. In this manner, the circuit 500 can be used in combination with any of the examples of FIGS. 1 through 10 above to provide a motion-activated sensor array.

[0163] Thus, circuit 500 is advantageous because it can be used to detect motion, for example, to activate array 200 when any motion is detected, to store in memory module 106 the voltage generated by the photodiodes when any motion is detected, and to stop / reset the operation of the photodiodes when there is no motion in the scene. All of these exemplary mechanisms result in significant energy savings during operation and are useful in applications where detection of motion is important, such as automated video surveillance.

[0164] While FIG. 1 shows a uniform sensor array 200 of sensor units 100a to 100n, it should be understood that this is not required and the sensor array 200 can have an irregular layout depending on its intended application / function. For example, the sensor array can include an inner array of small form factor sensor units designed to capture a central area of ​​a scene with high resolution. Such a central area can be surrounded by a frame containing larger sensor units designed to detect fast moving objects at the periphery of the scene with high sensitivity. This example mechanism mimics the so-called peripheral vision of the human eye, which is focused on detecting motion. FIG. 12 shows an example of such a sensor architecture 1100 based on two different types of photoreceptors, a first photoreceptor type E and a second photoreceptor type S. In FIG. 12, for the sake of clarity, both these photoreceptors types E and S are the same size and the frame is formed only from one peripheral layer of the first type E photoreceptors. In general, however, the first and second types of photoreceptors E and S have different shapes and / or sizes, and the frame formed by the first type of photoreceptors E includes any number of horizontal / vertical lines and layers of the first type of photoreceptors E. The second type of photoreceptors S form the inner part of the array and can include, for example, one or more of the first, second, third, fourth or fifth exemplary implementations described above and shown in Figures 3 to 10. The role of the second type is to obtain images with high resolution and high dynamic range. The first type of photoreceptors E arranged in a frame around the central part of the array 1100 can include, for example, the type described in Figure 11 and can be configured to detect motion events generated, for example, by moving objects entering or leaving the scene being observed by the array 1100. The motion of the object can be detected by measuring the change in the intensity of light falling on the photoreceptors E over time, as shown in the further exemplary implementation of Figure 11.

[0165] For clarity, Figure 13 depicts the same sensor architecture 1100 as Figure 12, showing how the current sources IX and IY of each circuit 500 described in Figure 11 can be summed along the X and Y directions of the array to detect changes in light intensity. In this case, each circuit 500 is represented as a unit E, shown with only two switches controlled by a signal DIFF and two current sources IX and IY.

[0166] The output signal of the inner part of the array made up of the second photoreceptors S is sent via line 1102. Motion information is detected and output via four output lines (ET, EL, ER, EB) along each of the four sides of the array 1100 shown in FIG. 10. In normal operation, the inner part of the array containing the second type of photoreceptors S is normally off, while the first type of photoreceptors E around the periphery of the array 1100 are always on and continuously detect motion along the periphery of the array 1100. As soon as a large change in light intensity is detected by the first type of photoreceptors E, the inner array of the second type of photoreceptors S is activated and an image with higher resolution and high dynamic range is obtained.

[0167] While the operation of the second type of photoreceptors S requires significant power and data bandwidth, the motion detection task performed by the first type of photoreceptors is simpler, more energy efficient, and less expensive in terms of data bandwidth requirements. The above splitting operation allows the motion detection task to be performed by turning on the more powerful second type of photoreceptors S, mimicking the behavior of the human retina while minimizing power consumption and data bandwidth to the external computer.

[0168] The operational information represented by the currents flowing through the EL, ET, EB, and ER lines in Figs. 12 and 13 can be used by the global control module 202 as feedback signals for logic implemented in the global control module 202 to activate certain actions. One example is the power saving mechanism described above, where such information is used to power on or off the internal array of the second type of photoreceptors S. In another example, these four signals can be aggregated into one "motion detected" signal and forwarded to an external application. In this case, for example, when the "motion detected" signal is HIGH (i.e., when a moving object is detected), an external controller can be woken up to process and store the images generated by the sensor. On the other hand, when the "motion detected" signal is LOW (i.e., when there is no moving object in the scene), the external controller can enter a standby / sleep / power saving mode, optimizing energy usage and space for storing associated videos / images at the system level. In yet another example, the values ​​of the currents flowing through the EL, ET, EB, and ER lines can be sampled and sent directly to an external application using dedicated output lines.

[0169] While the above description has focused on imaging sensors and the use of photosensitive diodes as the sensor elements 104, it should be understood that any sensor may be used in the same global / local feedback configuration. For example, instead of photodiodes, thermal sensors, pressure sensors, or other types of sensors that can be used in the array may be implemented.

[0170] The sensor array 200 described above may be implemented in a variety of devices and apparatuses. For example, if the sensor array 200 includes optical or infrared sensors 104, the sensor array 200 may be incorporated into a camera or surveillance device. It may also be incorporated into a telescope, satellite, or the like to observe a scene. If the sensors are designed to detect different wavelengths in the electromagnetic spectrum, the sensor array 200 may be used in a satellite or other spacecraft or observatory. If the sensors 104 include pressure or touch sensors, the sensor array 200 may be used in a smart floor panel, or the like.

[0171] An important application domain of the sensor array 200 is represented by artificial or machine vision, where the images produced by the sensor array 200 are post-processed by some computing device in order to extract information considered relevant for the particular application. Some examples (e.g. security or video surveillance applications where the presence or absence of motion in the scene is one type of important information) have already been given in the paragraphs above. More generally, an important feature enabled by the invention is the ability of the sensors and / or receptors forming the sensor array 200 to adapt (i.e. locally) to the strength of the incident signal in order to maximize the signal-to-noise ratio, even when the incident signal is very strong or very weak, and even when such light conditions are simultaneously present in the acquired image. Thus, the sensor array can show different responses at different locations within it, at different receptors, based on differences in the incident signal. Moreover, the flexible global feedback mechanism implemented by the global control module 202, which can be fully or partially implemented in software by an external computer, can also optimize the response of the sensors when used in a particular application. For example, in an automotive application, the global control module may analyze the image produced by the sensor array and, as mentioned above, adjust the timing signals V(t) or global thresholds used by the various modules to allow important areas of the image (e.g., areas where road signs are located and the application needs to read them) to be captured with greater accuracy, while other areas (e.g., areas containing sky and road surface) can be captured with less accuracy. This operation is similar to what happens in human vision, where the brain determines which parts of the observed scene need to be captured with high accuracy and sends signals to the eye to "configure" it accordingly.

[0172] To tune the sensors or photoreceptors of the sensor array as described above, one or more neural networks may be used to generate or modify either the timing signal V(t) or the global threshold. The one or more neural networks may be trained for a particular application. In the above example, the application may relate to driving a vehicle.

[0173] Control of the sensor array 200 may be performed by a computer or computing device. The computer or computing device may be in communication with or form part of the target control module 202. As described above, the global control module 202 may adjust various parameters of the embodiment, including global thresholds, a global clock signal, and other related signals. These signals may be adjusted dynamically in real time in response to changes in characteristics of the scene or other sensory environment being observed. The changing characteristics may be observed by observing differences between frames or individual exposures of the sensor 104. The signals may be adjusted by the computer, applying a neural network or any suitable machine learning algorithm.

[0174] In the above description, the execution of the methods and various functions of the system may be implemented using computing and / or electronic devices. Such devices may include one or more processors that are microprocessors, controllers, or other suitable types of processors that process computer-executable instructions that control the operation of the device to collect and record routing information. In some examples, such as when a system-on-chip architecture is used, the processor may include one or more fixed function blocks (also called accelerators) that implement parts of the methods in hardware (rather than software or firmware). Platform software, including an operating system or other suitable platform software, may be provided to the computing-based device to enable application software to be executed on the device.

[0175] Various functions described herein can be implemented in hardware, software, or a combination thereof. If implemented in software, such as a computer program, the functions can be stored or transferred as one or more instructions or codes on a computer-readable medium. Computer-readable media can include, for example, computer-readable storage media. Computer-readable storage media includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can be any available storage medium accessible by a computer. By way of example, and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, flash memory or other memory devices, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or other medium that can be used to carry or store the required program code in the form of instructions or data structures and that can be accessed by a computer. Disks and disks as used herein include compact discs (CDs), laser discs, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray (RTM) disks (BDs). Additionally, propagated signals are not included within the scope of computer-readable storage media. Computer-readable media also includes communication media, including any medium that facilitates transfer of a computer program from one place to another. For example, a connection can be a communication medium. For example, when software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, or microwave, it is included within the definition of communication media. Combinations of the above should also be included within the scope of computer-readable media.

[0176] Alternatively, or in addition, the functions described herein may be performed, at least in part, by one or more hardware logic components, such as, but not limited to, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0177] Although illustrated as a single system, it should be understood that the computing device may be a distributed system, such that, for example, multiple devices may communicate over a network connection and jointly perform the tasks described as being performed by the computing devices.

[0178] Those skilled in the art will appreciate that storage devices utilized to store program instructions can be distributed across a network. For example, a remote computer may store an example of a process written as software. A local or terminal computer may access a remote computer and download some or all of the software to execute the program. Alternatively, a local computer may download parts of the software as needed, execute some software instructions at a local terminal, and execute some at a remote computer (or computer network). Those skilled in the art will also appreciate that, utilizing conventional techniques known to those skilled in the art, all or part of the software instructions can be executed in dedicated circuitry such as DSPs, programmable logic arrays, etc.

[0179] Additionally, acts described herein may include computer-executable instructions implemented by one or more processors and / or stored on a computer-readable medium. Computer-executable instructions may include routines, subroutines, programs, threads of execution, etc. Additionally, results of the operations of the methods may be stored on a computer-readable medium or displayed on a display device.

[0180] The terms "computer" or "external memory" as used herein are intended to encompass computer-readable data storage comprised of computer-executable instructions that, when executed by a processor, perform a particular function. Computer-executable instructions may include routines, functions, etc. It should also be understood that a computer may be localized on a single device or distributed across multiple devices.

[0181] Moreover, the word "exemplary" or "example" as used herein is intended to mean "serving as an illustration or example of something."

[0182] Furthermore, to the extent that the term "including" is used in any of the detailed descriptions, such term is intended to be as inclusive as the term "including."

[0183] Although the methods are shown and described as a series of acts performed in a particular order, it should be understood and appreciated that the methods are not limited by the order. For example, some acts may occur in a different order than described herein. Further, some acts may occur simultaneously with other acts. Furthermore, in some cases, not all acts may be required to implement the methodologies described herein.

[0184] Although the order of operations of the methods described herein is exemplary, operations may be performed in any suitable order, or simultaneously where appropriate. Moreover, operations may be added or substituted, or individual operations may be deleted, in any method without departing from the scope of the subject matter described herein. Aspects of any of the examples described above may be combined with aspects of any other example to form further examples without losing the desired effect.

[0185] The above description of the embodiments is for illustrative purposes only, and it is understood that various modifications are possible for those skilled in the art. The above description may include one or more example embodiments. Of course, it is not possible to describe all possible modifications and variations of the above-mentioned device or method for the purpose of describing the above aspects, but those skilled in the art will recognize that many more modifications and combinations of various aspects are possible.

Claims

1. An imaging sensor unit, a sensor module including a plurality of optical sensors; a memory module coupled to the sensor module, the memory module configured to receive a sensor output from the sensor module; and a local control module coupled to the sensor module, the local control module comprising: receiving a local feedback signal from the sensor module; receiving the global feedback signal; and the local control module configured to control the memory module to store sensor output using the local feedback signal and the global feedback signal.

2. The global feedback signal includes a threshold value, and the local control module: comparing the local feedback signal to a global feedback signal threshold; The imaging sensor unit of claim 1 , configured to control the memory module to store a sensor output when the local feedback signal reaches the threshold.

3. The optical sensor a first photosensor sensitive to a first portion of the visible spectrum, the first photosensor configured to output a first sensor output component; a second photosensor sensitive to a second portion of the visible spectrum, the second photosensor configured to output a second photosensor output component; and a third photosensor sensitive to a third portion of the visible spectrum, the third photosensor configured to output a third photosensor output component; The imaging sensor unit of claim 2 , whereby the sensor output comprises first, second, and third photosensor output components.

4. The imaging sensor unit includes: a summing node configured to sum the first, second, and third photosensor output components to generate a sum value, wherein the local feedback signal comprises the sum value, whereby: the local control module is configured to compare the sum with a global feedback signal threshold; If the sum meets a threshold, the local control module: The imaging sensor unit of claim 3 , configured to cause a memory module to store the first, second, and third photosensor output components.

5. The imaging sensor unit includes: The winner-take-all node further includes: receiving the first, second, and third optical sensor output components; determining a winning photosensor output component by identifying one of the first, second and third photosensor output components that exhibits the greatest difference from an initial value of each of the first, second and third output components, wherein the local feedback signal includes the winning photosensor output component, thereby: the local control module is configured to compare the win optical sensor output component with a global feedback signal threshold; When the win optical sensor output component meets the threshold, the local control module: The imaging sensor unit of claim 3 , configured to cause the memory module to store the first, second, and third photosensor output components.

6. The imaging sensor unit of claim 3 , wherein the first photosensor is sensitive to red light, the second photosensor is sensitive to green light, and the third photosensor is sensitive to blue light.

7. the plurality of light sensors include broad-spectrum light sensors, the broad-spectrum light sensors configured to detect brightness of a viewed scene and generate a brightness output, the local feedback signal includes a brightness output, and when the brightness output satisfies a threshold of the global feedback signal, the local control module: The imaging sensor unit of claim 3 , configured to cause the memory module to store sensor output.

8. the plurality of light sensors includes a broad-spectrum light sensor for detecting light in low light conditions, the broad-spectrum light sensor having a higher sensitivity than the first, second, and third light sensors, the broad-spectrum light sensor configured to detect brightness of a viewed scene and generate a brightness output; the local control module is further configured to receive the second local feedback signal and the second global feedback signal, the second local feedback signal including a brightness output and the second global feedback signal including a second threshold; The local control module is configured to compare the second local feedback signal to the second threshold, whereby when the second local feedback signal meets the second threshold, the local control module: configured to cause the memory module to store a brightness output; 4. The imaging sensor unit of claim 3, wherein the imaging sensor further includes a readout module configured to output one or more of a brightness output, a first, second, and third photosensor element output from the imaging sensor based on a result of, or a function of, a comparison with a third global feedback signal received at the readout module.

9. The imaging sensor unit of claim 7 , wherein the broad spectrum light sensor is configured to detect white light.

10. The imaging sensor unit of claim 7 , wherein the broad spectrum light sensor is configured to detect infrared light.

11. 3. The imaging sensor unit of claim 2, wherein the imaging sensor is subject to a maximum exposure time, and the memory module is configured to store a sensor output if the local feedback signal does not meet a threshold within the maximum exposure time.

12. The local control module further comprises: The imaging sensor unit of claim 1 , configured to cause the memory module to store timing information indicating when the sensor output was stored.

13. The imaging sensor unit of claim 12 , wherein the memory module is configured to receive the timing signal, and the timing information is a record of the timing signal at the time the sensor output was stored.

14. 2. A sensor array including a plurality of imaging sensor units according to claim 1, wherein the sensor array comprises: a global control module configured to globally control the sensor array, the global control module configured to provide a global feedback signal to each of a plurality of imaging sensor units.

15. The sensor array of claim 14 , wherein the global feedback signal is adjustable to adjust characteristics of an image captured by the sensor array.

16. The sensor array of claim 14 , further comprising a readout circuit configured to read out stored sensor outputs from the plurality of imaging sensor units.

17. 17. The sensor array of claim 16, wherein the readout circuitry is further configured to read out the timing information for each of the plurality of imaging sensor units, the timing information indicating a time at which a stored sensor output was stored for each of the plurality of imaging sensor units.

18. 17. The sensor array of claim 16, wherein the sensor array is communicatively connected to a computing device, and the readout circuitry is configured to transmit information regarding at least a sensor output of each of the plurality of imaging sensor units to the computing device.

19. Further comprising one or more motion sensor units, the one or more motion sensor units comprising: An optical sensor; a motion sensor memory module configured to receive a timing signal; a motion sensor control module; The motion sensor control module receiving and comparing a sensor output of the optical sensor and the global motion threshold; configured to store timing information from the timing signal in the memory module when the sensor output of the optical sensor satisfies the global motion threshold, the timing information indicating a time when the global motion threshold was met; the motion sensor memory module includes a switch configured to change the storage location of the timing information; During a first active motion sensing phase of the optical sensor corresponding to a first frame of the observed scene, the switch is placed in a first position corresponding to a first storage location; During a subsequent second active motion sensor phase of the optical sensor corresponding to a second frame of the observed scene, the switch is placed in a second position corresponding to a second storage location; The local control module causing the memory module to store, at the first storage location, first timing information indicating a time when a global motion threshold for the first active motion sensor phase was met; configured to cause the memory module to store second timing information in the second storage location indicative of a time when a global motion threshold for the second active motion sensor phase was met; The local control module calculating a difference value corresponding to the difference between the first timing information and the second timing information; comparing the difference value to the second global motion threshold; 16. The sensor array of claim 15, configured to output a signal from the motion sensor indicating that motion has been detected between a first frame and a second frame of the observed scene when the difference value satisfies the second global motion threshold.

20. The sensor array of claim 19 , wherein the first global motion threshold and the second global motion threshold are adjustable.

21. 20. The sensor array of claim 19, wherein the first global motion threshold is a photosensor output threshold and the second global motion threshold is a time threshold.

22. 20. The sensor array of claim 19, wherein a signal indicating that motion is detected between the first frame and the second frame of the observed scene is configured to activate the plurality of imaging sensor units.

23. The sensor array of any one of claims 19 to 22, comprising a plurality of motion sensor units.

24. The sensor array of claim 23 , wherein the plurality of motion sensor units are arranged at least around the periphery of the sensor array.

25. 1. A method for controlling an imaging sensor unit, the imaging sensor unit comprising: a sensor module including a plurality of optical sensors; a local control module coupled to the sensor module; a memory module coupled to the sensor module; The method comprises: receiving a sensor output from the sensor module in the memory module; receiving a local feedback signal from the sensor module at the local control module; receiving the global feedback signal at the local control module; and using the local feedback signal and the global feedback signal to control the memory module to store sensor outputs.

26. operating the imaging sensor unit according to a cycle, the cycle comprising: beginning by activating the sensor module from an initial state such that the sensor module begins detecting and generating a sensor output; an active sensor phase, and Start by resetting the sensor module to an initial state.

26. The method of claim 25, wherein the sensor phase is defined by an inactive sensor phase.

27. entering an active sensor phase; and entering an inactive sensor phase after storing the sensor output.

28. The duration of the active sensor phase is limited by a maximum exposure period, whereby the method comprises: causing the memory module to store the sensor output during the maximum exposure period; 28. The method of claim 27, further comprising entering the inactive sensor phase such that the inactive sensor phase occurs after the maximum exposure time has elapsed.

29. 26. The method of claim 25, further comprising storing timing information indicating the time the sensor output is stored upon storing the sensor output.

30. The method further includes controlling the plurality of imaging sensor units, A method according to any one of claims 25 to 29, comprising the step of synchronously entering the active phase for each of the imaging sensor units.

31. A motion sensor, the motion sensor comprising: An optical sensor; a motion sensor memory module configured to receive a timing signal; a motion sensor control module; The motion sensor control module receiving and comparing a sensor output of the optical sensor and a global motion threshold; configured to cause the memory module to store timing information from the timing signal when the sensor output of the optical sensor satisfies the global motion threshold, the timing information indicating a time when the global motion threshold was met; the motion sensor memory module includes a switch configured to change the location where timing information is stored; During a first active motion sensing phase of the optical sensor corresponding to a first frame of the observed scene, the switch is placed in a first position corresponding to a first storage location; During a subsequent second active motion sensor phase of the optical sensor corresponding to a second frame of the observed scene, the switch is placed in a second position corresponding to a second storage location; The local control module causing the memory module to store first timing information in the first storage location indicating a time when a global motion threshold for the first active motion sensor phase was met; configured to cause the memory module to store second timing information in the second storage location indicative of a time when a global motion threshold for the second active motion sensor phase was met; The local control module calculating a difference value corresponding to the difference between the first timing information and the second timing information; comparing the difference value to the second global motion threshold; the motion sensor is configured to output a signal indicating that motion has been detected between the first and second frames of the observed scene when the difference value satisfies the second global motion threshold.

Images