Imaging system, illumination module and time-of-flight sensor

Abstract

The invention relates to an imaging system, an illumination module and a time-of-flight sensor. The system may include a lighting module. The lighting module may include a light source monitoring system. The light source monitoring system can sample light source signals and aggregate samples into data blocks. The light source monitoring system may selectively aggregate the data blocks to form groups of monitoring time windows. The light source monitoring system may detect a fault based on values characterizing light source performance during the monitoring time window.

Description

Technical Field

[0001] The present invention generally relates to light source monitoring, and more particularly to light source monitoring in an imaging system. Background Art

[0002] As an example, a time-of-flight (TOF) sensor for an imaging system may include an illumination module and a sensor module. The illumination module may include one or more light sources that emit light to illuminate an image scene. The emitted light is reflected from one or more objects and may be received by pixels in the sensor module to generate corresponding charges. Based on the generated charges, the sensor module may perform time-of-flight sensing calculations to determine depth and other scene information.

[0003] It may be desirable to monitor the operation of the illumination module to ensure compliance with safety guidelines and / or requirements. Summary of the Invention

[0004] According to a first aspect, there is provided an imaging system comprising: a sensor module having an image sensor pixel array; and an illumination module having a light source, a driver circuit coupled to the light source, and a light source monitoring system coupled to the light source and configured to receive a light source signal, wherein the light source monitoring system comprises: a data chunking circuit that generates a plurality of data chunk values based on samples from the light source signal; a grouping circuit that generates a plurality of grouping values, each grouping value aggregating at least some of the plurality of data chunk values; and a failure detection circuit that generates an output signal indicative of a fault based on a monitoring time window characterized by at least some of the plurality of group values.

[0005] According to a second aspect, there is provided an illumination module comprising: a light source; and a light source monitoring system coupled to the light source and configured to receive a light source signal, wherein the light source monitoring system comprises: a counter configured to generate samples of the light source signal; a plurality of averaging circuits coupled to the counter; a multiplexer having a plurality of inputs coupled to the plurality of averaging circuits and having an output; a first summing circuit coupled to the output of the multiplexer; a memory circuit coupled to the first summing circuit; a second summing circuit coupled to the memory circuit and the first summing circuit; and a comparison circuit coupled to the second summing circuit.

[0006] According to a third aspect, there is provided a time-of-flight sensor, comprising: a sensor module having an image sensor pixel array; and an illumination module having a light source, a driver circuit coupled to the light source, and a light source monitoring system coupled to the light source and configured to receive a drive signal provided to the light source from the driver circuit, wherein the light source monitoring system comprises: a first monitor configured to use the drive signal and a first observation time window having a first duration to detect one or more faults; and a second monitor configured to use the drive signal and a second observation time window having a second duration to detect the one or more faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of an exemplary system having an illumination module according to some embodiments.

[0008] Figure 2 is a schematic diagram of an exemplary illumination and sensor module according to some embodiments.

[0009] Figure 3 is a schematic diagram of an exemplary illumination module having a light source monitoring system according to some embodiments.

[0010] Figure 4 is a schematic diagram of an exemplary light source monitoring system according to some embodiments.

[0011] Figure 5 is a schematic diagram of an exemplary sampling and monitoring of a light source signal based on data block grouping according to some embodiments.

[0012] Figure 6 is a schematic diagram of an exemplary circuit for implementing a light source monitoring system according to some embodiments. DETAILED DESCRIPTION

[0013] An electronic system may include an illumination module having one or more light sources that provide illumination, such as infrared illumination, visible light illumination, and / or illumination of other wavelengths. If the illumination module malfunctions, for example due to one or more system failures, risks may be posed. Therefore, it may be necessary to monitor the operation of the illumination module. In some cases, the light source may operate at high frequencies using a control signal having one or more frequencies between 30 MHz and 200 MHz and operate in different modes, which further increases the difficulty of sampling and monitoring the light source signal given the large number of samples required and the various desired sampling time windows associated with different operating modes.

[0014] To monitor the operation of a lighting module and light sources therein, the lighting module may include a light source monitoring system. The light source monitoring system may sample the status of a light source signal over time. The light source monitoring system may aggregate the sampled data into data values (sometimes referred to herein as data block values) each associated with a data block, and may further aggregate a plurality of data block values into data values (sometimes referred to herein as data group values) each associated with a group or grouping of data blocks. In particular, the monitoring system may selectively generate data block values, each having a size set to average a desired number of samples and / or for a desired time period, and may selectively generate data group values to sum a desired number of appropriately sized data block values to form a desired monitoring time window for a given monitor. The monitoring time window may be a rolling time window, characterized by a rolling sum as the monitored data group value. Multiple such monitors may be provided to analyze the light source performance of multiple monitoring time windows in parallel.

[0015] By providing monitoring of the sampled data in different groupings of selected data blocks associated with different monitoring windows, the light source monitoring system may provide flexible and / or synchronized monitoring of various different operating modes. Using different data blocks and different data groupings may help compress the number of samples processed by the light source monitoring system, which may be particularly helpful in real-time monitoring of high-frequency operations, which are typically characterized by processing a relatively large number of data points in a small time period.

[0016] A configuration in which the lighting module forms part of an imaging system (such as a time-of-flight (TOF) imaging system) is sometimes described herein as an example. In some illustrative configurations, the imaging system may be part of an automotive system or vehicle, a digital camera, a computer, a cellular phone, or another type of electronic system. The lighting module in the imaging system may help focus light by illuminating the surrounding environment to capture one or more image frames including information about the surrounding environment. The imaging system may have sensor circuitry including one or more arrays of image sensor pixels (sometimes referred to herein as sensor pixels or pixels). The active pixels in the array may include photosensitive elements, such as photodiodes that convert incident light into charge. The array may have any number (e.g., hundreds or thousands or more) of pixels. The sensor circuitry may include control circuitry (such as circuitry for controlling the pixels) and readout circuitry for reading out an image signal corresponding to the charge generated by the photosensitive elements. If desired, the lighting module having the light source monitoring system may form part of other systems (such as non-imaging systems).

[0017] Figure 1 is a schematic diagram of an illustrative system having a lighting module, such as an imaging system 10 that uses sensor circuitry to capture images. Figure 1The imaging system 10 can be a standalone camera, a cellular phone, a tablet computer, a webcam, a video camera, a video surveillance system, an automotive imaging system, a video game system with imaging capabilities, an augmented reality and / or virtual reality system, an unmanned aerial vehicle system (such as a drone), an industrial system, or any other desired imaging system or device that captures digital image data. The camera module 12 (sometimes referred to as an imaging module) can be used to convert incident light into digital image data. The camera module 12 can include one or more corresponding sensor modules 16 (sometimes referred to as image sensor modules or image sensors). During an image capture operation, light from a scene can be focused onto the sensor module 16 through one or more corresponding lenses. For example, the sensor module 16 can include circuitry for generating an analog pixel image signal and circuitry for converting the analog pixel image signal into corresponding digital image data. The digital image data can be provided to the storage and processing circuitry 18.

[0018] The storage and processing circuitry 18 can include one or more integrated circuits, such as digital signal processing circuitry, image processing circuitry, a microprocessor, storage devices such as random access memory and non-volatile memory, and / or other types of processing and / or memory circuitry. The storage and processing circuitry 18 can be implemented using components separate from the camera module 12 and / or components that form part of the camera module 12. When the storage and processing circuitry 18 is implemented on an integrated circuit different from those implementing the camera module 12, the integrated circuit having the circuitry 18 can be vertically stacked or packaged together with the integrated circuits of the camera module 12. The captured image data can be processed and stored using the processing circuitry 18. As an example, the captured image data can be processed using an image processing engine on the processing circuitry 18, using a digital signal processing engine on the processing circuitry 18, using an imaging mode selection engine on the processing circuitry 18, and / or using other portions of the processing circuitry 18. If desired, the processed image data can be provided to the camera module 12 and / or the imaging system 10 (such as a computer, an external display, or other device) using a wired communication path and / or a wireless communication path coupled to the processing circuitry 18.

[0019] In some configurations described herein as illustrative examples, the camera module 12 can implement a time-of-flight (TOF) sensor or camera. In these configurations, the camera module 12 can include an illumination module 14 that is configured to emit light for illuminating an image scene or more specifically one or more objects in the image scene. The sensor module 16 can be configured to collect a reflected version of the emitted light and generate TOF information for the image scene, such as depth or distance information for one or more of the objects, a depth or distance map of the image scene, a visible light and / or infrared image of the image scene, and / or other TOF information.

[0020] Figure 2 is a schematic diagram of an exemplary illumination and sensor module, such as Figure 1 the illumination and sensor module in the imaging module 12 of Figure 2 shown. The illumination module 14 can emit light, and the sensor module 16 can receive corresponding reflected light after the emitted light is reflected from one or more objects 20 or an image scene generally containing the objects 20. The illumination module 14 can include one or more light sources, which are sometimes referred to herein as light emitters or illumination devices. In Figure 2 the example of

[0021]

[0022]

[0022] the light source emits light 22, and the light 22 reaches one or more objects 20 and is reflected from the one or more objects 20 as reflected light 24. The objects 20 can include any suitable inanimate or animate objects at different depths in the scene. The reflected light 24 can be received at the sensor module 16, or more specifically at one or more photosensitive elements in the active image pixels of the sensor module 16. The driver circuit and / or control circuit can control the pixels of the sensor module 16 by providing control signals to transistors or other actuating elements in the pixels, thereby generating one or more image frames based on the reflected light 24. In particular, based on the control signals received from the driver circuit and / or control circuit, the pixels can generate different portions of charge in response to the reflected light 24 during an accumulation or exposure period, can perform a readout operation on the generated charge portions during a readout period, and / or can perform other suitable operations during other periods.In some illustrative arrangements described herein as examples, the sensor module 16 may include a pixel array that contains sensor pixels arranged in rows and columns. Pixel control circuitry may be coupled to the lines (e.g., rows) of the pixels in the array, and pixel readout circuitry may be coupled to the lines (e.g., columns) of the pixels in the array. The pixel control circuitry may receive a row or column address from the timing control circuitry and provide corresponding row or column control signals, such as reset, anti-halo or global shutter, pixel select, modulation, storage, charge transfer, readout, sample and hold control signals, to the pixels via row or column control paths. One or more column and / or row readout paths may be coupled to each line of the pixels in the array. These readout paths may be used to read out image signals from the pixels and to provide bias signals (e.g., bias current or bias voltage) to the pixels. The pixel readout circuitry may receive image signals, such as analog pixel values generated by the pixels, and may include memory circuitry, amplifier circuitry or multiplier circuitry, analog-to-digital conversion (ADC) circuitry, bias circuitry, latch circuitry for selectively enabling or disabling different portions of the readout circuitry, or other circuitry coupled to one or more pixels in the array and / or for reading out image signals from the pixels. The readout circuitry may provide the resulting digital pixel data to the control and processing circuitry 18( Figure 1 ), for further processing, such as digital signal processing.

[0023] If desired, the pixel array may also be provided with a filter array having a plurality of color and / or filter elements each overlapping one or more pixels, thereby allowing a single image sensor to sample light of different colors or different wavelength groups. Generally, filter elements of any desired color and / or wavelength and any desired pattern may be formed over any desired number of image pixels. In an illustrative example of time-of-flight sensing using the Figure 1 and Figure 2 illumination module 14, the pixel array may be provided with a corresponding filter array that passes light having a color and / or frequency emitted from the illumination module 14.

[0024] Figure 1 The imaging module 12 in Figure 2 may be configured to be based on a modulated light signal (such as the modulated light 22 emitted by the Figure 2The phase difference between the light 24 received by the sensor module 16 in [description] is used to perform indirect TOF measurement. In these indirect TOF configurations, the sensor module 16 may include an active pixel array, and each active pixel is configured to modulate (demodulate) the received optical signal based on a modulation control signal having a modulation frequency received at the corresponding pixel transistor, so as to separate the charge generated by the photosensitive element into corresponding charge portions that can be used to generate TOF information.

[0025] Instead of being configured to perform indirect TOF, or in addition to being configured to perform indirect TOF, Figure 1 the image module 12 in [description] may be configured to perform direct TOF measurement in a pulse mode. In the direct TOF configuration, the sensor module 16 may directly measure the time difference between when the pulse of the emitted light 22 is transmitted and when the corresponding reflected form of the pulse (e.g., the pulse of the reflected light 24) is received and detected by the active pixel array in the sensor module 16 to generate TOF information.

[0026] Combined with Figure 1 and Figure 2 The TOF sensors described are merely illustrative. The illumination module 14 and the sensor module 16 may each include other suitable circuits, such as power management and supply circuits, processing circuits, control circuits, readout circuits, timing circuits, and / or clock generation circuits. Although the illumination module 14 and the sensor module 16 are shown as separate modules in Figure 1 and Figure 2 this is merely illustrative. If needed, the illumination module 14 and the sensor module 16 may be coupled to and include shared circuits in the camera module system, such as shared power management and / or supply circuits, shared modulation / demodulation circuits, shared clock generation circuits, shared timing controllers, shared signal generator circuits, shared control circuits, and / or shared storage circuits.

[0027] In some illustrative arrangements, the illumination module may be provided separately from the sensor module, may be provided outside the imaging module, and / or may be provided in other (non-imaging) systems.

[0028] Regardless of the type of system or module in which the illumination module is provided and / or the type of system or module with which the illumination module operates, it may be desirable to provide monitoring of the illumination module. This can help ensure the rapid and effective identification of different types of faults in the illumination module and / or enable desired fault mitigation techniques to be performed to reduce the possible harm to the surrounding environment.

[0029] Figure 3 An illustrative illumination module containing a monitoring system is shown. In Figure 3 the example of Figure 1 andFigure 2 The illumination module 14) therein may include one or more light sources 30, a modulation driver 32 (sometimes referred to herein as a driver circuit or a light source driver circuit), a clock generation circuit 34, and a light source monitoring system 36.

[0030] The light source 30 may include one or more lasers or laser diodes, one or more light emitting diodes (LEDs), and / or one or more other suitable types of light sources or illumination sources. The light source 30 may emit light of any suitable wavelength, such as visible light, infrared light, and / or light of other wavelengths.

[0031] The light source 30 may be controlled and driven by a corresponding driver circuit (such as the driver circuit 32) coupled to each light source 30. The driver circuit 32 may provide a drive signal (sometimes referred to as a control signal) having any suitable characteristics (such as a suitable waveform, a suitable peak amplitude or power, a suitable periodicity or frequency, and / or other characteristics) to each light source 30 to emit light 22 having corresponding desired characteristics. In some configurations described herein as examples, the driver circuit 32 may provide a drive signal having a high or low state at each phase of a clock signal having one or more frequencies between 30 MHz and 200 MHz. The drive signal may cause the light source 30 to generate modulated light 22 having light pulses based on the high and low states of the drive signal.

[0032] As an example, the drive signal may be in a high state during two consecutive time periods (clock phases) of the clock signal when the light source 30 is in an active state and emitting light 22, may be in a low state during two subsequent time periods (clock phases) of the clock signal when the light source 30 is in a non - active state and not emitting light 22, and / or may be in a high state then a low state during two subsequent time periods (clock phases) of the clock signal when the light source 30 is active for a first clock phase and then non - active for a second clock phase. This sequence of high and low states of the drive signal is merely illustrative.

[0033] The driver circuit 32 may be coupled to the clock generation circuit 34. The clock generation circuit 34 may include a phase - locked loop (PLL) that provides a clock signal to the driver circuit 32. Based on the clock signal, the driver circuit 32 may provide a corresponding drive signal to the light source 30.

[0034] To provide a (laser) light safety system, the illumination module 14 may include a light source monitoring system 36. To reduce the number of components required to monitor the operation of the illumination module 14 and / or to reduce disruption to an existing illumination module architecture, the light source monitoring system 36 may be coupled to the driver circuit 32 and / or the light source 30 such that the monitoring system 36 receives a drive signal provided from the driver circuit 32 to the light source 30. For example, the monitoring system 36 may be electrically connected to a drive contact pad at the driver circuit 32 and / or at the light source 30 that conveys the drive signal. In particular, the monitoring system 36 may sample the drive signal over time to generate sampled data for monitoring.

[0035] If desired, instead of or in addition to sampling the drive signal, the monitoring system 36 may sample other signals conveyed within, received by, and / or output by the illumination module 14. In one illustrative arrangement, instead of or in addition to sampling the drive signal, the monitoring system 36 may also include one or more photodetectors that sample an optical signal (e.g., light 22) to generate sampled data for monitoring. Generally, any desired electrical and / or optical light source signals of the light source may be sampled for monitoring.

[0036] In some illustrative arrangements described herein as examples, the electrical and / or optical light source signals being sampled may be modulated at a high frequency. Accordingly, a large number of samples may be generated over time. Additionally, in some applications such as TOF sensing, the illumination module 14 may operate in different operating modes corresponding to different imaging modes (e.g., in conjunction with the sensor module 16). As an example, the illumination module 14 may emit light for different image exposure time periods, for different image frame depths, and / or for other operating modes. The light emission may be monitored for each of these different operating modes and generally within various time intervals or monitoring time windows to appropriately characterize the illumination module performance, to determine whether the operation in any of these modes violates a light emission limit, and / or to determine whether a different system failure has occurred. It may be difficult to effectively monitor the illumination module performance, especially in these high-frequency multi-mode arrangements and in real-time monitoring. Centralized processing of the data would be advantageous.

[0037] Figure 4 An illustrative light source monitoring system is shown, such as Figure 3 the monitoring system 36 in Figure 4 which is configured to monitor a light source signal based on data blocks and data groups. As shown, the light source monitoring system 36 may include a data chunking circuit 40, a data block grouping circuit 42, and a failure (or fault) detection circuit 44.

[0038] In particular, the data chunking circuit 40 may aggregate, organize, or otherwise process data samples obtained from an observed drive signal (or optical signal) into data chunks of different sizes to produce corresponding data chunk values. In some configurations described herein as illustrative examples, the number of samples is represented by a bitstream of binary 0s and 1s, where each value of 0 or 1 is a sample. The value 0 may indicate the time (period) when the light source is in an inactive state (not producing light), and the value 1 may indicate the time (period) when the light source is in an active state (producing light). The values of the bitstream may each correspond to or indicate the state of the drive or optical signal at the corresponding clock phase of the clock signal used to generate the drive signal provided to the light source 30. For example, a high state of the signal may indicate that the light source is in an active state, and a low state of the signal may indicate that the light source is in an inactive state. When corresponding to a drive signal, the values of the bitstream may be detected by a digital counter in the system 36 coupled to the signal path that conveys the drive signal. When corresponding to an optical signal, the values of the bitstream may correspond to the on-off state of the light detected as emitted by the light source 30 and may be detected by a photodetector in the system 36.

[0039] The time period of a particular data chunk may include information from a desired number of samples, such as one sample, two samples, three samples, four samples, eight samples, or any suitable number of data samples. In particular, the data chunking circuit 40 may generate a data chunk value that indicates the number of time periods during which the light source is in an active state within the data chunk time period. In other words, this data chunk value may correspond to the number of samples having a value of 1 within that time period.

[0040] The data chunking circuit 40 may also average the number of light source activation time periods over multiple such data chunk time periods to generate an additional average time chunk value for an additional data chunk. In this way, these (average) time chunk values may include information from a greater number of samples, such as 32 samples (when averaging 8 four-sample values), 256 samples (when averaging 64 four-sample values), 2048 samples (when averaging 512 four-sample values), and / or other suitable numbers of samples.

[0041] The data chunking circuit 40 can generate multiple data chunk values associated with data chunks of different sizes in parallel within the same time interval. As an illustrative example, within the same time period of 5120 nanoseconds (ns), the data chunking circuit 40 that samples the light source signal can generate 512 chunks of 10 ns in size, 64 chunks of 80 ns in size, 8 chunks of 640 ns in size, and / or 1 chunk of 5120 ns in size. For each of the 512 chunks of 10 ns in size, the data chunking circuit 40 can generate a first data chunk value that indicates the number of light source activation time periods within that 10 ns time interval. For each of the 64 chunks of 80 ns in size, the data chunking circuit 40 can generate a second data chunk value that indicates the number of light source activation time periods within that 80 ns time interval. For each of the 8 chunks of 640 ns in size, the data chunking circuit 40 can generate a third data chunk value that indicates the number of light source activation time periods within that 640 ns time interval. For the single chunk of 5120 ns in size, the data chunking circuit 40 can generate a fourth data chunk value that indicates the number of light source activation time periods within that 5120 ns time interval. In an illustrative configuration where each 10 ns time interval contains 4 samples, each chunk of 10 ns in size can contain information of 4 samples, each chunk of 80 ns in size can contain information of 32 samples, each chunk of 640 ns in size can contain information of 256 samples, and each chunk of 5120 ns in size can contain information of 2048 samples.

[0042] The sizes and / or durations of the data chunks as described above are merely illustrative. Any suitable sizes and / or durations of the data chunks can be adjusted based on the implementation of the monitoring system.

[0043] The monitoring system 36 can also aggregate the data chunk values of different sizes generated by the data chunking circuit 40 to generate data group values respectively associated with data groups or data chunk groupings. In particular, the data chunk grouping circuit 42 in the system 36 can receive each data chunk value associated with data chunks of different sizes from the data chunking circuit 40. The data chunk grouping circuit 42 can use any suitable number of data chunk values associated with data chunks of the same desired size to generate corresponding data group values. The number and / or size of the data chunks used can be associated with the size of the monitored time window and based on the size of the monitored time window. Multiple such data group values can be constructed using different numbers of data chunks and / or using different sizes of data chunks to provide monitoring of multiple monitoring time windows with the desired granularity. In other words, the data chunk grouping circuit 42 can generate group values for multiple monitors in the failure detection circuit 44.

[0044] In the configurations described herein as illustrative examples, the data group value can be the sum of a suitable number of data block values and / or can be any other suitable type of aggregate value that contains information about multiple data block values. As some illustrative examples, the data block grouping circuit 42 can receive a plurality of block values each associated with a data block of 10 ns size, and can add 100 such block values to produce a group value for a 1 μs (microsecond) group time period, can add 1000 such block values to produce a group value for a 10 μs group time period, and / or can add any suitable number of such block values to produce a group value for a corresponding group time period of a desired duration; the data block grouping circuit 42 can receive a plurality of block values each associated with a data block of 80 ns size, and can add 1000 such block values to produce a group value for an 80 μs group time period, can add 4000 such block values to form a 320 μs group time period, and / or can add any suitable number of such block values to produce a group value for a corresponding group time period of a desired duration; the data block grouping circuit 42 can receive a plurality of block values each associated with a data block of 640 ns size, and can add 1000 such block values to produce a group value for a 640 μs group time period, can add 4000 such block values to produce a 2560 μs group time period, and / or can add any suitable number of such block values to produce a group value for a corresponding group time period of a desired duration; the data block grouping circuit 42 can receive a plurality of block values each associated with a data block of 5120 ns size, and can add 1000 such block values to produce a group value for a 5120 μs group time period, and / or can add any suitable number of such block values to produce a group value for a corresponding group time period of a desired duration.

[0045] As illustrated by the above examples, the data block grouping circuit 42 can generate group values for group time periods of various durations. Thus, the monitoring system 36, or more specifically, different monitors for different monitoring channels in the failure detection circuit 44, can each monitor the performance of the light source within a group time period of a corresponding duration.

[0046] Specifically, the data block grouping circuit 42 can be coupled to the failure detection circuit 44. The failure detection circuit 44 can obtain a plurality of group values for non-overlapping group time periods of the same duration to generate a rolling sum that includes a rolling monitoring time period for each non-overlapping group time period. The failure detection circuit 44 can also obtain a corresponding threshold level for each set of group values. The failure detection circuit 44 can compare the rolling sum with one or more threshold levels, and based on this comparison, can determine whether a failure has occurred. As an example, when the rolling sum exceeds one or more threshold levels, the failure detection circuit 44 can output a signal indicating a failure or fault.

[0047] The failure detection circuit 44 can receive group values for different sets of time periods and can generate corresponding running sums of the group values for each different set of time periods. As an example, the failure detection circuit 44 can obtain a first running sum that adds the group values for 200 1-μs group time periods (thereby forming a 200-μs running monitoring time window), a second running sum that adds the group values for 200 10-μs group time periods (thereby forming a 2-ms (millisecond) running monitoring time window), a third running sum that adds the group values for 200 80-μs group time periods (thereby forming a 16-ms running monitoring time window), a fourth running sum that adds the group values for 200 320-μs group time periods (thereby forming a 64-ms running monitoring time window), a fifth running sum that adds the group values for 200 640-μs group time periods (thereby forming a 128-ms running monitoring time window), a sixth running sum that adds the group values for 200 2560-μs group time periods (thereby forming a 512-ms running monitoring time window), a seventh running sum that adds the group values for 200 5120-μs group time periods (thereby forming a 1024-ms running monitoring time window), and / or any other suitable running sum. In this manner, the failure detection circuit 44 can include multiple (e.g., seven in the above example) monitors or monitoring channels for monitoring time windows of different durations.

[0048] The number and / or duration of the group time periods for each running sum as described in the above example are merely illustrative. If desired, any suitable number and / or duration of group time periods can be used to generate any suitable number of running sums for monitoring. The failure detection circuit 44 can set and / or obtain different threshold levels for comparison with different running sums and / or can set and / or obtain the same threshold level for comparison with at least some (e.g., all) of the running sums.

[0049] If desired, the system 36 can include a failure mitigation circuit coupled to one or more (e.g., all) of the outputs of the failure detection circuit 44 and can perform one or more mitigation operations in response to a detected fault or error. As an example, these mitigation operations can include deactivating one or more components of the lighting module 14, such as one or more faulty light sources, portions of the driver circuit having a fault, and / or the clock generation circuit; adjusting the control signal provided to the light source; outputting a notification to a user of the lighting module 14; and / or other types of mitigation operations. Adjusting the control signal can include reducing the light source output power or otherwise adjusting the operation of the light source to correct the detected failure or fault.

[0050] Figure 5 is a schematic diagram of a set of exemplary samples in a light source signal, represented as multiple groups, each group including a corresponding number of blocks. In Figure 5In an example, a signal (such as Figure 3 the signal transmitted from the driver circuit 32 to the light source 30 in

[0051] may include a plurality of samples 50 obtained and detected by the optical monitoring system 36. The system 36 may obtain the samples 50 in a data set 52, and each data set 52 further includes a plurality of data blocks 54. In particular, for each data block 54, the system 36 may generate a count indicating the number or average number of times the signal is asserted or the light source is activated during the block time period. For example, whenever the signal is asserted or the light source is activated, the sampled bit stream may have a value of 1. The number of times the bit stream has a value of 1 during the block time period may be averaged. For each data set 52, the system 36 may generate a sum that adds up the counts of each data block 54 included in that data set 52. For each observation or monitoring time period, the system 36 may generate an additional sum that adds up the sums of each data set 52 in the observation or monitoring time window. The observation time window may be a rolling time period of a fixed length that is updated and monitored over time, and thus, the additional sum may be a rolling sum that is similarly updated over time.

[0052] In Figure 5 an example, the time period 60 may represent the monitoring time window monitored by the system 36. The samples in the time period 60 may be represented by data sets 62-1, 62-2, 62-3,..., 62-N. Each of the data sets 62 may include a corresponding set of data blocks 64, and each data block contains information about a non-overlapping subset of the samples in the time period 60.

[0053] The time period 60 may represent the monitoring time window at a specific moment. In other words, the information from groups 62-1 to 62-N may be aggregated (e.g., summed) to generate a value for monitoring. For example, the generated value may be compared with one or more threshold levels indicating one or more system failures. The time window monitored by the system 36 may be a running or rolling time period that changes over time. Thus, at a subsequent moment, the information from group 62-1 may be removed and the information from group 62-(N + 1) may be added, such that the information from groups 62-2 to 62-N generates a new value, such as a new sum of the rolling sum, for monitoring and comparing with one or more threshold levels. In this way, a rolling time window that produces a rolling sum formed by a set of rolling groups 52 may be used to monitor the operation of the lighting module.

[0054] Figure 5 shows in the system 36 (e.g., Figure 4An example of an exemplary monitor or monitoring channel in the failure detection circuit 44. To provide multiple monitors in the detection circuit 44 that monitor multiple observation time windows of various durations and / or with different granularities, the system 36 can organize the samples 50 into multiple data blocks of different sizes. Figure 5 This set of exemplary data blocks 54 in can have a first size, where each data block contains four samples. The system 36 can additionally organize the same set of samples 50 into other groups of data blocks of a second size, a third size, a fourth size, and / or any other suitable size. Depending on the granularity and / or size of the time window to be monitored, the system 36 can provide monitors (e.g., in the failure detection circuit 44), each monitor monitoring an observation time window with a different duration and / or different granularity. Each of the different observation time windows can provide a value based on a different number of groups, where each group aggregates the block values from a different number of data blocks and / or data blocks of different sizes.

[0055] In other words, the duration of the monitoring time window (such as the monitoring time window 60), the number of groups 62, the number of data blocks 64, and / or the size of each of the data blocks and / or each of the data groups (e.g., the number of samples) can be adjusted for the system 36 to provide other monitors in addition to Figure 5 the monitors shown in the example of.

[0056] Figure 6 is a schematic diagram of an exemplary implementation of a light source monitoring system (such as Figure 4 the light source monitoring system 36 in). As Figure 6 shown in the example of, the data chunking circuit 40 can include one or more counter circuits 70 (such as one or more digital counters), an adder circuit 72, a multiplier circuit 74, and multiple averaging circuits 78-1, 78-2, 78-3. For each light source 30, there can be one counter circuit in the data chunking circuit 40. The counter circuit 70 can detect and count each activation state of the light source signal for the corresponding clock phase of the clock signal. The counter circuit 70 can then generate a sample indicating the number of light source activation time periods at the clock phase of the clock signal. The adder circuit 72 can add together the counts generated by the counter circuit 70 for each set number of samples. The set number of samples can be four or any other suitable number. The adder circuit 72 can then generate a count of the number of activation time periods for each set number of time periods of the light source signal (e.g., the number of 1s in the bit stream).

[0057] If desired, the count generated by adder circuit 72 may be scaled at multiplier circuit 74 by an adjustable scaling factor. In some illustrative configurations described herein as examples, the modulation frequency of the light source signal may vary over time, resulting in a change in the number of samples within a fixed time period. A computing circuit, such as an arithmetic circuit, may be coupled to Figure 3 the clock generation circuit 34 therein and may generate a scaling factor based on the frequency of the clock signal generated by the clock generation circuit 34. By scaling the number of samples, multiplier circuit 74 may generate a count per time interval rather than a count per set of samples.

[0058] The resulting count output from multiplier circuit 74 may be provided along path 76-1 as a count value for a first data block or a data block of a first size (e.g., a data block value of 4 samples as determined by the number of samples added by adder circuit 72).

[0059] Additionally, over time, multiple resulting counts output by multiplier circuit 74 may also be transmitted to first averaging circuit 78-1. First averaging circuit 78-1 may average multiple resulting counts output by multiplier circuit 74 over time (e.g., eight or other suitable number of counts). A ceiling circuit 80-1 may apply a ceiling function to produce a resulting average count on path 76-2. This average count on path 76-2 may be for a second data block or a data block of a second size.

[0060] Additionally, over time, multiple resulting average counts output by averaging circuit 78-1 may also be transmitted to second averaging circuit 78-2. Second averaging circuit 78-2 may average multiple resulting average counts output by averaging circuit 78-1 over time (e.g., eight or other suitable number of average counts). A ceiling circuit 80-2 may apply a ceiling function to produce another resulting average count on path 76-3. This average count on path 76-3 may be for a third data block or a data block of a third size.

[0061] Additionally, over time, multiple resulting average counts output by averaging circuit 78-2 may also be transmitted to third averaging circuit 78-3. Third averaging circuit 78-3 may average multiple resulting average counts output by averaging circuit 78-2 over time (e.g., eight or other suitable number of average counts). A ceiling circuit 80-3 may apply a ceiling function to produce another resulting average count on path 76-4. This average count on path 76-4 may be for a fourth data block or a data block of a fourth size.

[0062] Typically, any suitable number of averaging circuits 78 and corresponding upper limit circuits 80 can be provided to supply different data block values in parallel to the data block grouping circuit 42. These averaging circuits 78 can be coupled to the counter 70. If needed, the adder circuit 72 and multiplier circuit 74 can be coupled between these averaging circuits 78 and the counter 70.

[0063] The data block grouping circuit 42 can include a multiplexer 82 and a summing circuit 84. A multiplexer (such as multiplexer 82) can have inputs coupled to paths 76-1, 76-2, 76-3, and 76-4. The values transmitted on the path 76 can represent data block values of different sizes, and each data block value contains information about a different number of samples. The multiplexer can then select one of the data block values on paths 76-1, 76-2, 76-3, and 76-4 based on a control signal received at the multiplexer to be passed to the output of the multiplexer 82.

[0064] The summing circuit 84 (sometimes referred to as a group summing circuit) can be coupled to the output of the multiplexer 82. The summing circuit 84 can use a series of data block values from one of paths 76-1, 76-2, 76-3, or 76-4 generated over time to generate a group value (e.g., a group sum). As an example, when summing or otherwise aggregating the values of data blocks 64-1, 64-2, ..., 64-M, the summing circuit 84 can perform the addition operation described to generate the group value of data group 62-3. The values of data blocks 64-1, 64-2, ..., 64-M can all be provided on the same path 76 (e.g., path 76-2) because data blocks 64-1, 64-2, ..., 64-M can have the same size. In fact, when summing the corresponding data block values to generate group values for other data groups 62 (such as data groups 62-1, 62-2, 62-N, 62-(N+1)), the data block values from the same path 76 (e.g., path 76-2) can be used because each group 62 can aggregate data block values of the same size. Figure 5 These group values (such as group 62(

[0065] These group values (such as group 62( Figure 5) Each of the grouped values can be generated sequentially based on summing the series of expected data block values from the same path 76. When generating each grouped value, the summing circuit 84 can output the generated value to a memory or data storage circuit (such as the first-in, first-out (FIFO) memory circuit 88 in the fault detection circuit 44) coupled to the summing circuit 84 for storage. If needed, the data block grouping circuit 42 can also include a grouped value shifting circuit 86 coupled between the summing circuit 84 and the memory circuit 88. The value shifting circuit 86 can adjust the generated grouped value, or more specifically, remove the excess portion (e.g., fix or remove a certain number of bits) of the grouped value generated by the summing circuit 84 to allow for standardized and compact storage at the memory circuit 88.

[0066] The fault detection circuit 44 can include a memory circuit 88, a summing circuit 90, and a comparison circuit 92. The memory circuit 88 can be configured to store any suitable number of grouped values for multiple groups over a monitoring time window. By Figure 5 way of example, the memory circuit 88 can store N grouped values, each grouped value corresponding to one of the groups 62-1, 62-2, 62-3,..., 62-N.

[0067] The summing circuit 90 (sometimes referred to as a sliding summing circuit or generally as an arithmetic circuit) can be configured to generate a rolling sum associated with a rolling monitoring time window. In particular, the summing circuit 90 can obtain the current sum of the N grouped values for the current monitoring time period and provide this current sum to the comparison or comparator circuit 92 in the detection circuit 44. The current sum can be stored in the summing circuit 90. The comparison circuit 92 can compare the current sum with one or more threshold levels. Based on this comparison, the detection circuit 44 can provide an output that indicates whether any faults are detected based on the current sum associated with the current monitoring time period.

[0068] For subsequent sums in the set of rolling sums, the summing circuit 90 can subtract the oldest grouped value from the current sum and add Figure 5 the newest grouped value in Figure 5 to the current sum to obtain the subsequent sum. As Figure 5 shown, the grouped value of group 62-1 can be subtracted from the illustrated time period 60, and the grouped value of group 62-(N+1) can be added to obtain the subsequent sum for the rolling time period. The summing circuit 90 can obtain the oldest grouped value from the first entry stored in the FIFO memory 88 (and first removed), and can obtain the new grouped value directly from the summing circuit 84 (if needed, through the shifting circuit 85). This newly generated subsequent sum can be the new current sum and is compared with one or more threshold levels at the comparison circuit 92. This new current sum can then be updated in a similar manner to continue generating the rolling sum.

[0069] Although the block grouping circuit 42 and the detection circuit 44 are shown with a set of grouping and detection components (i.e., multiplexer 82, group summing circuit 84, group value shifting circuit 86, FIFO memory circuit 88, sliding summing circuit 90, and threshold comparison circuit 92), this configuration is merely illustrative. As described above, the monitoring system 36 can perform multiple monitoring operations in parallel using multiple monitors or monitoring channels, each monitor or monitoring channel for monitoring a different monitoring time window, which are associated with corresponding rolling sum values obtained from group values containing different numbers of data block values. Thus, the block grouping circuit 42 can include one or more additional sets of multiplexers 82, group summing circuits 84, and group value shifting circuits 86, and the detection circuit 44 can include one or more additional sets of FIFO memory circuits 88, sliding summing circuits 90, and threshold comparison circuits 92 to form these additional monitors or monitoring channels.

[0070] If desired, at least some of these components in circuits 42 and 44 can be shared among the components of different monitoring channels. As an example, the multiplexer 82 can have a shared output or multiple individual outputs coupled to multiple group summing circuits for different monitoring channels, the group summing circuit 84 can be used to calculate group values for different monitoring channels, the group value shifting circuit 86 can be shared among the group summing circuits for different monitoring channels, multiple FIFO memory circuits 88 can be implemented on different portions of a shared memory integrated circuit (e.g., an integrated circuit implementing random access memory), and / or the comparison circuit 92 can be shared among different monitoring channels.

[0071] In Figure 6 the example of, an illustrative configuration of the data block circuit 40 with a counter 70, adder circuit 72, multiplier circuit 74, multiple averaging circuits 78, and multiple upper limit circuits 80 is shown for sampling and chunking samples of a single light source signal. If desired, one or more additional sets of counters 70, adder circuits 72, multiplier circuits 74, averaging circuits 78, and upper limit circuits 80 can be included as part of the data chunking circuit 40 for sampling and chunking samples of one or more additional light source signals of corresponding light sources in parallel with the sampling and chunking of samples of the first light source signal. The additional one or more sets of counters 70, adder circuits 72, multiplier circuits 74, averaging circuits 78, and upper limit circuits 80 can similarly include corresponding monitoring channels implemented by some parts of the block grouping circuit 42 and the detection circuit 44.

[0072] Various embodiments showing a light source monitoring system have been described.

[0073] As an example, an imaging system may include a sensor module having an array of image sensor pixels and may include an illumination module having a light source, a driver circuit coupled to the light source, and a light source monitoring system coupled to the light source and configured to receive a light source signal. The light source monitoring system may include: a data chunking circuit that generates a plurality of data chunk values based on samples from the light source signal; a grouping circuit that generates a plurality of group values, each group value aggregating at least some of the plurality of data chunk values; and a failure detection circuit that generates an output signal indicating a fault based on a monitoring time window characterized by at least some of the plurality of group values.

[0074] As another example, an illumination module may include a light source and a light source monitoring system coupled to the light source and configured to receive a light source signal. The light source monitoring system may include a counter configured to generate samples of the light source signal, a plurality of averaging circuits coupled to the counter, a multiplexer having a plurality of inputs coupled to the plurality of averaging circuits and having an output, a first summing circuit coupled to the output of the multiplexer, a memory circuit coupled to the first summing circuit, a second summing circuit coupled to the memory circuit and the first summing circuit, and a comparison circuit coupled to the second summing circuit.

[0075] As yet another example, a time-of-flight sensor includes: a sensor module having an array of image sensor pixels; an illumination module having a light source; a driver circuit coupled to the light source; and a light source monitoring system coupled to the light source and configured to receive a drive signal provided to the light source from the driver circuit. The light source monitoring system may include a first monitor and a second monitor, the first monitor being configured to detect one or more faults using the drive signal and a first observation time window having a first duration, and the second monitor being configured to detect one or more faults using the drive signal and a second observation time window having a second duration.

[0076] The foregoing is merely an illustrative explanation of the principles of the present invention, and those skilled in the art can make various modifications without departing from the scope and essence of the present invention. The above embodiments can be implemented individually or in any combination.

Claims

1. An imaging system, comprising: A sensor module having an image sensor pixel array; And An illumination module having a light source, a driver circuit coupled to the light source, and a light source monitoring system coupled to the light source and configured to receive a light source signal, wherein the light source monitoring system comprises: A data chunking circuit that generates a plurality of data chunk values based on samples from the light source signal; A grouping circuit that generates a plurality of group values, each group value aggregating at least some of the plurality of data chunk values; and A failure detection circuit that generates an output signal indicating a fault based on a monitoring time window characterized by at least some of the plurality of group values.

2. The imaging system according to claim 1, wherein each of the plurality of data chunk values contains information about a plurality of samples.

3. The imaging system according to claim 2, wherein the information about the plurality of samples includes the time at which the light source signal is asserted.

4. The imaging system according to claim 2, wherein each of the plurality of group values contains a sum of the plurality of data chunk values.

5. The imaging system according to claim 4, wherein the failure detection circuit generates an additional sum for the monitoring time window, the additional sum adding at least some of the plurality of group values.

6. The imaging system according to claim 5, wherein the failure detection circuit performs a comparison of the additional sum with one or more threshold levels and generates the output signal based on the comparison.

7. The imaging system according to claim 6, wherein the monitoring time window is a rolling time window that changes over time, and wherein the additional sum is a rolling sum that changes over time.

8. The imaging system according to claim 1, wherein the failure detection circuit monitors the monitoring time window for a first monitoring channel of the light source signal and monitors one or more additional monitoring time windows for one or more additional monitoring channels of the light source signal.

9. The imaging system according to claim 1, wherein the light source monitoring system is electrically connected to a path between the light source and the driver circuit, the path transmitting a drive signal from the driver circuit to the light source, and wherein the light source signal is the drive signal.

10. The imaging system according to claim 1, wherein the light source monitoring system includes a photodetector, and wherein the light source signal is based on light emitted by the light source and detected by the photodetector.

11. An illumination module, comprising: A light source; And A light source monitoring system coupled to the light source and configured to receive a light source signal, wherein the light source monitoring system comprises: A counter configured to generate samples of the light source signal; A plurality of averaging circuits coupled to the counter; A multiplexer having a plurality of inputs coupled to the plurality of averaging circuits and having an output; A first summing circuit coupled to the output of the multiplexer; A memory circuit coupled to the first summing circuit; A second summing circuit coupled to the memory circuit and the first summing circuit; and A comparison circuit coupled to the second summing circuit.

12. The lighting module according to claim 11, wherein a given averaging circuit among the plurality of averaging circuits generates a set of average values over time, and wherein the first summing circuit generates a set of grouped values over time based on the set of average values.

13. The lighting module according to claim 12, wherein the memory circuit stores the set of grouped values, and wherein the second summing circuit generates a running sum over time based on the set of grouped values.

14. The lighting module according to claim 11, further comprising: An adder circuit; And A multiplier circuit, wherein the adder circuit and the multiplier circuit are coupled between the counter and the plurality of averaging circuits.

15. The lighting module according to claim 14, further comprising: A plurality of ceiling circuits that perform a ceiling function on the outputs of the plurality of averaging circuits and are coupled between the plurality of averaging circuits and the multiplexer.

16. The lighting module according to claim 15, further comprising: A grouped value shifting circuit that changes the number of bits of the output of the first summing circuit and is coupled between the first summing circuit and the storage circuit.

17. A time-of-flight sensor, comprising: A sensor module having an image sensor pixel array; And A lighting module having a light source, a driver circuit coupled to the light source, and a light source monitoring system coupled to the light source and configured to receive a drive signal provided to the light source from the driver circuit, wherein the light source monitoring system includes: A first monitor configured to detect one or more faults using the drive signal and a first observation time window having a first duration; And A second monitor configured to detect the one or more faults using the drive signal and a second observation time window having a second duration.

18. The time-of-flight sensor according to claim 17, wherein the first monitor generates a first value of the first observation time window based on a first set of average values aggregated from samples of the drive signal, and wherein the second monitor generates a second value of the second observation time window based on a second set of average values aggregated from the samples of the drive signal.

19. The time-of-flight sensor according to claim 18, wherein the first set of average values has a plurality of average values each containing information about a first number of samples, and wherein the second set of average values has a plurality of average values each containing information about a second number of samples greater than the first number of samples.

20. The time-of-flight sensor according to claim 18, wherein the number of average values of the second set of average values is greater than the number of average values of the first set of average values.

Images