Optical sensor diagnostics

Abstract

A method (100) for diagnosing an optical sensor, the optical sensor comprising a photodetector and an integrator. The method comprises: exposing (110) the photodetector to incident light; obtaining (120) an initial integration signal at an initial frame; performing at least once: changing (130) at least one control parameter of the optical sensor, exposing (110) the photodetector to incident light, and obtaining (140) one or more subsequent integration signals at subsequent frames; obtaining (150) a characteristic of the optical sensor from the obtained integration signals; comparing (160) the obtained characteristic to a predetermined characteristic of the optical sensor to diagnose the optical sensor.

Description

Technical Field

[0001] This invention relates to the field of optical sensors. More specifically, it relates to a method and system for diagnosing optical sensors. Background of the Invention

[0002] Generally, optical sensors, and more specifically image sensors (2D or 3D), convert the amount of light into an output value. For example, the output value can be a voltage or a digital code. When using these sensors in safety-critical systems, diagnostic coverage is an important characteristic.

[0003] For the electrical components of such sensor systems, diagnostic functionality can be added without significant impact. However, for the conversion components, where incident light is converted into electrical values, adding diagnostic procedures is not straightforward and is often expensive.

[0004] For example, this can be accomplished using an additional light source specifically designed for diagnosing optical sensors. However, this results in a complex and expensive optical design. The operation of such sensors is discontinuous. For diagnostics, the additional light source needs to be turned on and monitored by the optical sensor to determine if it is still functioning. Therefore, the frame rate of the optical sensor is sacrificed for diagnostics. In image sensors, typically not all pixels are covered by such diagnostic systems.

[0005] In some existing technology systems, diagnostics are achieved by providing redundant optical systems. In this case, an additional optical sensor is used to verify the operation of another optical sensor. However, implementing redundancy does significantly increase cost and complexity.

[0006] Additionally, some systems use software algorithms to detect faults. Such systems are typically nondeterministic in terms of predictability or coverage.

[0007] Therefore, there is a need for a good method and system for diagnosing optical sensors that are less complex than redundant systems. Summary of the Invention

[0008] The purpose of embodiments of the present invention is to provide a good method and system for diagnosing optical sensors.

[0009] The above objectives are achieved by the method and apparatus according to the present invention.

[0010] In a first aspect, embodiments of the present invention relate to a method for diagnosing an optical sensor, the optical sensor including a photodetector and an integrator.

[0011] The method includes:

[0012] Expose the photodetector to incident light.

[0013] The initial integral signal is obtained at the initial frame.

[0014] Perform the following steps at least once: change at least one control parameter of the optical sensor, expose the photodetector to the incident light, and obtain one or more subsequent integrated signals in subsequent frames.

[0015] The process involves obtaining the integrated signal frame by frame, with each frame including resetting the optical sensor, accumulating signals from the circuit connecting the photodetector and the integrator during the integration time, and reading at least one sequence of the integrated signal.

[0016] The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0017] The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor.

[0018] An advantage of embodiments of the present invention is that faults in the optical sensor can be detected by changing the control parameters of the optical sensor and by comparing the expected change in the integrated signal with the measured change in the integrated signal. One or more control parameters can be changed.

[0019] In embodiments of the invention, the predefined characteristics may include a threshold within which the acquired characteristics should fall. If this is not the case, the method may be applicable to indicating faults in optical sensors.

[0020] In embodiments of the present invention, the time periods for resetting, accumulating, and reading are referred to as frames or subframes. Several subframes may exist within a macroframe. Control parameters may be changed at the macroframe level or the subframe level. They may be changed randomly or modulated repeatedly.

[0021] An advantage of embodiments of the present invention is that, by using multiple integrated signals to obtain the characteristics of the optical sensor, changes in incident light intensity that may be misinterpreted as a malfunction of the optical sensor can be filtered out.

[0022] An advantage of embodiments of the present invention is that the diagnostic method can be run without affecting the frame rate of the optical sensor. This is, for example, impossible in prior art systems that use an additional light source to diagnose the optical sensor.

[0023] In an embodiment of the present invention, at least one control parameter is the integral time.

[0024] An advantage of embodiments of the present invention is that optical sensors can be diagnosed by changing the integration time. If, for the same intensity of incident light, the accumulated signal does not correspond to the change in integration time, this indicates a malfunction of the optical sensor.

[0025] In embodiments of the present invention, the integration time has a ratio ranging from 50% to 150%.

[0026] As mentioned above, the integration time can be altered within a certain range, typically limited to 50%-150% of the original integration time. The integration time is at the low end, constrained by the minimum required sensitivity. When the integration time is reduced too low, the resulting pixel response may be too low for the application, and the signal-to-noise ratio (SNR) drops below acceptable levels. At the high end, the integration time is limited by saturation. When the integration time is increased too much, the pixel begins to saturate. This means that the pixel output is limited in such cases and is no longer responsive to additional light. At this point, the SNR experiences a sharp degradation.

[0027] In embodiments of the present invention, the duration of the integration time may be finite, such that changes in the intensity of the incident light are impossible during the integration period.

[0028] In embodiments of the invention, the time period for changing the control parameters of the optical sensor is thus negligible compared to the integration time.

[0029] In an embodiment of the present invention, at least one control parameter is the gain of the optical sensor.

[0030] In embodiments of the invention, the gain of the optical sensor is changed by altering the capacitance value of the integrator. The integrator can be, for example, a capacitor comprising two or more sub-capacitors connected in parallel. The total capacitance value can be changed by connecting or disconnecting additional sub-capacitors.

[0031] In an embodiment of the present invention, the integral signal V 输出 (t) is modeled as g*t+V 偏移 , where g and V 偏移 The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0032] In this equation, parameter g is a function of the gain of the incident light and the optical sensor. The gain can be changed by altering the capacitance of the integrator. Parameter t is the integration time.

[0033] In an embodiment of the present invention, the characteristics of the optical sensor are obtained by dividing the subsequent integrated signal.

[0034] This result can be compared with predetermined characteristics to determine whether the optical sensor is still working correctly.

[0035] In an embodiment of the present invention, when the integration time is changed, the ratio of the subsequent integration signal can be compared with the ratio of the integration time.

[0036] In an embodiment of the present invention, when the capacitance value is changed, the ratio of the integrated signal can be compared with the inverse ratio of the capacitance value to determine whether the optical sensor is still working correctly.

[0037] In a second aspect, embodiments of the present invention relate to an optical sensor. The optical sensor includes a photodetector, an integrator, and circuitry connecting the photodetector and the integrator, such that a signal from the photodetector, or a processed version thereof, can be accumulated on the integrator. The optical sensor also includes a controller configured to:

[0038] The initial integral signal is obtained at the initial frame.

[0039] Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames.

[0040] The process involves obtaining the integrated signal frame by frame, with each frame including resetting the optical sensor, accumulating signals from the circuit connecting the photodetector and the integrator during the integration time, and reading at least one sequence of the integrated signal.

[0041] The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0042] The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor.

[0043] In embodiments of the present invention, the integrator may be a capacitor.

[0044] In embodiments of the present invention, the integrator may include a plurality of parallel capacitors, wherein the controller is configured to connect one or more capacitors to the integrator or disconnect one or more capacitors to the integrator.

[0045] In an embodiment of the invention, the optical sensor includes a gain transistor configured to connect to or disconnect a capacitor among a plurality of parallel capacitors from the integrator.

[0046] An advantage of this embodiment of the invention is that the gain can be varied between frames and / or subframes. This change in gain should be reflected in the change in the integrated signal and used by the controller to trigger fault diagnosis of the optical sensor.

[0047] In an embodiment of the invention, the optical sensor includes a reset transistor configured to reset the integrator voltage.

[0048] In a third aspect, embodiments of the present invention relate to a camera including a plurality of pixels. Each pixel includes a photodetector, an integrator, and circuitry connecting the photodetector and the integrator. The camera includes a controller configured for each pixel to:

[0049] The initial integral signal is obtained at the initial frame.

[0050] Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames.

[0051] The process involves obtaining the integrated signal frame by frame, with each frame including resetting the optical sensor, accumulating signals from the circuit connecting the photodetector and the integrator during the integration time, and reading at least one sequence of the integrated signal.

[0052] The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0053] The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor.

[0054] In embodiments of the invention, the camera is a 3D camera, and the controller is configured to acquire the depth value of a pixel as a characteristic of the pixel. The acquired characteristic is compared with predefined characteristics to verify that the pixel depth does not significantly change for different measurement values.

[0055] In a fourth aspect, embodiments of the invention relate to an indirect time-of-flight system for measuring the time of flight of a modulated light signal reflected from an object. The indirect time-of-flight system includes an optical sensor according to embodiments of the invention for receiving the reflected light signal. The time-of-flight system is configured to obtain a measurement of the time of flight, which is a characteristic of the optical sensor used for diagnostic purposes.

[0056] Under typical conditions, the time of flight does not change significantly between measurements. Therefore, even when the control parameters of the optical sensor are changed, the obtained time of flight measurement should not change significantly between measurements. In this case, the predetermined characteristic of the optical sensor is that the time of flight measurement should remain substantially constant between measurements.

[0057] In an embodiment of the invention, the circuit connecting the photodetector and the integrator includes a mixer configured to mix a signal from the photodetector with a signal phase-dependent to the modulated optical signal, such that the mixed signal can be accumulated on the integrator.

[0058] Since the circuit connecting the photodetector and the integrator includes a mixer, the mixed signal (the signal from the photodetector and the signal phase-dependent with the modulated optical signal) will be accumulated on the integrator.

[0059] In a fifth aspect, embodiments of the present invention relate to a computer program product, if implemented on a processing unit, for performing a method for diagnosing an optical sensor according to embodiments of the present invention.

[0060] Specific and preferred aspects of the invention are set forth in the appended independent and dependent claims. Features from the dependent claims may be suitably combined with features of the independent claims and other dependent claims, and not merely as expressly set forth in the claims.

[0061] These and other aspects of the invention will be apparent from the embodiments described herein, and are illustrated with reference to these embodiments. Attached Figure Description

[0062] Figure 1 A flowchart of a method according to an embodiment of the present invention is shown.

[0063] Figure 2 A schematic diagram of an optical sensor according to an embodiment of the present invention is shown.

[0064] Figure 3 A schematic diagram of a camera according to an embodiment of the present invention is shown.

[0065] Figure 4 The voltage of the optical sensor changes over time.

[0066] Figure 5 The output voltage of the optical sensor is shown to change over time for different light levels.

[0067] Figure 6The diagram illustrates an embodiment of the invention, showing the relationship between integration time and output voltage, and wherein the output voltage is sampled for different integration times.

[0068] Figure 7 A schematic diagram illustrating a sequence of frames switching between two integration times according to an embodiment of the present invention is shown.

[0069] Figure 8 The diagram illustrates the use of offset and gain characteristics by showing the output voltage as a function of time for different offset values ​​and gains according to an embodiment of the invention.

[0070] Figure 9 A schematic diagram of an indirect time-of-flight system according to an embodiment of the present invention is shown.

[0071] Figure 10 A schematic diagram illustrating a sequence of phases corresponding to a depth frame of a time-of-flight system according to an embodiment of the present invention.

[0072] Any reference numerals in the claims should not be construed as limiting the scope.

[0073] In different accompanying drawings, the same reference numerals refer to the same or similar elements. Detailed Implementation

[0074] The invention will be described with reference to specific embodiments and particular drawings, but the invention is not limited thereto but is defined only by the claims. The described drawings are illustrative only and are not restrictive. In the drawings, some elements may be enlarged and not drawn to scale for illustrative purposes. Scale and relative scale do not correspond to an actual reduction in the implementation of the invention.

[0075] The terms "first," "second," etc., used in the specification and claims are used to distinguish between similar elements and are not necessarily used to describe a temporal, spatial, hierarchical, or any other order. It should be understood that the terms thus used are interchangeable where appropriate, and the embodiments of the invention described herein can be operated in a different order than that described or explained herein.

[0076] It should be noted that the term "comprising" as used in the claims should not be construed as limiting itself to the devices listed thereafter; it does not exclude other elements or steps. Therefore, the term should be interpreted as specifying the presence of the features, integers, steps, or components stated as mentioned, but does not exclude the presence or addition of one or more other features, integers, steps, or components, or groups thereof. Thus, the scope of the statement "device comprising devices A and B" should not be limited to a device consisting solely of components A and B. This means that for the purposes of this invention, the only relevant components of the device are A and B.

[0077] Throughout this specification, the reference to "an embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the phrase "in one embodiment" or "in an embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, in one or more embodiments, as will be obvious to those skilled in the art from this disclosure, particular features, structures, or characteristics can be combined in any suitable manner.

[0078] Similarly, it should be understood that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, drawing, or description for the purpose of simplification and to aid in understanding one or more of the various inventive aspects. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the appended claims, inventive aspects exist in fewer features than all the features of a single foregoing disclosed embodiment. Therefore, the claims appended following the detailed description are thus explicitly incorporated into this detailed description, wherein each claim itself represents a separate embodiment of the invention.

[0079] Furthermore, while some embodiments described herein include features that are included in other embodiments but not others, it will be understood by those skilled in the art that combinations of features from different embodiments are intended to fall within the scope of the invention and form different embodiments. For example, any embodiment of the claimed embodiments in the appended claims can be used in any combination.

[0080] Numerous specific details are set forth in the description provided herein. However, it should be understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0081] In a first aspect, embodiments of the present invention relate to a method 100 for diagnosing an optical sensor, the optical sensor including a photodetector and an integrator. A flowchart of such a method is shown below. Figure 1 As shown. An initial integrated signal is obtained at 120 while the optical sensor is exposed to incident light at 110, and one or more subsequent integrated signals are obtained at 140. Before obtaining the subsequent integrated signals, one or more control parameters of the optical sensor at 130 are changed.

[0082] One or more sensor characteristics of an optical sensor are obtained from an initial integrated signal and from one or more subsequent integrated signals. One or more control parameters of the optical sensor affect the integrated signal. This effect can be predetermined for one or more control parameters. The optical sensor can be diagnosed by comparing one or more predetermined characteristics with one or more obtained characteristics. The effect of the control parameters on the integrated signal is stored in predefined characteristics. These predefined characteristics are compared with the characteristics obtained when the same control parameters are modified.

[0083] In a second aspect, embodiments of the present invention relate to an optical sensor 200. Figure 2 The figure illustrates an exemplary embodiment of such an optical sensor according to an embodiment of the present invention. The optical sensor includes a photodetector 210, an integrator 220, and circuitry connecting the photodetector 210 and the integrator 220, such that a signal from the photodetector, or a processed version of the signal, can be accumulated on the integrator. The optical sensor 200 includes a controller 230. In embodiments of the invention, the controller may be a digital controller. For example, it may be an FPGA controller, or it may be a processor running a computer program thereon, configured to perform a method according to an embodiment of the present invention. The controller is configured to:

[0084] The initial integral signal is obtained at the initial frame.

[0085] Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames.

[0086] The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0087] The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor.

[0088] In embodiments of the invention, the integrated signal is obtained frame by frame. A frame includes at least one sequence of resetting the optical sensor and accumulating signals from the photodetector. The signal is accumulated on an integrator. The frame also includes reading the integrated signal. In embodiments of the invention, the accumulated signal can represent the amount of incident light. The signal is accumulated during the integration time. Reading the integrated signal can be accomplished, for example, by means of an analog-to-digital converter (ADC) that converts the voltage on the integrator into a digital signal that can be processed by a controller.

[0089] In a third aspect, embodiments of the present invention relate to a camera 300, which includes a plurality of pixels 305. Figure 3An example of this is illustrated in the diagram. Each pixel includes a photodetector 210, an integrator 220, and circuitry connecting the photodetector 210 and the integrator 220, such that a signal from the photodetector, or a processed version thereof, can be accumulated on the integrator. The camera 300 includes a controller 330. The controller 330 is configured to perform the following operations on each pixel:

[0090] The initial integral signal is obtained at the initial frame.

[0091] Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames.

[0092] The characteristics of the optical sensor are obtained from the acquired integrated signal.

[0093] The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor.

[0094] The control parameter that can be modified is the integration time. In such embodiments, the initial integration time (for obtaining the initial integrated signal) and subsequent integration times (for obtaining the subsequent integrated signal) are different. The difference between the different integrated signals obtained using different integration times is used to check for correct operation. The relationship between the output and the integration time is predictable and therefore can be predefined. This predefined characteristic is used to diagnose the optical sensor. Diagnosis may include checking the correct operation of the optical sensor and / or deriving the main performance characteristics of the optical sensor (e.g., pixels).

[0095] Figure 4 The middle figure illustrates an example of how the voltage of an optical sensor changes over time. This optical sensor could be, for example, a pixel sensor of an image sensor. It could also be part of an indirect time-of-flight sensor. This optical sensor measures the level of light incident on (multiple) pixels (the integration time) over a certain period of time.

[0096] exist Figure 4 In the example, the pixel voltage is reset to a known value at the start of the frame (time t1). Then, during the time interval (t2-t3), the incident light changes the pixel voltage by generating a photoinduced current. After the integration time, the output level is sampled and read. Thus, the first integrated signal is obtained. Typically, V... 重置 -V1 is output as a value. This sequence is repeated for each frame, so that one frame corresponds to a time period (t1-t4). The invention relates to repeating this cycle, but the cycle has different control parameters, such as gain and / or integration time (t3-t2).

[0097] In the example, for a given pixel and sensor design, the output voltage V1 at the end of the integration period depends on two variables. One variable is the incident light level. This will change Figure 4 The slope of the graph between t2 and t3. Another variable is the integration time. This is also... Figure 5 As shown in the figure, where V 输出 =(V 重置 -V1) is shown on the Y-axis. Different lines correspond to different light levels, with the line with the highest slope corresponding to the highest light level, and the horizontal line corresponding to no incident light. The length of the integration time (t2-t3) is shown on the X-axis.

[0098] from Figure 5 As can be seen from the figure, there is a linear relationship between the chosen integration time and the output voltage for any given light level.

[0099] In one embodiment of the invention, the predetermined characteristic is a linear, predictable relationship between the integration time and the output value. This predetermined characteristic is used to diagnose the optical sensor by comparing it with an obtained characteristic. Diagnosis may include, for example, determining the performance characteristics of the optical sensor and / or determining the correct operation of the sensor.

[0100] To achieve this, in some embodiments of the invention, the integration time is varied between frames. Figure 6 In the example, two values ​​for the integration time are used: T int1 and T int2 And T int1 >T int2 .

[0101] Sensor characteristics can be derived from the obtained voltage and known time settings. One sensor characteristic is the offset value. This offset value can be obtained for each individual pixel or sensor. It is the voltage measured in the absence of incident light. Another sensor characteristic is the pixel conversion gain. This conversion gain can also be obtained for each individual pixel or sensor.

[0102] These characteristics characterize the complete input / output processing chain of the readout electronics and thus allow for diagnostics of this complete processing chain of the optical sensor (i.e., from the interface of the incident light to the interface of the output value that can be measured).

[0103] In embodiments of the invention, predefined characteristics may include thresholds within which the acquired characteristics should fall. These can be used to determine the performance or correct behavior of the optical sensor. To diagnose pixels, preset thresholds can be used as pass / fail criteria on the acquired characteristics. Pass / fail states can then be provided to a controller (e.g., to an application processor) to allow for correct decisions regarding the proper operation of the sensor.

[0104] Additionally, the acquired characteristics, or their processed version, can be output to the controller to allow external tracking of performance and status. In this case, the controller makes (multiple) pass / fail decisions.

[0105] In its most basic operation, the method according to an embodiment of the invention involves switching between two integration times. This is in Figure 7 The diagram illustrates this. Each frame in the diagram includes a reset step, an integration step, and a read step. The output value (i.e., the integrated signal) obtained in each frame can be output and used in the controller without sacrificing the system's frame rate. In embodiments of the invention, a frame can be used as a reference (e.g., the first frame), and the output value of the second frame can be adjusted based on the ratio of the integration time by using a predetermined characteristic that expresses the expected relationship between the integration time and the output value. Therefore, characteristics independent of the integration time can be obtained.

[0106] For example, adjustments to camera pixels can be made as follows: When a pixel has a response of 100 lsb for an integration time of 1000 μs in the first frame, a result of 120 lsb for an integration time of 1200 μs in the second frame is expected. Correction for the result in the second frame will be performed by multiplying by T. int1 / T int2 This is done to obtain a 100 lsb corrected pixel output for the second frame.

[0107] For each pixel (or for a group of pixels), the V of the first frame can be evaluated and tracked. out and frame 2's V out The ratio between the two. This value corresponds to the gain for a given incident light intensity. This gain can be used as the obtained characteristic and compared with a predetermined characteristic. When a pixel (or a group of pixels) no longer has results within the expected boundaries (for the gain), they can be marked as faulty.

[0108] In embodiments of the invention, filtering can be applied to the obtained characteristics before comparing them with predetermined characteristics. Assuming, for example, the values ​​used in the above example, the corrected pixel values ​​of the second frame can be compared with the pixel values ​​of the first frame. If the result deviates by more than a preset threshold, it can be marked as out of bounds. This decision can be made based solely on two frames, as in this example, or long-term trends can be filtered out.

[0109] In an embodiment of the present invention, the integral signal V 输出 (t) can be modeled as g*t+V 偏移 Where g is the gain (for a fixed light intensity), and V 偏移 This is a characteristic of optical sensors. By changing the T... int1 To T int2 During the integration time, two points can be obtained (T). int1 V 输出1 ) and (T int2 V 输出2 And it can derive a linear relationship V. 输出 (t) = Gain * t + V 偏置 This allows for the determination of pixel or sensor gain and inherent offset, and the comparison of these gains and inherent offsets with predetermined values ​​for the sensor for diagnostic purposes.

[0110] One way to accomplish this is by analyzing the output images captured twice and calculating the gain and offset for each pixel of the camera. By averaging this value over time, biases can be detected. Possible biases include pixels with decreased gain (non-responsive pixels) or pixels with an offset value (e.g., "white pixels"). This is in... Figure 8 As shown in the figure, Figure 8 The figure shows the output voltage V of optical sensors with different operating characteristics. out The changes over time are shown. An ideal curve is presented, corresponding to the nominal operating characteristics of the optical sensor. Two additional curves with different gains (gain_1 and gain_2) are shown. When the gain drops below gain_1, an error can be identified by the controller. In this case, the pixel is a non-responsive pixel. Two additional curves with different offsets (offset_1 and offset_2) are shown. When the offset is higher than offset_2 or less than offset_1, an error can be generated by the controller. By averaging over time, individual anomalies caused by scene changes can be filtered out from the integrated signal.

[0111] Another approach to data analysis involves acquiring the scene using an integration time of 1 to obtain a first integrated signal, followed by acquisition using a second integration time to obtain a second integrated signal. Based on the data captured in the first exposure and predetermined characteristics, the expected pixel response for the second exposure can be predicted, and an error signal can be calculated based on the difference between the actual measurement and the predicted value for each pixel (i.e., by comparing the obtained characteristics with predetermined characteristics). The error signal can also be averaged over time to reject individual outliers caused by scene changes.

[0112] By detecting changes in pixel behavior using the methods described above, suspicious pixels can be identified. This information can then be relayed to algorithms that process the data. These algorithms can then choose to ignore data from these pixels or perform further analysis to examine their state.

[0113] Furthermore, the number of pixels exhibiting deviation over time can be monitored and thus used as an indicator of the image sensor's operational status. An increase in deviation pixels can serve as a warning related to sensor performance degradation. This can be used to trigger diagnostics.

[0114] In the case of a 3D sensor, the distance of each exposure in two exposures can be calculated separately. An error signal can be generated by calculating the depth difference for each pixel between the two exposures. Under normal operating conditions, the calculated depth should not depend on the integration time or pixel gain. If a depth difference is detected between two exposures, this indicates a faulty circuit.

[0115] In embodiments of the invention, the integrated signal is obtained frame by frame. A frame thus includes at least one sequence of resetting the optical sensor, accumulating the signal from the photodetector, and reading the integrated signal. If this sequence is performed multiple times, each sequence is called a subframe. In this case, several subframes exist within a macroframe (such as in multi-phase TOF).

[0116] A change in at least one control parameter (e.g., a change in integration time) can be applied at the subframe level or the macroframe level. When the control parameter is changed, the optical sensor can be diagnosed by checking whether its primary output remains unchanged. The primary output is the main output of the optical sensor design. For example, it could be the depth of a 3D camera. For a 3D camera, this depth should not change when the integration time is changed.

[0117] The control parameters can be fixed or can be varied between subframes. As discussed earlier, the control parameters can be integration times. This method can use a fixed set of integration times. These integration times can be stored in the optical system's memory, or they can be retrieved externally. The integration times can be varied randomly between frames or subframes. Alternatively, the integration times can be varied repeatedly between frames or subframes.

[0118] Another control parameter that can be changed between frames or subframes is the gain of the optical sensor. Changing this gain results in a change in the conversion slope of the optical sensor. The characteristics obtained as a result of the gain change can be compared with the predetermined characteristics of the gain change to diagnose the operation of the optical sensor. In this case, the optical sensor (which may be, for example, a camera pixel) has a conversion gain that can be changed by the user or autonomously by the application.

[0119] Figure 2 An example of such a pixel is illustrated. It shows an optical sensor 200, which includes a photodetector 210, an integrator 220, and circuitry connecting the photodetector 210 and the integrator 220. The optical sensor is configured such that a signal from the photodetector can be accumulated on the integrator. The optical sensor 200 includes a controller 230. This controller is adapted to perform methods according to embodiments of the invention disclosed in the preceding paragraphs.

[0120] exist Figure 2 In this simplified schematic, for clarity, the readout circuit and transmission gate are not shown. The signal referred to as "gain" is intended to serve as digital control of the conversion gain. When the "gain" is low, only C1 is used as a capacitor to integrate the photocurrent from the photodiode.

[0121] When the "gain" is set to high, C2 is connected in parallel with C1, and therefore the photocurrent will be distributed between the two capacitors. The change in conversion gain depends on the ratio of C1 to C2.

[0122] The reset transistor is only used during the reset phase and is inactive during the integration or readout phase.

[0123] In this example, the gain of the optical sensor is modifiable by changing the value of the capacitor that needs to be charged. However, the invention is not limited to this. Other ways of changing the gain are also possible. The gain can also be achieved, for example, using a variable sense amplifier gain.

[0124] In a fourth aspect, embodiments of the present invention relate to an indirect time-of-flight system 400. For an indirect time-of-flight (iToF) system, pixel diagnosis should be performed without affecting the frame rate of the output distance information. Figure 9 The schematic diagram illustrates an exemplary embodiment of such an iTOF system 400 according to an embodiment of the present invention. Such a system may include a transmitter for emitting an amplitude-modulated optical signal into a scene. The modulated signal may have a certain modulation frequency f. MOD The pulse signal. According to an embodiment of the invention, the iTOF system includes an optical sensor 200 or a camera. The optical sensor includes pixels 305, or the camera includes an array of pixels 305, wherein each pixel includes a photodetector 210 (e.g., a photodiode), an integrator 220, and circuitry connecting the photodetector 210 and the integrator 220. In this example, pixel 305 includes a photonic mixer device, which includes an integrator 220 and circuitry (mixer 406) connecting the photodetector and the integrator 220. The integrator includes two storage elements into which charge from the photodetector is stored according to a control signal to the mixer 406.

[0125] The optical sensor or camera further includes a controller 230. The controller 230 includes a reference timing control block 416, a phase difference calculation unit 410, and a distance calculation unit 424. Additionally, Figure 9 A phase shifter 412 is shown having an input 414 operatively connected to the output 422 of a timing control block. The phase shifter 412 is further operatively connected to a photodetector 210 to provide control signals to a mixer 406 within each pixel.

[0126] Figure 9 The light emitting unit is not shown, but in order to generate the required modulated illumination, the timing control block 416 output 422 is operatively connected to the light emitting unit to generate a modulation signal for driving the light source. The light source can be, for example, a VCSEL or any other suitable light source. The emitted light can pass through an optical system. This optical system can be shared with the photodetector 210 or it can have a separate optical system.

[0127] In operation, the light emitting unit illuminates the scene using an amplitude-modulated signal. One or more pixels 305 detect reflected light from objects in the scene, where in each pixel, a photodetector 210 generates a current proportional to the detected light. This current is then read into an integrator 220 using a mixer 406. The mixer 406 is configured to receive a pulse signal having the same modulation frequency f as the light emitting signal. MOD The current generated by photodetector 210 is modulated using a pulse signal. Therefore, pixel 305 may include an interface for receiving signals from phase shifter 412.

[0128] To detect the phase difference between the returned light and the emitted light, a phase shifter 412 is used to shift the phase of the mixing signal to obtain m phase-shifted mixing signals (signals Θ0, Θ1, ..., Θ...). m-1 Thus, m integral values ​​P(P0, P1, ..., P2) are obtained. m-1 (where m is a natural number greater than 1, typically between 2 and 8) This phase shift can be controlled by controller 230. The iTOF system 400 also includes a calculation unit 410 configured to obtain the phase difference between the returned and emitted light. This calculation unit may, for example, be a Discrete Fourier Transform (DFT) block 410 for output harmonic I and Q values ​​425, where the phase difference can be calculated from the first harmonic value. For this purpose, timing information from a timing control block may be required, as indicated by arrow 420. Based on the phase difference between the returned and emitted light, the distance to the object can be solved using block 424.

[0129] Exemplary embodiments of the iToF system according to embodiments of the present invention will be further described below. In the example, m = 4, that is, phase shifts of 0°, 90°, 180°, and 270° are used for the mixing signal. In this example, the mixer 406 is controlled by a pulse signal with a 50% duty cycle (other duty cycles are also possible). Therefore, the integration time is limited by the number of pulses per phase.

[0130] Figure 10 The diagram illustrates four phases of an exemplary iToF system, which enable distance measurement. The sequence of these phases is referred to as a depth frame.

[0131] In embodiments of the invention, the control parameters (e.g., integration time) of the optical sensor can be varied for different depth frames while remaining constant within the depth frame. See also... Figure 10 P0, P90, P180, and P270 will have a certain length T1 for the first depth frame, but will become a different length T2 for subsequent frames (other sensor parameters can be changed).

[0132] In typical cases, when the flight time does not change significantly between depth frames, the depths obtained from two depth frames should be substantially equal, regardless of the integration time used. Therefore, the difference between the depths obtained for different depth frames can be used as a first diagnostic value. Thus, the controller 230 is configured to change at least one control parameter of the optical sensor and to obtain the obtained depth from the integrated signal.

[0133] Additionally, the controller can be configured to compare the phase data itself between depth frames for diagnostic purposes: the P0 of the first depth frame with integration time T1 can be compared with the P0 data from the second frame with integration time T2. As explained in the previous paragraph, the result for each phase should be proportional to the integration time.

Claims

1. A method (100) for diagnosing an optical sensor, said optical sensor comprising a photodetector and an integrator, The method includes: The photodetector is exposed to incident light at (110). The initial integral signal (120) is obtained at the initial frame. Perform the following steps at least once: change at least one control parameter of the optical sensor (130), expose the photodetector to incident light (110), and obtain one or more subsequent integrated signals (140) in subsequent frames. Specifically, the integrated signal is obtained frame by frame, and each frame includes resetting the optical sensor, accumulating signals from the circuit connecting the photodetector and the integrator within the integration time, and reading at least one sequence of the integrated signal. The characteristics of the optical sensor (150) are obtained from the integrated signal. The obtained characteristics are compared with predetermined characteristics of the optical sensor (160), wherein the predetermined characteristics are defined according to at least one control parameter, in order to diagnose the optical sensor and detect whether there is a fault in the optical sensor.

2. The method (100) according to claim 1, characterized in that, At least one control parameter is the integral time.

3. The method (100) according to claim 2, characterized in that, The integration time has a ratio ranging from 50% to 150%.

4. The method (100) according to claim 1, characterized in that, At least one control parameter is the gain of the optical sensor.

5. The method (100) according to claim 1, characterized in that, The integral signal V 输出 (t) is modeled as g*t+V 偏移 , where g and V 偏移 The characteristics of the optical sensor are obtained from the acquired integrated signal.

6. The method (100) according to claim 1, characterized in that, The characteristics of the optical sensor are obtained by dividing the subsequent integrated signal.

7. An optical sensor (200) comprising a photodetector (210), an integrator (220), and circuitry connecting the photodetector (210) and the integrator (220), such that a signal from the photodetector or a processed version thereof can be accumulated on the integrator, the optical sensor (200) comprising a controller (230) configured to: The initial integral signal is obtained at the initial frame. Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames. in, The integrated signal is obtained frame by frame, and each frame includes resetting the optical sensor, accumulating the signal from the circuit connecting the photodetector and the integrator within the integration time, and reading at least one sequence of the integrated signal. The characteristics of the optical sensor are obtained from the acquired integrated signal. The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor and detect whether there is a fault in the optical sensor.

8. The optical sensor (200) according to claim 7, characterized in that, The integrator (220) includes a plurality of parallel capacitors, wherein the controller is configured to connect one or more capacitors of the integrator or disconnect one or more capacitors of the integrator.

9. The optical sensor (200) of claim 8, wherein the optical sensor includes a gain transistor (240) configured to connect or disconnect a capacitor from one of the plurality of parallel capacitors from the integrator.

10. The optical sensor (200) of claim 7, wherein the optical sensor (200) includes a reset transistor (250) configured to reset the integrator voltage.

11. A camera (300) comprising a plurality of pixels (305), each pixel comprising a photodetector (210), an integrator (220), and circuitry connecting the photodetector (210) and the integrator (220), the camera (300) comprising a controller (330), the controller (330) being configured for each pixel to: The initial integral signal is obtained at the initial frame. Perform the following steps at least once: change at least one control parameter of the optical sensor and obtain one or more subsequent integration signals in subsequent frames. in, The integrated signal is obtained frame by frame, and each frame includes resetting the optical sensor, accumulating the signal from the circuit connecting the photodetector and the integrator within the integration time, and reading at least one sequence of the integrated signal. The characteristics of the optical sensor are obtained from the acquired integrated signal. The obtained characteristics are compared with predetermined characteristics of the optical sensor, which are defined according to at least one control parameter, to diagnose the optical sensor and detect whether there is a fault in the optical sensor.

12. The camera (300) according to claim 11, characterized in that, The camera is a 3D camera, and the controller (310) is configured to obtain the pixel depth of each pixel (305) as a characteristic of the pixel, and the obtained characteristic is compared with the predetermined characteristic to verify that the pixel depth does not change significantly for different measurements.

13. An indirect time-of-flight system (400) for measuring the time of flight of a modulated light signal reflected from an object, the indirect time-of-flight system (400) comprising an optical sensor (200) according to claim 7 or a camera (300) according to claim 12 for receiving the reflected light signal, wherein, The time-of-flight system (400) is configured to obtain a measurement of the time of flight, and wherein the measurement is for diagnosing the characteristics of the optical sensor.

14. The indirect time-of-flight system (400) according to claim 13, characterized in that, The circuit connecting the photodetector (210) and the integrator (220) includes a mixer (406) configured to mix a signal from the photodetector with a signal phase-dependent on the modulated optical signal, such that the mixed signal can be accumulated on the integrator.

15. A computer program product, if implemented on a processing unit, for performing the method for diagnosing an optical sensor according to claim 1.

Images