Light detection device, correction method, and program
The photodetection device addresses interference in event detection by identifying and correcting for signal overlap, enhancing accuracy in event signal generation.
Patent Information
- Application Number
- JP2023078373
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-21
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Conventional photodetection devices face interference issues due to overlapping generation of gray scale and event signals, leading to potential errors in event detection accuracy.
A photodetection device with a pixel array section and correction method that identifies and corrects for interference by specifying the intensity of overlap between gradation data and event detection, using a correction section to enhance event data accuracy.
The solution effectively reduces interference-related errors, improving the accuracy of event signals by correcting for overlapping signal generations.
Smart Images

Figure 2024162648000001_ABST
Abstract
Description
[Technical field]
[0001] The present technology relates to a light detection device, a correction method, and a program, and, for example, to a light detection device, a correction method, and a program with improved accuracy of event detection. [Background technology]
[0002] For example, there is a sensor device that detects an event signal that indicates a change in the amount of received light for each pixel, such as an event-based vision sensor (EVS). Some sensor devices of this type are equipped with a pixel for detecting an event signal and a pixel for detecting a grayscale signal so that a grayscale signal (a signal indicating the intensity of the amount of received light) of a pixel in which an event is detected can be obtained (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-129265 Summary of the Invention [Problem to be solved by the invention]
[0004] In the above-mentioned conventional technology, the generation of the gray scale signal and the processing for detecting the event overlap in time, which may cause interference due to a control signal or the like. If interference occurs, there is a possibility that an error may occur in the event signal. It is desired to reduce the influence of interference and improve the accuracy of the event signal.
[0005] The present technology has been made in consideration of such circumstances, and makes it possible to improve the accuracy of an event signal. [Means for solving the problem]
[0006] An optical detection device according to one aspect of the present technology is an optical detection device including: an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event; a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light; an interference intensity determination unit that determines an intensity of interference that occurs due to an overlap in a period of generation of the gradation data and detection of the event; and a correction unit that corrects the event data according to the intensity of interference determined by the interference intensity determination unit.
[0007] A correction method according to one aspect of the present technology is a correction method in which a photodetection device includes an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data, which is data according to the luminance of the incident light, and includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit identifying an intensity of interference that occurs due to an overlap in a period of generation of the gradation data and a period of detection of the event, and correcting the event data according to the identified intensity of interference.
[0008] A program according to one aspect of the present technology is a program for causing a computer to control a photodetector having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data, which is data according to the luminance of the incident light, to execute a process of identifying an intensity of interference that occurs due to an overlap between a period of generation of the gradation data and a period of detection of the event, and correcting the event data according to the identified intensity of interference.
[0009] In the light detection device, the correction method, and the program according to one aspect of the present technology, a pixel array unit is provided in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a grayscale data generation unit that generates grayscale data corresponding to the luminance of the incident light. The intensity of interference caused by an overlap between the generation of the grayscale data and the detection of the event is identified, and the event data is corrected according to the identified intensity of interference.
[0010] The light detection device may be an independent device or an internal block constituting a single device.
[0011] The program can be provided by transmitting it via a transmission medium or by recording it on a recording medium. [Brief description of the drawings]
[0012] [Figure 1] 1 is a diagram illustrating a configuration example of a light detection device according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a pixel configuration. [Diagram 3] FIG. 4 is a diagram illustrating an example of the configuration of a gray-scale signal generating unit. [Figure 4] FIG. 4 is a diagram illustrating an example of generation of a gray scale signal. [Diagram 5] FIG. 4 illustrates an example of the configuration of an event signal generating unit; [Figure 6] 4 is a circuit diagram illustrating an example of the configuration of an event signal generating unit. FIG. [Figure 7] 4 is a circuit diagram illustrating an example of the configuration of an event signal generating unit. FIG. [Figure 8] 5A and 5B are diagrams illustrating an example of generation of a grayscale signal and an event signal. [Figure 9] 13 is a diagram illustrating an example of the configuration of a predicted overlapping row detection unit; FIG. [Figure 10] FIG. 11 is a diagram illustrating an example of event data. [Figure 11]FIG. 11 illustrates an example of a procedure for duplicate line detection processing; [Figure 12] FIG. 4 illustrates an example of the configuration of a correction unit. [Figure 13] FIG. 13 is a diagram illustrating another example of the configuration of the correction unit. [Figure 14] FIG. 13 is a diagram illustrating another example of the configuration of the correction unit. [Figure 15] FIG. 13 is a diagram illustrating another example of the configuration of the correction unit. [Figure 16] FIG. 13 is a diagram for explaining event data before and after correction. [Figure 17] FIG. 13 is a diagram for explaining event data before and after correction. [Figure 18] 13 is a flowchart illustrating a process related to correction of event data. [Figure 19] 11 is a flowchart for explaining a first interference intensity identifying process. [Figure 20] FIG. 13 is a diagram for explaining an HOPB area. [Figure 21] 11 is a diagram for explaining conditions related to identification of interference intensity. FIG. [Figure 22] 13 is a flowchart for explaining a second interference intensity identifying process. [Diagram 23] 11 is a diagram for explaining conditions related to identification of interference intensity. FIG. [Figure 24] 13 is a flowchart for explaining a third interference intensity identifying process. [Diagram 25] FIG. 13 is a diagram showing an example of an interference intensity LUT 621. [Figure 26] 13 is a flowchart illustrating a first event data correction process. [Figure 27] FIG. 13 is a diagram for explaining removal of a noise event. [Figure 28] FIG. 11 is a diagram for explaining a first correction rule. [Figure 29] FIG. 11 is a diagram for explaining a second correction rule. [Diagram 30]FIG. 11 is a diagram for explaining a third correction rule. [Diagram 31] 13 is a flowchart illustrating a second event data correction process. [Diagram 32] FIG. 13 is a diagram showing the relationship between interference intensity and correction factor. [Diagram 33] FIG. 2 illustrates an example of the configuration of a PC. [Diagram 34] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Diagram 35] 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be described.
[0014] <Configuration of the photodetector> FIG. 1 is a diagram showing a configuration according to one embodiment of a photodetector to which the present technology is applied.
[0015] The photodetection device 1 includes a pixel array unit 10, an access control circuit 20, an event signal output circuit 30, a gradation signal output circuit 40, a timing control unit 50, an event signal processing unit 60, a gradation signal processing unit 70, and an overlapping predicted row detection unit 80. The photodetection device 1 further includes a timestamp generation unit 90 and an image processing unit 2.
[0016] The pixel array unit 10 is configured by arranging a plurality of pixels 100 in a two-dimensional matrix. Here, the pixels 100 generate grayscale signals that are signals corresponding to the luminance of incident light. The pixels 100 also detect the above-mentioned change in the luminance of the incident light in the same direction as an event, and further generate an event signal that is a signal based on the detected event. The pixels 100 include a photoelectric conversion unit that performs photoelectric conversion of the incident light, and generate the above-mentioned grayscale signals and event signals based on the result of the photoelectric conversion.
[0017] For example, a photodiode can be used for the photoelectric conversion unit. Signal lines 11, 12, and 21 are wired to each pixel 100. The pixel 100 generates a grayscale signal and an event signal under the control of a control signal transmitted by the signal line 21. The pixel 100 outputs the generated grayscale signal via the signal line 12, and outputs the generated event signal via the signal line 11. The signal line 21 is arranged for each row in a two-dimensional matrix shape, and is wired in common to the multiple pixels 100 arranged in one row. The signal lines 11 and 12 are arranged for each column in a two-dimensional matrix shape, and are wired in common to the multiple pixels 100 arranged in one column.
[0018] 2, the pixel 100 includes a grayscale signal generating unit 110 that generates a grayscale signal and an event signal generating unit 120 that generates an event signal. In the pixel array unit 10, the pixels 100 arranged in the same row simultaneously generate grayscale signals. The generation of the grayscale signals is performed sequentially for each row with a shift in timing.
[0019] The access control circuit 20 outputs a control signal for the above-mentioned pixel 100. The access control circuit 20 in the figure outputs a control signal for each row of the two-dimensional matrix of the pixel array section 10 via a signal line 12.
[0020] The event signal output circuit 30 processes the event signal for each pixel 100 output from the pixel array unit 10, and outputs the processed event signal. The processing by the event signal output circuit 30 corresponds to, for example, a process of converting an analog event signal output from the pixel 100 into a digital data signal.
[0021] The grayscale signal output circuit 40 processes the grayscale signals generated by the pixels 100 and outputs the processed grayscale signals. The grayscale signal output circuit 40 in the figure simultaneously processes grayscale signals from a plurality of pixels 100 arranged in one row of the pixel array unit 10. The processing by the grayscale signal output circuit 40 corresponds to, for example, a process of converting an analog grayscale signal output from the pixel 100 into a digital signal.
[0022] The event signal processing unit 60 processes the event signal from the event signal output circuit 30. For example, the event signal output circuit 30 performs a process of converting the event signal into data in a predetermined format. The event signal output circuit 30 outputs the processed event signal as event data to an external device.
[0023] The gradation signal processing unit 70 processes the gradation signal from the gradation signal output circuit 40. For example, the gradation signal processing unit 70 performs a process of converting the gradation signal into data of a predetermined format. The gradation signal processing unit 70 outputs the processed gradation signal as gradation data.
[0024] The timing control unit 50 controls the pixels 100, etc. The timing control unit 50 controls the pixels 100, etc. by generating and outputting control signals for the pixels 100, etc. This control corresponds to control for making the gradation signal generation unit 110 generate a gradation signal, control for making the event signal generation unit 120 detect an event, and control for making the event signal generation unit 120 generate an event signal based on the detected event.
[0025] Control signals used for these controls are output to the pixels 100 for each row of the pixel array unit 10 via the access control circuit 20. The timing control unit 50 also generates control signals for the event signal processing unit 60 and the gradation signal processing unit 70. The generated control signals are output to the access control circuit 20, the event signal processing unit 60, the gradation signal processing unit 70, and the overlapping predicted row detection unit 80. The timing control unit 50 and the access control circuit 20 are an example of a row control unit.
[0026] The time stamp generating unit 90 generates a time stamp and supplies it to the event signal processing unit 60 .
[0027] The overlapping predicted row detection unit 80 detects overlapping predicted rows, which are rows where the period of gray scale signal generation and the period of event detection are predicted to overlap. As described above, the pixel 100 includes the gray scale signal generation unit 110 and the event signal generation unit 120, and can generate gray scale signals and event signals separately. For this reason, the period of gray scale signal generation and the period of event detection may overlap. In this case, interference occurs in the gray scale signal generation and the event detection.
[0028] This occurs because the grayscale signal generating unit 110 and the event signal generating unit 120 are coupled by stray capacitance or the like, and a change in the control signal of one of the grayscale signal generating unit 110 and the event signal generating unit 120 affects the signal level of the other. When this interference occurs, errors occur in the grayscale signal and the event signal. Therefore, the above-mentioned overlapping predicted rows are detected and used to reduce the influence of the interference. In the figure, the overlapping predicted row detection unit 80 generates an interference occurrence flag indicating the occurrence of interference based on the detected overlapping predicted rows, and outputs it to the event signal processing unit 60.
[0029] The image processing unit 2 processes grayscale data, which is grayscale signal data. It is also possible to adopt a configuration including an event data processing unit that processes event data, which is event signal data.
[0030] <Pixel configuration> 2 is a diagram showing an example of the configuration of a pixel. The figure is a block diagram showing an example of the configuration of a pixel 100. The pixel 100 includes a grayscale signal generating unit 110 and an event signal generating unit 120. The grayscale signal generating unit 110 generates a grayscale signal based on a control signal supplied from the access control circuit 20 via a signal line 21. The generated grayscale signal is transmitted to the grayscale signal output circuit 40 via a signal line 12. The event signal generating unit 120 generates an event signal based on a control signal supplied from the access control circuit 20 via the signal line 21. The generated event signal is transmitted to the event signal output circuit 30 via a signal line 11.
[0031] <Configuration of the Gradation Signal Generator> 3 is a diagram showing a configuration example of the grayscale signal generating unit. The diagram shows a circuit showing a configuration example of the grayscale signal generating unit 110. The grayscale signal generating unit 110 includes a photoelectric conversion unit 201, a charge holding unit 203, and MOS transistors 211 to 214. The MOS transistors 211 to 214 can be n-channel MOS transistors. The signal line 21 connected to the pixel 100 includes a signal line TRG, a signal line RST, and a signal line SEL. A power supply line Vdd that supplies power is wired to the pixel 100.
[0032] The anode of the photoelectric conversion unit 201 is grounded, and the cathode is connected to the source of the MOS transistor 211. The drain of the MOS transistor 211 is connected to the source of the MOS transistor 212, the gate of the MOS transistor 213, and one end of the charge holding unit 203. The other end of the charge holding unit 203 is grounded. The drain of the MOS transistor 213 is connected to the power supply line Vdd, and the source is connected to the drain of the MOS transistor 214. The source of the MOS transistor 214 is connected to the signal line 12. The signal lines TRG, RST, and SEL are connected to the gates of the MOS transistors 211, 212, and 214, respectively.
[0033] The photoelectric conversion unit 201 is an element that performs photoelectric conversion of incident light. The photoelectric conversion unit 201 generates and holds electric charges through photoelectric conversion.
[0034] The MOS transistor 211 transfers the charge held in the photoelectric conversion unit 201 to the charge holding unit 203. The MOS transistor 211 is controlled by a control signal transmitted by a signal line TRG.
[0035] The charge holding portion 203 is an element that holds electric charges. The charge holding portion 203 can be configured by a semiconductor region formed on a semiconductor substrate.
[0036] The MOS transistor 212 resets the charge holding unit 203. The MOS transistor 212 is controlled by a control signal transmitted through a signal line RST.
[0037] The MOS transistor 213 is an element that generates a grayscale signal according to the charge held in the charge holding unit 203. The generated grayscale signal is output to a source terminal.
[0038] The MOS transistor 214 is an element that outputs the grayscale signal generated by the MOS transistor 213 to the signal line 12. The MOS transistor 214 is controlled by a control signal transmitted through a signal line SEL.
[0039] <Gradation signal generation> FIG. 4 is a diagram showing an example of generation of a grayscale signal. This figure is a timing diagram showing an example of generation of a grayscale signal in the grayscale signal generating unit 110. In this figure, "SEL" represents a selection signal transmitted by a signal line SEL. "RST" represents a reset signal transmitted by a signal line RST. "TRG" represents a transfer signal transmitted by a signal line TRG. The part of the binarized waveform of these control signals with a value of "1" represents an ON signal, which is a signal that makes a MOS transistor conductive. The dashed line in this figure represents a level of 0V. As shown in this figure, the grayscale signal is generated by a shutter 501, an exposure 502, and a readout 503. In the initial state, the selection signal, the reset signal, and the transfer signal have values of "0", "1", and "0", respectively.
[0040] The shutter 501 is in a period corresponding to an electronic shutter, during which the reset signal and the transfer signal have a value of "1", and the MOS transistors 211 and 212 are conductive. This causes the charges in the photoelectric conversion unit 201 and the charge holding unit 203 to be discharged and reset.
[0041] Exposure 502 is a period in which the transfer signal has a value of “0” and charges generated by photoelectric conversion in the photoelectric conversion unit 201 are accumulated in the photoelectric conversion unit 201 .
[0042] Readout 503 is a period during which the selection signal has a value of "1", the reset signal has a value of "0", and a readout is performed in which the grayscale signal generated by the MOS transistor 111 is output to the signal line 12. During this period, the transfer signal has a value of "1", and the charge in the photoelectric conversion unit 201 is transferred to the charge holding unit 203. The MOS transistor 111 generates a grayscale signal according to the charge in the charge holding unit 203, and outputs it to the signal line 12.
[0043] In this way, a grayscale signal is generated by the three periods of shutter 501, exposure 502, and readout 503, and is output from grayscale signal generator 110. As described above, grayscale signals are generated for each row. At this time, shutter 501, exposure 502, and readout 503 are applied sequentially while shifting the timing for each row. This will be described later with reference to FIG. 9.
[0044] <Event signal generation section configuration> 5 is a diagram showing an example of the configuration of the event signal generation unit. The figure is a block diagram showing an example of the configuration of the event signal generation unit 120. The event signal generation unit 120 in the figure includes a photoelectric conversion unit 130, a current-voltage conversion circuit 140, a differentiation circuit 150, a luminance change detection unit 160, and an output unit 170.
[0045] The photoelectric conversion section 130 performs photoelectric conversion of incident light, similar to the photoelectric conversion section 201. The photoelectric conversion section 130 can be configured by a photodiode.
[0046] The current-voltage conversion circuit 140 converts the photocurrent from the photoelectric conversion unit 130 into a voltage signal. During this conversion, the current-voltage conversion circuit 140 performs logarithmic compression of the voltage signal. The converted voltage signal is output to the differentiation circuit 150. The configuration of the current-voltage conversion circuit 140 will be described in detail later.
[0047] The differentiation circuit 150 extracts the change in the voltage signal output from the current-voltage conversion circuit 140 and accumulates the extracted change to generate a signal according to the amount of change in the voltage signal. This signal corresponds to a signal according to the change in luminance of the incident light. This signal is called an optical signal. The differentiation circuit 150 outputs the generated optical signal to the luminance change detection unit 160 via the signal line 121. The differentiation circuit 150 also receives an input of a control signal from the access control circuit 20. This control signal is a signal that resets the circuit that detects the amount of change in the voltage signal. The configuration of the differentiation circuit 150 will be described in detail later.
[0048] The luminance change detection unit 160 detects a luminance change of the incident light. The luminance change detection unit 160 in the figure detects a change in the optical signal output from the differentiation circuit 150 based on a threshold value. That is, when the change in the optical signal exceeds the threshold value, the change in the optical signal is detected as an event. Here, an event in the direction in which the optical signal increases is called an on event, and an event in the direction in which the optical signal decreases is called an off event. The luminance change detection unit 160 detects an on event and an off event using the voltages of the on event detection signal and the off event detection signal supplied from the access control circuit 20 as threshold values. The detection result is output to the output unit 170. The configuration of the luminance change detection unit 160 will be described in detail later.
[0049] The output section 170 outputs the ON event and the OFF event detected by the luminance change detection section 160 based on a control signal from the access control circuit 20 as an event signal.
[0050] <Circuit configuration of event signal generation section> Fig. 6 and Fig. 7 are circuit diagrams showing examples of the configuration of the event signal generating section. Fig. 6 is a circuit diagram showing examples of the configuration of the current-voltage conversion circuit 140 and the differentiation circuit 150. Note that the figure also shows a photoelectric conversion section 202. Fig. 7 is a circuit diagram showing examples of the configuration of the luminance change detection section 160 and the output section 170.
[0051] The current-voltage conversion circuit 140 in the figure includes MOS transistors 215 to 217. In the figure, Vdd represents a power supply line Vdd that supplies power. Vb1 represents a signal line Vb1 that supplies a bias voltage. The MOS transistors 215 and 217 can be n-channel MOS transistors. The MOS transistor 216 can be a p-channel MOS transistor.
[0052] The anode of the photoelectric conversion unit 202 is grounded, and the cathode is connected to the source of the MOS transistor 215 and the gate of the MOS transistor 217. The sources of the MOS transistor 215 and the MOS transistor 216 are connected to the power supply line Vdd, and the gate of the MOS transistor 216 is connected to a signal line Vb1. The source of the MOS transistor 217 is grounded, and the drain is connected to the gate of the MOS transistor 215, the drain of the MOS transistor 216, and an output signal line of the current-voltage conversion circuit 140. One end of the capacitor of the differentiation circuit 150 is connected to this output signal line.
[0053] The MOS transistor 215 is a MOS transistor that supplies a current to the photoelectric conversion unit 202. A sink current (photocurrent) corresponding to incident light flows through the photoelectric conversion unit 202. The MOS transistor 215 supplies this sink current. At this time, the gate of the MOS transistor 215 is driven by the output voltage of a MOS transistor 217 described later, and outputs a source current equal to the sink current of the photoelectric conversion unit 202. Since the gate-source voltage Vgs of the MOS transistor becomes a voltage corresponding to the source current, the source voltage of the MOS transistor 215 becomes a voltage corresponding to the current of the photoelectric conversion unit 202. As a result, the photocurrent of the photoelectric conversion unit 202 is converted into a voltage signal.
[0054] The MOS transistor 217 is a MOS transistor that amplifies the source voltage of the MOS transistor 215. The MOS transistor 216 constitutes a constant current load of the MOS transistor 217. An amplified voltage signal is output to the drain of the MOS transistor 217. This voltage signal is output to the differentiation circuit 150 and is also fed back to the gate of the MOS transistor 215. When the Vgs of the MOS transistor 215 is equal to or lower than the threshold voltage, the source current changes exponentially with respect to the change in Vgs. Therefore, the output voltage of the MOS transistor 217 that is fed back to the gate of the MOS transistor 215 becomes a voltage signal that is logarithmically compressed from the photocurrent of the photoelectric conversion unit 202, which is equal to the source current of the MOS transistor 215.
[0055] <Differential circuit configuration> The differentiation circuit 150 in the figure includes capacitors 204 and 205, MOS transistors 218 and 219, and a constant current circuit 231. The MOS transistors 218 and 219 can be p-channel MOS transistors.
[0056] As described above, one end of the capacitor 204 is connected to the output of the current-voltage conversion circuit 140, and the other end of the capacitor 204 is connected to the gate of the MOS transistor 218, the drain of the MOS transistor 219, and one end of the capacitor 205. The other end of the capacitor 205 is connected to the drain of the MOS transistor 218, the drain of the MOS transistor 219, the sink side terminal of the constant current circuit 231, and the signal line 121. The source of the MOS transistor 218 is connected to the power supply line Vdd, and the gate of the MOS transistor 219 is connected to the signal line AZ. The sink side terminal of the constant current circuit 231 is grounded.
[0057] Capacitor 204 corresponds to a coupling capacitor. This capacitor 204 blocks the DC component of the output voltage of current-voltage conversion circuit 140 and passes only the AC component. Also, a current based on a change in the output voltage of current-voltage conversion circuit 140 is supplied to the gate of MOS transistor 218 via capacitor 204. The AC component of the output voltage of current-voltage conversion circuit 140 corresponds to a change in photocurrent.
[0058] The MOS transistor 218 and the constant current circuit 231 configure an inverting amplifier circuit. The change in the output voltage of the current-voltage conversion circuit 140 is input to the gate of the MOS transistor 218 via the capacitor 204, and is inverted and amplified by the MOS transistor 218 and output to the drain. Therefore, a current based on the change in the output voltage of the current-voltage conversion circuit 140 flows through the capacitor 205, and the capacitor 205 is charged and discharged. In other words, the change in the output voltage of the current-voltage conversion circuit 140 is accumulated (integrated). An optical signal that is a signal corresponding to the amount of change in the voltage signal output by the current-voltage conversion circuit 140 is output to the signal line 121.
[0059] The MOS transistor 219 resets the differentiation circuit 150. By making this MOS transistor 219 conductive, both ends of the capacitor 205 are shorted. The integrated change in the output voltage of the current-voltage conversion circuit 140 is discharged and reset. This reset causes the output voltage of the differentiation circuit 150 to become, for example, the voltage at the midpoint between the power supply line Vdd and the ground line. This reset is controlled by an AZ control signal transmitted by a signal line AZ. Hereinafter, the reset of the differentiation circuit 150 will be referred to as an AZ operation.
[0060] <Configuration of the luminance change detection unit> 7, the luminance change detection unit 160 includes MOS transistors 220 to 223. The MOS transistors 220 and 222 can be p-channel MOS transistors. The MOS transistors 221 and 223 can be n-channel MOS transistors. Signal lines ON and OFF from the access control circuit 20 are connected to the luminance change detection unit 160. The signal line ON is a signal line that transmits an on-event detection signal. The signal line OFF is a signal line that transmits an off-event detection signal.
[0061] The signal line 121 is connected to the gate of the MOS transistor 220 and the gate of the MOS transistor 222. The source of the MOS transistor 220 is connected to the power supply line Vdd, and the drain is connected to the drain of the MOS transistor 221 and the gate of the MOS transistor 225 of the output section 170. The gate of the MOS transistor 221 is connected to the signal line ON, and the source is grounded. The source of the MOS transistor 222 is connected to the power supply line Vdd, and the drain is connected to the drain of the MOS transistor 223 and the gate of the MOS transistor 227 of the output section 170. The gate of the MOS transistor 223 is connected to the signal line OFF, and the source is grounded.
[0062] The circuits of the MOS transistors 220 and 221 constitute a comparison circuit. The output of this comparison circuit changes depending on the magnitude relationship between the drain current on the sink side of the MOS transistor 221 and the drain current on the source side of the MOS transistor 220. When the output voltage of the differentiation circuit 150 is lower than a threshold based on the voltage of the on-event detection signal, specifically, a voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd, the source current of the MOS transistor 220 becomes smaller than the sink current of the MOS transistor 221.
[0063] Therefore, the output voltage becomes L level. On the other hand, when the output voltage of the differentiation circuit 150 becomes higher than the threshold voltage (the voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd), the sink current of the MOS transistor 221 becomes smaller than the source current of the MOS transistor 220. Therefore, the output voltage transitions to H level.
[0064] In this way, the comparator circuit consisting of the MOS transistors 220 and 221 compares the output voltage of the differentiation circuit 150 with the threshold voltage of the on-event detection signal, and detects an on-event, which is a change in the direction in which the luminance of the incident light increases. Note that when the on-event detection signal is a voltage higher than the threshold voltage, for example, the power supply voltage of the power supply line Vdd, the output of the comparator is always at the L level. In other words, by applying the threshold voltage as the on-event detection signal, it is possible to detect an on-event.
[0065] The circuit of the MOS transistors 222 and 223 also constitutes a comparison circuit. When the output voltage of the differentiation circuit 150 is lower than a threshold based on the voltage of the off-event detection signal, specifically, a voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd, the output voltage becomes an L level. On the other hand, when the output voltage of the differentiation circuit 150 becomes higher than the threshold voltage (the voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd), the output voltage transitions to an H level.
[0066] By setting the threshold voltage of the off-event detection signal to a voltage lower than the threshold voltage of the on-event detection signal, the comparison circuit of the MOS transistors 222 and 223 detects an off-event, which is a change in the direction of decreasing the luminance of incident light. When the off-event detection signal is at a voltage lower than the threshold voltage, for example, the ground potential, the output of the comparator is always at H level. In other words, by applying the threshold voltage as the off-event detection signal, it is possible to detect an off-event.
[0067] <Output section configuration> The output section 170 includes MOS transistors 224 to 227. The MOS transistors 224 to 227 can be n-channel MOS transistors. The drain of the MOS transistor 224 is connected to one side of the signal line 11, and the drain of the MOS transistor 226 is connected to the other side of the signal line 11. The gates of the MOS transistors 224 and 226 are commonly connected to a signal line OUT. The source of the MOS transistor 224 is connected to the drain of the MOS transistor 225, and the source of the MOS transistor 225 is grounded. The source of the MOS transistor 226 is connected to the drain of the MOS transistor 227, and the source of the MOS transistor 227 is grounded.
[0068] When an event read signal is input to the signal line OUT, the MOS transistors 224 and 226 become conductive. As a result, the drain voltages of the MOS transistors 225 and 227 are output to the signal line 11. Since the on-event detection signal and the off-event detection signal of the luminance change detection unit 160 are applied to the gates of the MOS transistors 225 and 227, an event signal including an on-event signal and an off-event signal is output to the signal line 11.
[0069] In this manner, the grayscale signal generating unit 110 and the event signal generating unit 120 generate the grayscale signal and the event signal, respectively. When generating the grayscale signal, the timing control unit 50 generates a grayscale address signal, which is an address signal of a row for generating the grayscale signal, and outputs the grayscale address signal to the access control circuit 20.
[0070] The access control circuit 20 outputs a selection signal, a reset signal, and a transfer signal to the pixels 100 in a row based on the gradation address signal. On the other hand, when generating an event signal, the timing control unit 50 outputs an on-event detection signal, an off-event detection signal, an AZ control signal, an output signal, and an address signal for an EVS (Event-based Vision Sensor), which is an address signal when reading out the event signal, to the access control circuit 20. The access control circuit 20 sequentially outputs an on-event detection signal, an off-event detection signal, and an AZ control signal to all the pixels 100 in the pixel array unit 10. Thereafter, the access control circuit 20 sequentially outputs the output signal for each row of the pixel array unit 10, and outputs (reads out) the event signal. This will be described with reference to FIG. 8.
[0071] 2 shows an example in which the grayscale signal generating unit 110 and the event signal generating unit 120 each include a photoelectric conversion unit. Alternatively, the pixel 100 may be configured in such a way that the grayscale signal generating unit 110 and the event signal generating unit 120 share one photoelectric conversion unit.
[0072] <Generation of grayscale signals and event signals> 8 is a diagram showing an example of generation of a grayscale signal and an event signal. The figure is a timing diagram showing an example of generation of a grayscale signal and an event signal. The horizontal axis of the figure represents time, and the vertical axis represents row address. Note that the figure shows an example of generation of grayscale signals during a frame period, which is a period during which grayscale signals for all rows of the pixel array section 10 are generated, and generation of event signals during that period.
[0073] The rectangles in the figure represent the periods of shutter 501, exposure 502, and readout 503 described in Fig. 4. As shown in the figure, shutter 501, exposure 502, and readout 503 are performed sequentially while shifting the timing for each row, and one frame of grayscale signal is generated. Such a method of generating a grayscale signal is called a rolling shutter method.
[0074] The solid lines in the figure represent the timing of on-event detection ("ON" in the figure), off-event detection ("OFF" in the figure), AZ operation ("AZ" in the figure), and event signal output (readout, "RD" in the figure). As shown in the figure, on-event detection, off-event detection, and AZ operation are performed simultaneously for the pixels 100 in all rows.
[0075] For this reason, overlap occurs between the row where the grayscale signal is generated and the row where the on-event detection, the off-event detection, and the AZ operation are performed. In the pixels 100 in such a row, the above-mentioned interference occurs. Therefore, an overlap predicted row is detected, which is a row where an overlap is predicted during the period of grayscale signal generation and event detection (for example, the period from the on-event detection and the off-event detection to the AZ operation). In the figure, an overlap predicted row 500 is shown.
[0076] <Configuration of the duplicated predicted line detection unit> 9 is a diagram showing an example of the configuration of an overlapping predicted line detection unit. The figure is a block diagram showing an example of the configuration of an overlapping predicted line detection unit 80. The overlapping predicted line detection unit 80 in the figure includes an overlapping line prediction unit 81, a memory unit (#1) 82, and a memory unit (#2) 83.
[0077] The overlapping row predictor 81 detects the overlapping predicted row based on the control signal from the timing controller 50. The overlapping row predictor 81 receives the gradation address signal, the on-event detection signal, the off-event detection signal, the AZ control signal, the output signal, and the EVS address signal. The overlapping row predictor 81 detects the overlapping predicted row from these control signals. For example, the overlapping predicted row can be detected from the gradation address signal when the on-event detection signal, the off-event detection signal, and the AZ control signal are input. In this case, an error due to interference occurs in either the gradation signal or the event signal.
[0078] The overlapping row prediction unit 81 can also detect the overlapping predicted row from the gradation address signal and the EVS address signal. In this case, since the overlapping predicted row can be detected before generating the event signal, etc., the generation of the gradation signal and the event signal of the overlapping predicted row can be avoided, thereby reducing the influence of interference.
[0079] The overlapping row prediction unit 81 stores information on the overlapping predicted rows detected from the on-event detection signal, the off-event detection signal, and the gradation address signal in a storage unit (#1) 82. The overlapping row prediction unit 81 generates an interference occurrence flag based on the detected information on the overlapping predicted rows, and outputs it to the event signal processing unit 60.
[0080] The overlapping row prediction unit 81 stores information on the overlapping predicted rows detected from the AZ control signal and the gradation address signal in a memory unit (#2) 83. If the AZ operation is affected by interference, the event signal in the next frame period will be affected. Therefore, the information on the overlapping predicted rows in this case is stored in a memory unit (#2) 83 different from the memory unit (#1) 82. When moving to the next frame period, the overlapping row prediction unit 81 generates an interference occurrence flag based on the information on the overlapping predicted rows stored in the memory unit (#2) 83, and outputs it to the event signal processing unit 60.
[0081] <Event data> FIG. 10 is a diagram showing an example of event data. This figure shows an example of event data generated by the event signal processing unit 60. This figure shows a frame 510 of event data for one frame period. In this figure, "FS" is a block indicating the start of the frame. "FE" is a block indicating the end of the frame. "PH" is a block indicating the header of the packet. "PF" is a block indicating the footer of the packet. "EBD" is a block indicating embedded data. "Event" is a block in which an event signal for each row is held. This "Event" is placed between "PH" and "PF". Note that an area 551 in this figure represents the area of data of a row corresponding to the overlapping predicted row 500.
[0082] In the figure, an "Event" corresponding to the area 551 stores an event signal affected by interference. Therefore, an interference occurrence flag is added to the "PH" of the "Event". The thick rectangle in the figure represents the "PH" to which the interference occurrence flag is added. This makes it possible to identify data affected by interference. The identified data can be removed in a device that uses the event data. For example, the event signal processing unit 60 adds mask information to the "PH" of a row based on the interference occurrence flag output from the overlapping predicted row detection unit 80. The interference occurrence flag can also be placed on the "PF" side. The interference occurrence flag is an example of information indicating an overlapping predicted row.
[0083] <Duplicate row detection process> FIG. 11 is a diagram showing an example of a processing procedure of the overlapped line detection process. First, the overlapped line prediction unit 81 judges whether a vertical synchronization signal indicating the start of a frame period has been input (step S100). If the vertical synchronization signal has not been input (step S110, No), the overlapped line prediction unit 81 proceeds to processing of step S103. On the other hand, if the vertical synchronization signal has been input (step S100, Yes), the overlapped line prediction unit 81 initializes the storage unit (#1) 82 (step S101). Next, the overlapped line prediction unit 81 transfers information in the storage unit (#2) to the storage unit (#1) (step S102), and proceeds to processing of step S103. In step S103, the overlapped line prediction unit 81 initializes the storage unit (#2) 83 (step S103).
[0084] Next, the overlapping row prediction unit 81 judges whether an event is being detected (step S104). If an event is being detected (step S104, Yes), the overlapping row prediction unit 81 stores the row generated by the gradation signal in the storage unit (#1) 82 (step S105) and proceeds to the process of step S106. On the other hand, if an event is not being detected (step S104, No), the overlapping row prediction unit 81 judges whether AZ operation is being performed (step S107). If AZ operation is not being performed (step S107, No), the overlapping row prediction unit 81 proceeds to the process of step S106. On the other hand, if AZ operation is being performed (step S107, Yes), the overlapping row prediction unit 81 stores the row generated by the gradation signal in the storage unit (#2) 82 (step S108) and proceeds to the process of step S106.
[0085] In step S106, overlapping row prediction unit 81 judges whether an event signal is being read (step S106). If an event signal is not being read (step S106, No), overlapping row prediction unit 81 ends the process. On the other hand, if an event signal is being read (step S106, Yes), overlapping row prediction unit 81 judges whether row data is stored in memory unit (#1) 82 (step S109). If row data is not stored in memory unit (#1) 82 (step S109, No), overlapping row prediction unit 81 ends the process.
[0086] On the other hand, if row data is stored in the memory unit (#1) 82 (step S109, Yes), the overlapping row predictor 81 outputs an interference occurrence flag to the event signal processor 60 (step S110). Next, the event signal processor 60 adds interference information to the output data (step S111).
[0087] In this way, the photodetector 1 detects a predicted overlap row, which is a row where the generation of a grayscale signal and the detection of an event are predicted to overlap, and adds information indicating the occurrence of interference to the event data including the event signal of the row. This makes it possible to avoid using the event signal affected by interference, and reduce the effect of the interference.
[0088] <Configuration related to amendment> By the above-mentioned process, it is possible to detect event data that may be affected by interference and output an interference occurrence flag. For example, by correcting the event data for which an interference occurrence flag has been output, it is possible to reduce the influence of interference and improve the accuracy of the event data.
[0089] It is also possible to uniformly correct event data for which an interference occurrence flag has been output. In this case, the same correction is performed on event data with strong interference intensity and event data with weak interference intensity. There is a possibility that the correction is not performed appropriately because the correction is weak for event data with strong interference intensity and strong for event data with weak interference intensity. Therefore, an explanation will be given below about how the correction according to the interference intensity is performed to enable appropriate correction.
[0090] Fig. 12 is a diagram showing a configuration example of a correction unit 601 that performs correction according to interference intensity. The correction unit 601 shown in Fig. 12 includes an event data acquisition unit 611, an interference correction unit 621, and an interference intensity identification unit 613. The correction unit 601 shown in Fig. 12 is provided in a sensor, for example, in the light detection device 1 (Fig. 1). The correction unit 601 can be provided in the event signal processing unit 60 (Fig. 1), or provided outside the event signal processing unit 60 at a position where the event data from the event signal processing unit 60 is processed.
[0091] An event data acquisition unit 611 in the correction unit 601 acquires event data, and supplies the acquired event data to an interference correction unit 612 and an interference intensity identification unit 613. The interference correction unit 612 corrects the event data based on information on interference intensity supplied from the interference intensity identification unit 613, and outputs the corrected event data to, for example, a downstream application 602.
[0092] 13 is a diagram showing another example of the configuration of the correction unit 601 that performs correction according to the interference intensity. The correction unit 601 may be configured to be provided in an application (application processor) 602. In this case, for example, an interference correction unit 612 and / or an interference intensity identification unit 613 are provided in the application 602 that acquires and processes event data output from the event signal processing unit 60 (FIG. 1).
[0093] Fig. 14 is a diagram showing yet another example of the configuration of the correction unit 601 that performs correction according to the interference intensity. The correction unit 601 shown in Fig. 14 differs from the correction unit 601 shown in Fig. 13 in that event data and an interference occurrence flag are input to an interference correction unit 612, but has the same configuration in other respects. When the correction unit 601 shown in Fig. 14 is provided in, for example, the light detection device 1 (Fig. 1), it can also be configured to include an event data acquisition unit 611. The correction unit 601 shown in Fig. 14 may be provided in the sensor, or may be provided in an application processor 602 outside the sensor.
[0094] Fig. 15 is a diagram showing yet another example of the configuration of the correction unit 601 that performs correction according to interference intensity. The correction unit 601 shown in Fig. 15 is similar to the correction unit 601 shown in Fig. 14 in that event data and an interference occurrence flag are input to an interference correction unit 612, but differs in that it includes an interference intensity Look Up Table (LTU) 321.
[0095] 15 may be provided in the sensor, or in an application processor 602 outside the sensor. The interference intensity LUT 621 is a table in which a predetermined row in one frame is associated with the interference intensity, as described later, and the interference correction unit 612 refers to the interference intensity LUT 621 and performs correction according to the interference intensity.
[0096] In the case of a configuration provided with the interference intensity LUT 621, it is also possible to adopt a configuration in which the interference occurrence flag is not input, or the interference occurrence flag is not generated, and the part related to the generation of the interference occurrence flag is omitted. In other words, the interference intensity LUT 621 can include the interference occurrence flag, i.e., information on the pixel where interference occurs, and by doing so, it is possible to omit the processing and functions related to the interference occurrence flag.
[0097] <Event data before and after correction> The event data before and after correction will be described with reference to Fig. 16. The upper part of Fig. 16 shows an example of event data in a predetermined row before correction, and the lower part of Fig. 16 shows an example of event data in a predetermined row after correction.
[0098] In Fig. 16, one square represents one pixel (one event signal generating unit 120). P indicates that it is event data of an on-event, and the event data is represented by "01". N indicates that it is event data of an off-event, and the event data is represented by "10". A blank cell indicates that there is no event, and the event data is represented by "00". The numbers 1 to 20 shown at the top of the figure are numbers added for explanation purposes.
[0099] Pixel 1 has no event and the event data is "00". Pixel 2 has an on event and the event data is "01". Pixel 3 has no event and the event data is "00". Pixel 4 has an off event and the event data is "10". Pixel 5 has no event and the event data is "00". Pixel 6 and pixel 7 have on events and the event data is "01".
[0100] Pixels 8 to 11 have no event and the event data is "00". Pixels 12 to 14 have an off event and the event data is "10". Pixels 15 and 16 have no event and the event data is "00". Pixel 17 has an on event and the event data is "01". Pixels 18 to 20 have no event and the event data is "00".
[0101] When the correction unit 601 acquires such event data, the interference correction unit 612 performs correction according to the interference intensity, as will be described later in detail, and outputs the corrected event data as shown in the lower part of Fig. 16. The parts of the event data that have been replaced by the correction will now be described.
[0102] Pixels 2, 7, and 17 had on-event event data, but are corrected to no-event data "00" through correction. Pixels 12 and 13 had off-event event data, but are corrected to no-event data "00" through correction. In this way, pixels where an on-event or off-event has occurred may be corrected to no-event pixels by performing correction.
[0103] Pixel 9 had event data without an event, but is corrected to on-event event data "01." Pixel 10 had event data without an event, but is corrected to off-event event data "10." In this way, by performing correction on a pixel detected as having no event, it may be corrected to a pixel in which an on-event or off-event has occurred.
[0104] A data string as shown in the lower part of Fig. 16 is output to the subsequent stage as corrected event data. In this case, the event data for one pixel before correction is represented by 2 bits, and the event data after correction is also represented by 2 bits. As shown in Fig. 17, the 2-bit event data before correction may be converted to 4-bit event data by correction and output to the subsequent stage.
[0105] The upper part of Fig. 17 is the same as the event data before correction shown in the upper part of Fig. 16, and represents the event data before correction input to the interference correction unit 612. Pixel 1, pixel 3, pixel 5, pixel 8, pixel 11, pixel 15, pixel 16, and pixels 18 to 20 are event data without an event, and remain event data without an event even after correction.
[0106] These pixels are the same event data before and after correction, and are pixels that are treated as event data that does not need to be corrected, or pixels that have become the same event data as a result of correction. In such cases, the 2-bit event data "00" before correction is concatenated with the 2-bit event data "00" after correction, in other words, the event data before correction is converted to 4-bit event data "0000" that is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.
[0107] Pixels 4 and 14 are off-event event data, and remain off-event event data even after correction. These pixels are the same event data before and after correction, and are pixels that were treated as event data that did not need to be corrected, or pixels that became the same event data as a result of correction.
[0108] In such a case, the corrected 2-bit event data "10" is concatenated to the pre-correction 2-bit event data "10". In other words, the pre-correction event data is converted to 4-bit event data "1010" which is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.
[0109] Pixel 6 is an ON event, and remains an ON event after correction. This pixel is the same event data before and after correction, and is a pixel that was treated as event data that does not need to be corrected, or a pixel that became the same event data as a result of correction.
[0110] In such a case, the corrected 2-bit event data "01" is concatenated to the pre-correction 2-bit event data "01". In other words, the pre-correction event data is converted to 4-bit event data "0101" which is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.
[0111] Pixels 2, 7, and 17 have on-event event data, but are corrected to no-event event data after correction. In such a case, the 2-bit event data "00" after correction is concatenated with the 2-bit event data "01" before correction, converting it to 4-bit event data "0100", and this converted event data is output to the subsequent stage as corrected event data.
[0112] Pixels 12 and 13 have event data of an off event, but are corrected to event data of no event after correction. In such a case, the 2-bit event data after correction, "00", is concatenated with the 2-bit event data before correction, "10", and converted to 4-bit event data, "1000", and this converted event data is output to the subsequent stage as corrected event data.
[0113] Pixel 9 is a pixel with no-event event data, but is corrected to on-event event data after correction. In this case, the 2-bit event data "01" after correction is concatenated with the 2-bit event data "00" before correction, converting it to 4-bit event data "0001", and this converted event data is output to the subsequent stage as corrected event data.
[0114] Pixel 10 is a pixel with no-event event data, but after correction it is corrected to off-event event data. In this case, the 2-bit event data "10" after correction is concatenated with the 2-bit event data "00" before correction, converting it to 4-bit event data "0010", and this converted event data is output to the subsequent stage as corrected event data.
[0115] 17 is output to the subsequent stage as corrected event data. For example, if the subsequent stage is an application 602, the application 602 may perform processing using the corrected 4-bit event data, or may perform processing using the latter 2-bit event data (2 bits added after correction processing).
[0116] When the application 602 uses the corrected 4-bit event data, it can be configured to compare the first 2 bits (event data before correction) with the last 2 bits (event data after correction) and execute different processing if they are the same, i.e., no correction was made, and if they are different, i.e., correction was made.
[0117] <Correction section processing> The processing of the correction unit 601 will be described with reference to the flowchart in Fig. 18. Here, the description will be continued by taking the processing of the correction unit 601 in the configuration shown in Fig. 12 as an example, but the basic processing can be performed in the same way in the correction unit 601 shown in any of Figs. 13 to 15.
[0118] In step S201, the event data acquisition unit 611 acquires event data.
[0119] In step S202, the interference intensity identifying unit 613 identifies the interference intensity. The process in step S202 will be described later with reference to FIGS.
[0120] In step S203, the interference correction unit 612 performs correction on the event data according to the interference intensity to generate corrected event data. The process in step S203 will be described later with reference to FIGS.
[0121] In step S204, the corrected event data is output to a downstream processing unit.
[0122] <Regarding the first interference intensity identification process> The interference intensity identification process in step S202 will be described with reference to the flowchart in Fig. 19. The process based on the flowchart shown in Fig. 19 is an interference intensity identification process (referred to as a first interference intensity identification process) performed in a photodetector 1 including a pixel array unit 10 having an optical black region, as shown in Fig. 20.
[0123] In step S221, the number of events in a HOPB (Horizontal Optical black) region of the row of interest is counted. The HOPB region will be described with reference to Fig. 20. The pixel region of the pixel array unit 10 is configured to have an effective pixel region 652 (Visible region in the figure) that actually receives light, amplifies signal charges generated by photoelectric conversion, and reads them out to a column signal processing circuit, and a HOPB region 651 for outputting optical black that serves as a reference for the black level.
[0124] 20 shows an example in which an optical black area is provided in the vertical direction (the lengthwise direction in the figure), and this area is referred to as an HOPB area 651. The optical black area may also be provided in the horizontal direction. In FIG. 20, an example in which the HOPB area 651 is provided on both the left and right sides of the figure is shown, but a configuration in which the HOPB area 651 is provided only on the left side or only on the right side is also possible.
[0125] In step S221 (FIG. 19), events occurring in a row of interest in the HOPB area 651 are counted. The HOPB area 651 is a light-shielded area where there is no change in illuminance and where it is expected that no events will occur. If interference occurs, there is a possibility that an event will occur even in such an area. If strong interference occurs, the number of events occurring in the HOPB area 651 will also increase. Using this, the interference intensity is detected.
[0126] In step S221, the number of events occurring in the HOPB area 651 of the row (row of interest) for which interference intensity is to be measured is counted. In step S222, the number of counted events is converted into interference intensity. For example, a table as shown in Fig. 21 is provided, and the table is referenced to convert the number of events into interference intensity.
[0127] 21, table 671 is a table in which the number of counted events is associated with interference intensity. Table 671 shown in Fig. 21 shows, for example, a case in which the number of event data in one row of HOPB area 651 is 500. This is just one example, and the number of event data in HOPB area 651 may be set to a value other than 500, and each value described below may of course be a value other than that shown below.
[0128] If the number of counted events is "10 or less", the interference intensity is converted to "weak". If the number of counted events is "11 or more and 100 or less", the interference intensity is converted to "medium". If the number of counted events is "101 or more", the interference intensity is converted to "strong".
[0129] Note that a configuration other than the configuration in which table 671 is provided and referenced to convert the number of events into interference intensity may be used, for example a configuration in which interference intensity is calculated from the number of events using a predetermined arithmetic expression.
[0130] The number of events may be counted separately for on-events and off-events, or may be counted together without distinguishing between on-events and off-events.
[0131] On-events and off-events may be counted separately, and the interference strength of the on-events and the interference strength of the off-events may be estimated separately, or on-events and off-events may be counted together without distinction, and the interference strength may be estimated using the count value obtained by counting these together.
[0132] Here, the interference intensity is classified into three categories, strong, medium, and weak, in other words, an example is shown in which the interference intensity is classified discretely, but it may also be estimated and output as a continuous value, for example, interference intensity = 0.8.
[0133] In step S 223 , the interference intensity converted from the number of events in step S 222 is output from the interference intensity identifying unit 613 to the interference correcting unit 612 .
[0134] In this manner, when a light-shielded region is provided in the pixel array unit 10, the interference intensity determination unit 613 can be configured to determine the interference intensity using event data obtained from within this light-shielded region.
[0135] <Second interference intensity identification process> The interference intensity identifying process (hereinafter referred to as second interference intensity identifying process) in step S202 will be described with reference to the flowchart in FIG.
[0136] The first interference intensity identification process described with reference to Fig. 19 has been described by taking as an example a process of counting the number of events and identifying the interference intensity using the HOPB area 651 of the pixel array unit 10. The second interference intensity identification process described with reference to Fig. 22 has been described by taking as an example a process of counting the number of events and identifying the interference intensity using the effective pixel area 652 (Fig. 20) of the pixel array unit 10.
[0137] In step S241, the number of events occurring in the row of interest in the effective pixel area 652 is counted. The value of the counted number of events is set as a value a.
[0138] In step S242, the larger value of the number of events between the non-interfering rows above and below the row of interest is acquired. The acquired value is designated as value b. In step S242, the number of events obtained from the row that is above the row of interest in the effective pixel area 652 and is determined to be free of interference, and the number of events obtained from the row that is below the row of interest in the effective pixel area 652 and is determined to be free of interference, are acquired and compared, and the larger value is acquired as value b.
[0139] In this example, the larger event count value of the non-interfering rows above and below the row of interest is obtained and used, but other values may be used. For example, the average value of the event counts of the non-interfering rows above and below the row of interest may be calculated and used.
[0140] An example will be described in which the number of events in the non-interfering rows above and below the row of interest is used, but it is also possible to configure the system to use either the number of events in the non-interfering row above the row of interest or the number of events in the non-interfering row below the row of interest.
[0141] In the effective pixel region 652, normal events also exist, in other words, unlike the HOPB region 651, events occurring in a normal state where no interference occurs also exist, so the intensity of interference cannot be estimated by simply counting the number of events as in the HOPB region 651. Therefore, the interference intensity of the row of interest is identified by comparing the number of events in the row of interest with the number of events in the non-interfering rows above and below the row of interest, in other words, the number of events in rows where events are normally detected. The interference intensity is estimated based on how much more the number of events in the row of interest is compared with the number of events in the non-interfering rows.
[0142] To perform such a comparison, in step S242, the larger value b of the number of events of the non-interfering rows above and below the row of interest is acquired. Whether the rows above and below are non-interfering rows can be determined, for example, using an interference occurrence flag. For example, a row located above the row of interest and for which the interference occurrence flag is not set is treated as the upper non-interfering row, and a row located below the row of interest and for which the interference occurrence flag is not set is treated as the lower non-interfering row.
[0143] Since the non-interfering row is a row for which the interference occurrence flag is not set, it is not necessarily a row adjacent to the target row. For example, if the interference occurrence flag is set for the row immediately above the target row, that row is avoided and it is determined whether the interference occurrence flag is set for the row above that, and if the interference occurrence flag is set, it is determined whether the interference occurrence flag is set for the row above that. This process is repeated until a row is found that is determined to not have the interference occurrence flag set.
[0144] In step S243, the interference intensity is identified by comparing the value a with the value b. For example, the conditions shown in FIG. 23 can be set so that the interference intensity that meets the conditions is identified. The conditions shown in FIG. 23 show an example of the conditions when the number of event data in one row is 1920. The number of event data in one row is not limited to 1920, and may be a number of event data other than 1920. Furthermore, the conditions are set appropriately according to the number of event data. The numerical values in the conditions shown in FIG. 23, for example, 1.5, 10, and 200, are examples and are not limited, and appropriate values can be set appropriately according to the number of event data, etc.
[0145] Referring to FIG. 23, if the condition "value a≦1.5×value b+10" is met, the interference strength is estimated to be "weak." If the condition "value a≧1.5×value b+200" is met, the interference strength is estimated to be "strong." If the condition "the above two conditions are not met" is met, the interference strength is estimated to be "medium."
[0146] The number of events may be counted separately for on-events and off-events, or may be counted together without distinguishing between on-events and off-events.
[0147] On-events and off-events may be counted separately, and the interference strength of the on-events and the interference strength of the off-events may be estimated separately, or on-events and off-events may be counted together without distinction, and the interference strength may be estimated using the combined count number.
[0148] Here, the interference intensity is classified into three categories, strong, medium, and weak, in other words, an example is shown in which the interference intensity is classified discretely, but it may be estimated and output as a continuous value, for example, interference intensity = 0.8. In this case, the above conditions can be changed so that the interference intensity can be calculated by a predetermined arithmetic expression in which, for example, values a and b are substituted.
[0149] In this manner, the interference intensity can be determined by using event data obtained from within the effective pixel area 652 of the pixel array section 10.
[0150] <About the third interference intensity identification process> The interference intensity identifying process (referred to as third interference intensity identifying process) in step S202 will be described with reference to the flowchart in FIG.
[0151] The third interference intensity identification process is performed when the correction unit 601 is configured to include an interference intensity LUT 621 as shown in Fig. 15. In step S261, a timestamp and a line of interest of an event are set. The timestamp is information that can be treated equivalently to a frame number, and is information for identifying a frame (information for identifying the point in time at which the frame was acquired). In step S261, a frame to be processed and a line (line of interest) for identifying the interference intensity in that frame are set.
[0152] In step S262, the interference intensity corresponding to the timestamp and the row of interest is acquired from the interference intensity LUT 621. Fig. 25 is a diagram showing an example of the interference intensity LUT 621. The interference intensity LUT 621 is a table in which the timestamp, the row of interest, and the interference intensity are associated with each other.
[0153] The interference intensity LUT 621 is a table in which a timestamp, a row of interest, and an interference intensity are associated with each other. In the interference intensity LUT 621 shown in Fig. 25, for example, "00000000" is associated as a timestamp, "0" is associated as a row of interest, and "weak" is associated as an interference intensity, and "00000000" is associated as a timestamp, "1" is associated as a row of interest, and "medium" is associated as an interference intensity. In the interference intensity LUT 621 shown in Fig. 25, for example, "00000015" is associated as a timestamp, "100" is associated as a row of interest, and "strong" is associated as an interference intensity.
[0154] Such an interference intensity LUT 621 is created, for example, when a pattern of how the interference intensity changes can be obtained, and is provided in the correction unit 601. Frames or rows in which interference is likely to occur may depend on the device, and the interference intensity LUT 621 can be created when there are similar patterns depending on the device.
[0155] When the transition of the interference intensity is periodic, it is sufficient that the LUT for one period is stored in the interference intensity LUT 621. The interference intensity LUT 621 may be provided with a mechanism for learning and updating by acquiring an interference occurrence flag.
[0156] The table of the interference intensity LUT 621 can be configured so that a table is provided for each of two types of events, ON events and OFF events, or a table that does not distinguish between ON events and OFF events can be provided as a single table.
[0157] Here, the interference intensity is classified into three categories, namely, strong, medium, and weak, in other words, an example is shown in which the interference intensity is classified discretely, but it may also be estimated as a continuous value, for example, interference intensity = 0.8, and the value of the continuous value may be stored in the interference intensity LUT 621.
[0158] In step S262, the interference intensity corresponding to the timestamp and the row of interest set as the frame to be processed in step S261 is read out from the interference intensity LUT 621. In step S263, the interference intensity read out from the interference intensity LUT 621 is output from the interference intensity LUT 621 to the interference correction unit 612.
[0159] In this way, when there is a specific pattern in the way interference intensity appears, it is possible to create an interference intensity LUT 621 in advance and to specify the interference intensity by referring to the interference intensity LUT 621.
[0160] Returning to the flowchart of Fig. 18, in step S202, an interference intensity identification process is executed. The interference intensity identification process is executed by executing any one of the first to third interference intensity identification processes described above, or by executing a combination of the first to third interference intensity identification processes. When the interference intensity is identified by executing the interference intensity identification process, in step S203, a correction process of the event data is executed.
[0161] <Correction process of first event data> The event data correction process executed in step S203 will now be described with reference to a flowchart of FIG 26. The first event data correction process is described with reference to the flowchart of FIG 26.
[0162] In step S301, the interference correction unit 612 acquires event data of a row of interest (a row to be processed) output from the event data acquisition unit 611. In step S302, the interference correction unit 612 acquires the interference intensity of the row of interest identified by the interference intensity identification unit 613.
[0163] Here, the explanation will be continued by taking as an example a case where the interference intensity identifying unit 613 identifies three types of interference intensity, weak, medium, and strong, as the interference intensity, and outputs them to the interference correcting unit 612. The event data correction method is switched depending on the interference intensity. In step S303, the interference correcting unit 612 determines whether the interference intensity is weak or not.
[0164] If it is determined in step S303 that the interference intensity is weak, the process proceeds to step S304. If the interference intensity is weak, it is determined that the number of false events caused by interference is small, and no correction process is performed (step S304), and the input event data is output to a subsequent processing unit. As described with reference to FIG. 17, when 2-bit data before correction is converted to 4-bit data and output, in step S304, the 2-bit data before correction is repeated and output as 4-bit data.
[0165] On the other hand, if it is determined in step S303 that the interference strength is not weak, the process proceeds to step S305. In step S305, it is determined whether the interference strength is medium. If it is determined in step S305 that the interference strength is medium, the process proceeds to step S306. In step S306, noise events are removed.
[0166] The removal of noise events performed in step S306 will be described with reference to Fig. 27. When the interference intensity is medium, normal events are held, and false events due to interference are considered to have been added as noise, and a process for removing such noise events is performed.
[0167] The left diagram in A of Fig. 27 is a diagram showing an arrangement of pixels (pixels including the event signal generation unit 120) before correction and event data of an event that occurs in the event signal generation unit 120. Of the pixels 120-1 to 120-7 shown in A of Fig. 27, the pixel that is located in the target row determined to have medium interference intensity and is set as the correction target is set as pixel 120-4.
[0168] The pixels in the non-interfering row above the pixel 120-4 of interest are designated as pixels 120-1 to 120-3. The pixel 120-1 is located at the upper left of the pixel 120-4 of interest, the pixel 120-2 is located directly above the pixel 120-4 of interest, and the pixel 120-3 is located at the upper right of the pixel 120-4 of interest. Note that, although the pixels 120-1 to 120-3 are illustrated as being located in rows adjacent to the pixel 120-4 of interest in A of FIG. 27, when the non-interfering rows are not adjacent, the pixels 120-1 to 120-3 may be located in non-interfering rows located several rows away.
[0169] The pixels in the non-interfering row below the pixel 120-4 of the pixel of interest are designated as pixels 120-5 to 120-7. The pixel 120-5 is located on the lower left side of the pixel of interest 120-4, the pixel 120-6 is located directly below the pixel of interest 120-4, and the pixel 120-7 is located on the lower right side of the pixel of interest 120-4. Note that, although the pixels 120-5 to 120-7 are illustrated in A of FIG. 27 as being located in a row adjacent to the pixel of interest 120-4, when the non-interfering row is not adjacent, the pixels 120-5 to 120-7 may be located in a non-interfering row located several rows away.
[0170] The following description will be given by taking such pixel arrangement as an example. Here, the pixels where an event occurs are shown with diagonal lines, and the pixel of interest 120-4 is shown as a pixel where an ON event or an OFF event occurs. In the example shown in A of FIG. 27, pixel 120-1 is also shown as a pixel where an ON event or an OFF event occurs, and is shown with diagonal lines. Here, pixels with high density diagonal lines such as pixel 120-1 are pixels where an ON event occurs, and pixels with low density diagonal lines such as pixel 120-4 are pixels where an OFF event occurs.
[0171] A total of six pixels are set as search target pixels: pixels 120-1 to 120-3 at ±1 pixel in the x direction in the non-interfering row located above the pixel of interest 120-4 in the interfering row, and pixels 120-5 to 120-7 at ±1 pixel in the x direction in the non-interfering row located below the pixel of interest 120-4. If no event of the same polarity as the event of the pixel of interest 120-4 is present among the six pixels set as search target pixels, the pixel is determined to be an isolated point and is targeted for deletion as a noise event.
[0172] In the example shown in A of Fig. 27, no OFF event has occurred in six pixels, search pixels 120-1 to 120-3 and 120-5 to 120-7, for the OFF event pixel of interest 120-4. In this case, it is determined that there is no event of the same polarity (OFF event) as pixel of interest 120-4 in the search target pixels, and they are processed as isolated points. As shown in the right diagram of A of Fig. 27, pixel of interest 120-4 in which an OFF event has been detected is corrected to pixel 120-4 in which no event has occurred.
[0173] In this way, when the event data of the pixel of interest does not match any of the event data of the multiple (six in this case) search pixels, the event data of the pixel of interest is corrected to data without an event.
[0174] Please refer to B of Fig. 27. The left diagram of B of Fig. 27 shows an example of event occurrence in a pixel arrangement before correction. In the example shown in the left diagram of B of Fig. 27, an OFF event occurs in the pixel of interest 120-4, an ON event occurs in the pixel 120-1 to be searched, and an OFF event occurs in the pixel 120-7 to be searched.
[0175] In the case of the event occurrence situation shown in B of Fig. 27, an OFF event occurs in the pixel of interest 120-4, and an OFF event also occurs in pixel 120-7, which is the pixel to be searched. In this case, it is determined that the pixel to be searched has an event of the same polarity (OFF event) as the pixel of interest 120-4, and is held as a normal event. As shown in the right diagram of B of Fig. 27, the pixel of interest 120-4 in which the OFF event was detected is treated as the pixel 120-4 in which the OFF event occurs.
[0176] In this way, if the event data of the pixel of interest matches the event data of at least one of the multiple (six in this case) search pixels, the event data of the pixel of interest is maintained as is.
[0177] Here, an example has been described in which six pixels are searched for, but fewer than six pixels or more than six pixels may be set as the search target pixels.
[0178] In this way, when the interference intensity is medium, a correction is performed to remove isolated points.
[0179] Returning to the explanation with reference to the flowchart in Fig. 26, if it is determined in step S305 that the interference intensity is not medium, that is, in this case, if it is determined that the interference intensity is strong, the process proceeds to step S307. In step S307, correction is performed on all event data. If the interference intensity is strong, there is a possibility that many erroneous events have occurred due to interference, so correction is performed on all event data in the row of interest.
[0180] The correction of all event data executed in step S307 will be described with reference to Fig. 28 to Fig. 30. In Fig. 28 to Fig. 30, the upper part shows the pixel arrangement and an example of event occurrence before correction, and the lower part shows the pixel arrangement and an example of event occurrence after correction. As in the case shown in Fig. 27, the pixel to be corrected is the pixel of interest 120-4, and the event data of the pixel of interest 120-4 is corrected according to the event occurrence status of the six pixels surrounding it, namely, pixels 120-1 to 120-7 (excluding pixel 120-4).
[0181] The correction rule for the case where the interference intensity is strong, which will be described with reference to Fig. 28, is set as correction rule 1. Correction rule 1 is a correction that performs masking and deletes the event of the pixel of interest.
[0182] An example will be described in which an OFF event occurs in the pixel of interest 120-4. Before correction shown in the upper part of A in Fig. 28, an ON event occurs in both pixels 120-2 and 120-6, which are search pixels located directly above and below the pixel of interest 120-4. As shown in the lower part of A in Fig. 28, since the event of the pixel of interest is deleted in correction rule 1, the pixel of interest 120-4 is corrected to event data in which the event has been deleted and no event has occurred, and is output.
[0183] Before correction shown in the upper part of B in Fig. 28, an ON event occurs in both pixel 120-2 and pixel 120-7, which are search pixels located directly above and to the lower right of pixel of interest 120-4. As shown in the lower part of B in Fig. 28, since the event of the pixel of interest is deleted under correction rule 1, pixel of interest 120-4 is corrected to event data in which the event has been deleted and no event has occurred, and is output.
[0184] Before the correction shown in the upper part of C in Fig. 28, an ON event occurs in pixel 120-2, which is a search pixel located directly above pixel of interest 120-4. As shown in the lower part of C in Fig. 28, since the event of the pixel of interest is deleted in correction rule 1, pixel of interest 120-4 is corrected to event data in which the event has been deleted and no event has occurred, and is output.
[0185] Before correction shown in the upper part of D in Fig. 28, an ON event occurs in pixel 120-2 of the search pixel located directly above pixel of interest 120-4, and an OFF event occurs in pixel 120-6 of the search pixel located directly below pixel of interest 120-4. As shown in the lower part of D in Fig. 28, since the event of the pixel of interest is deleted in correction rule 1, the event of pixel of interest 120-4 is deleted and corrected to event data in which no event has occurred, and output.
[0186] 29, correction rule 2 for the case where the interference intensity is strong will be described. Correction rule 2 is a rule that overwrites the event data of the pixel of interest 120-4 when events occurring in search pixels located in non-interfering rows above and below the pixel of interest 120-4 and directly above and below the pixel of interest 120-4 match, and deletes the event of the pixel of interest when they do not match.
[0187] The examples shown in the upper rows of A to D in FIG. 29 show examples of pixel arrangements and event occurrences before correction, and are similar to the examples shown in the upper rows of A to D in FIG.
[0188] Referring to the upper part of A in Fig. 29, before correction, ON events occur in both pixels 120-2 and 120-6, which are search pixels located directly above and below the pixel of interest 120-4. In this way, when events of the pixels located above and below the pixel of interest 120-4 match, the event of the pixel of interest 120-4 is rewritten to the matching event. Therefore, as shown in the lower part of A in Fig. 29, the pixel of interest 120-4 was an OFF event, but the OFF event is rewritten to the ON events occurring in the pixels above and below it, and output.
[0189] Before correction shown in the upper part of B in Fig. 29, an ON event occurs in both pixel 120-2 and pixel 120-7 of the search pixels located directly above and to the lower right of pixel of interest 120-4, but no event occurs in pixel 120-6 of the search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, since the events of the pixels located above and below pixel of interest 120-4 do not match, the event of pixel of interest 120-4 is deleted and rewritten as no event, as shown in the lower part of B in Fig. 29, and output.
[0190] Before the correction shown in the upper part of C in Fig. 29, an ON event occurs in pixel 120-2 of the search pixel located directly above pixel of interest 120-4, but no event occurs in pixel 120-6 of the search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, the events of the pixels located above and below pixel of interest 120-4 do not match, so that the event of pixel of interest 120-4 is deleted and rewritten as no event, and output, as shown in the lower part of C in Fig. 29.
[0191] Before the correction shown in the upper part of D in Fig. 29, an ON event occurs in pixel 120-2 of the search pixel located directly above the pixel of interest 120-4, and an OFF event occurs in pixel 120-6 of the search pixel located directly below the pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, the events of the pixels located above and below the pixel of interest 120-4 do not match, so that the event of the pixel of interest 120-4 is deleted and rewritten as no event, and output, as shown in the lower part of D in Fig. 29.
[0192] 30, correction rule 3 for the case where the interference intensity is strong will be described. Correction rule 3 is a rule that overwrites the event data of pixel of interest 120-4 when events occurring in search pixels located in non-interfering rows above and below pixel of interest 120-4 match, and when events occurring in search pixels directly above and below pixel of interest 120-4 do not match but an event occurring in a search pixel in a diagonal direction matches instead of the pixel directly above or below, and deletes the event of the pixel of interest when they do not match.
[0193] The examples shown in the upper rows of A to D in FIG. 30 show examples of pixel arrangements and event occurrences before correction, and are similar to the examples shown in the upper rows of A to D in FIG.
[0194] Referring to the upper part of A in Fig. 30, before correction, ON events occur in both pixels 120-2 and 120-6, which are search pixels located directly above and below the pixel of interest 120-4. In this way, when events of the pixels located above and below the pixel of interest 120-4 match, the event of the pixel of interest 120-4 is rewritten to the matching event. Therefore, as shown in the lower part of A in Fig. 30, the pixel of interest 120-4 is an OFF event, but the OFF event is rewritten to the ON events occurring in the pixels above and below it, and output.
[0195] Before the correction shown in the upper part of B in Fig. 30, an ON event occurs in both pixel 120-2 and pixel 120-7 of the search pixels located directly above and to the lower right of the pixel of interest 120-4, but no event occurs in pixel 120-6 of the search pixel located directly below the pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below the pixel of interest 120-4 do not match, but the events of the pixel located above and diagonally below the pixel of interest 120-4 match as an ON event, so that the event of the pixel of interest 120-4 is corrected from an OFF event to an ON event and output, as shown in the lower part of B in Fig. 30.
[0196] Before the correction shown in the upper part of C in Fig. 30, an ON event occurs in pixel 120-2 of the search pixel located directly above pixel of interest 120-4, but no event occurs in pixel 120-6 of the search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below pixel of interest 120-4 do not match, and they also do not match with the events of pixels located diagonally, so that the event of pixel of interest 120-4 is deleted and rewritten as no event and output, as shown in the lower part of C in Fig. 30.
[0197] Before the correction shown in the upper part of D in Fig. 30, an ON event occurs in pixel 120-2 of the search pixel located directly above the pixel of interest 120-4, and an OFF event occurs in pixel 120-6 of the search pixel located directly below the pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below the pixel of interest 120-4 do not match, and they also do not match with the events of the pixels located diagonally, so that the event of the pixel of interest 120-4 is deleted and rewritten as no event and output, as shown in the lower part of D in Fig. 30.
[0198] When the interference intensity is strong, the event data of the pixel of interest is corrected by applying any one of the correction rules 1 to 3. The correction rules 1 to 3 can also be configured to be used in combination.
[0199] In the above-mentioned embodiment, the non-interfering row may include the interfering row after correction. In other words, the interfering row after correction may be treated as the non-interfering row when processing the target row. In such a case, the interfering row with medium or weak interference intensity and the correction is completed may be treated as the non-interfering row, but the interfering row with strong interference intensity and the correction is completed may not be treated as the non-interfering row.
[0200] In the case where a non-interfering line is located at a position that is a predetermined number of lines or more away from the line of interest, the corrected interfering line that is close to the line of interest may be treated as a non-interfering line and the above-mentioned correction processing may be performed.
[0201] <Correction process of second event data> The event data correction process executed in step S203 will now be described. Fig. 31 is a flowchart for explaining the second event data correction process. The second event data correction process is a correction process in which the interference intensity is output as a continuous value.
[0202] In step S321, the interference correction unit 612 acquires the event data of the row of interest (row to be processed) output from the event data acquisition unit 611. In step S322, the interference correction unit 612 acquires the interference intensity of the row of interest identified by the interference intensity identification unit 613. The interference intensity acquired here is a continuous value, for example, a value such as 0.8.
[0203] In step S323, the interference intensity is converted into a correction factor. For example, the interference intensity is converted into a correction factor based on the graph shown in FIG. 32. In the graph shown in FIG. 32, the horizontal axis indicates the interference intensity, and the vertical axis indicates the correction factor. s0 and s1 shown on the horizontal axis each represent a control parameter. When the interference intensity is converted into a correction factor based on the graph shown in FIG. 32, if the interference intensity is 0 to s0, the correction factor is converted into "0", and if the interference intensity is s1 or more, the correction factor is converted into "1". If the interference intensity is s0 to s1, the correction factor is converted into a value expressed by a linear function according to the value of the interference intensity.
[0204] Returning to the flowchart shown in FIG. 31, in step S324, a pixel to be corrected in the row of interest, for example a pixel corresponding to pixel of interest 120-4 in the case described with reference to FIGS. 28 to 30, is set.
[0205] In step S325, a uniform random number of [0.0, 1.0] is generated. In step S326, the generated random number is compared with the converted correction factor to determine whether the value of the random number is equal to or less than the correction factor. This determination is made in order to control the number of event data to be processed according to the correction factor, for example, to reduce the number of event data to be corrected when the correction factor is small, and to increase the number of event data to be corrected when the correction factor is large.
[0206] If it is determined in step S326 that the value of the random number is equal to or less than the correction factor, the process proceeds to step S327, in which the pixel of interest is corrected.
[0207] In step S326, the random number is determined to be equal to or smaller than the correction factor when the interference intensity is determined to be strong or medium in the first event data correction process. Therefore, the correction of the pixel of interest performed in step S327 can be performed using the processes performed in steps S306 and S307 (FIG. 26).
[0208] On the other hand, if it is determined in step S326 that the value of the random number is not equal to or less than the correction factor, the process proceeds to step S328, in which no correction process is performed on the pixel of interest, and the event data of the pixel of interest is output.
[0209] In step S326, the random number is determined not to be equal to or less than the correction factor when the interference intensity is determined to be weak in the first event data correction process. Therefore, the correction performed in step S328 can be the same as the process performed in step S304 (FIG. 26), that is, the process of not performing the correction process in this case.
[0210] When the processing in step S327 or step S328 is completed, the process proceeds to step S329. In step S329, it is determined whether or not one row of pixels has been processed. If it is determined in step S328 that there are pixels in the target row that have not yet been processed, the process returns to step S324, and the processing from step S324 onwards is repeated with the unprocessed pixels in the target row as the processing target.
[0211] On the other hand, if it is determined in step S329 that all pixels in the target row have been processed, the process of the flowchart shown in FIG. 31 ends, and the process proceeds to step S204 (FIG. 18).
[0212] In step S204, the corrected event data of the target row is output to a downstream processing unit.
[0213] In this way, the event data is corrected according to the interference intensity. By performing the correction according to the interference intensity, it is possible to perform an appropriate correction and improve the accuracy of the event data.
[0214] <Input / output response in the correction section> A description will now be given of input / output responses in the correction unit 601. By changing the event data input to the correction unit 601 and observing the event data output from the correction unit 601, it is possible to determine whether the correction unit 601 applies the present technology or not.
[0215] Event data that is not interfered with is prepared, and an interference occurrence flag is assigned to this event data that is not interfered with. An erroneous event is assigned to the row to which the interference occurrence flag is assigned. This assigned erroneous event may be an on event or an off event.
[0216] The event data with the false event added is input to the correction unit 601, and the output of the correction unit 601 is observed. At this time, the false event addition rate is changed between 0 and 100%, and the output is observed each time the rate is changed. In other words, the interference strength is changed between weak (none) and strong, and the output is observed each time the rate is changed.
[0217] As described above, when the correction unit 601 to which the present technology is applied performs correction, when the false event attachment rate is changed between 0 and 100%, correction is performed according to the false event attachment rate, in other words, correction is performed according to the interference intensity, so that the output event data differs for each attachment rate. Specifically, the lower the false event attachment rate, the more the original event data is retained in the output.
[0218] If the correction unit 601 performs correction uniformly instead of according to the interference intensity, the output event data will not change and will be the same even if the false event assignment rate changes.
[0219] In this way, the output result is different between the correction unit 601 to which the present technology is applied and the correction unit to which the present technology is not applied. According to the correction unit 601 to which the present technology is applied, it can be confirmed by such a confirmation method that appropriate correction is performed.
[0220] <About recording media> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, capable of executing various functions by installing various programs.
[0221] 33 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program. In the computer, a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, and a RAM (Random Access Memory) 2003 are connected to one another by a bus 2004. An input / output interface 2005 is further connected to the bus 2004. An input unit 2006, an output unit 2007, a storage unit 2008, a communication unit 2009, and a drive 2010 are connected to the input / output interface 2005.
[0222] The input unit 2006 includes a keyboard, a mouse, a microphone, etc. The output unit 2007 includes a display, a speaker, etc. The storage unit 2008 includes a hard disk, a non-volatile memory, etc. The communication unit 2009 includes a network interface, etc. The drive 2010 drives removable media 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0223] In the computer configured as above, the CPU 2001 loads, for example, a program stored in the storage unit 2008 into the RAM 2003 via the input / output interface 2005 and the bus 2004, and executes the program, thereby performing the series of processes described above.
[0224] The program executed by the computer (CPU 2001) can be provided by being recorded on a removable medium 2011 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
[0225] In the computer, the program can be installed in the storage unit 2008 via the input / output interface 2005 by mounting the removable medium 2011 in the drive 2010. The program can also be received by the communication unit 2009 via a wired or wireless transmission medium and installed in the storage unit 2008. Alternatively, the program can be installed in the ROM 2002 or the storage unit 2008 in advance.
[0226] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.
[0227] <Applications to moving objects> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.
[0228] FIG. 34 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.
[0229] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 34, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.
[0230] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.
[0231] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.
[0232] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.
[0233] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.
[0234] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.
[0235] The microcomputer 12051 can calculate control target values for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.
[0236] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.
[0237] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.
[0238] The audio / video output unit 12052 transmits at least one output signal of audio and video to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 34, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.
[0239] FIG. 35 is a diagram showing an example of the installation position of the imaging unit 12031.
[0240] In FIG. 35, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.
[0241] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.
[0242] 35 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.
[0243] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.
[0244] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.
[0245] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.
[0246] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.
[0247] In this specification, the term "system" refers to an entire apparatus that is made up of a plurality of devices.
[0248] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.
[0249] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.
[0250] The present technology can also be configured as follows. (1) a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light; an interference intensity identifying unit that identifies an intensity of interference occurring due to an overlap between a period of generating the gradation data and a period of detecting the event; a correction unit that corrects the event data in accordance with the intensity of the interference identified by the interference intensity identification unit; A light detection device comprising: (2) the pixel array portion has a light-shielded region, The interference intensity identifying unit counts the number of events detected by the event data generating unit located in the light-shielded area, and identifies the intensity of the interference according to the counted number of events. The light detection device according to (1) above. (3) The interference intensity identification unit is obtaining a first number of events by counting events in a row of interest that identifies a strength of interference in the pixel array unit; Obtaining a second number of events in a non-interfering row located above the row of interest and free from interference; Obtaining a third number of events in a non-interfering row that is located below the row of interest and has no interference; The intensity of the interference is identified by determining whether or not a predetermined condition is satisfied using the first number of events and the larger number of the second number of events and the third number of events. The light detection device according to (1) above. (4) obtaining an interference occurrence flag indicating that an overlap predicted row, which is a row in which an overlap between a period of the generation of the gradation data and a period of the detection of the event is predicted, has been detected; The non-interfering row is a row in which the interference occurrence flag is not set. The optical detection device according to (3) above. (5) a table in which information for identifying a frame, a row of interest for identifying an intensity of interference of the pixel array unit, and the intensity of interference are associated with each other; The interference intensity identification unit refers to the table and identifies the intensity of the interference in the target row. The light detection device according to (1) above. (6) When the interference intensity identifying unit identifies that the intensity of the interference is weak, the correction unit does not perform correction. The optical detection device according to any one of (1) to (5). (7) When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correcting unit corrects the event data to indicate that no event has occurred in the pixel of interest that is the correction target. The optical detection device according to any one of (1) to (6). (8) When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correction unit: Obtaining a first event of a pixel located above the pixel of interest to be corrected and directly above the pixel of interest in a non-interfering row where there is no interference, and a second event of a pixel located below the pixel of interest and directly below the pixel of interest in a non-interfering row where there is no interference; If the first event and the second event match, correcting the event of the pixel of interest to event data indicating that the first event has occurred; If the first event and the second event do not match, the event data is corrected to indicate that no event has occurred in the pixel of interest. The optical detection device according to any one of (1) to (6). (9) When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correction unit: obtain a first event of a pixel located above the pixel of interest to be corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and obtain a second event of a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in the non-interfering row where there is no interference; If the first event and the second event match, correcting the event of the pixel of interest to event data indicating that the first event has occurred; If the first event and the second event do not match, the event data is corrected to indicate that no event has occurred in the pixel of interest. The optical detection device according to any one of (1) to (6). (10) When the interference intensity identifying unit identifies the intensity of the interference as medium, the correction unit: A pixel located above the pixel of interest to be corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference are set as search pixels; maintain event data of the event of the pixel of interest if the event of the pixel of interest and at least one of the events of the plurality of search pixels match; If an event that matches the event of the pixel of interest does not occur in any of the plurality of search pixels, the event data is corrected to indicate that no event has occurred in the pixel of interest. The optical detection device according to any one of (1) to (9). (11) The interference intensity identifying unit outputs the intensity of the interference as a continuous value, The correction unit is converting said continuous values into correction factors; Generate random numbers, If the random number is equal to or smaller than the correction factor, correcting the event data of the pixel of interest being processed; If the random number is not equal to or less than the correction factor, the event data of the pixel of interest being processed is maintained. The optical detection device according to any one of (1) to (9). (12) The input and output of the correction unit are event data with the same number of bits, When the input event data is corrected, the corrected event data is output. The optical detection device according to any one of (1) to (11). (13) The correction unit adds corrected event data to the input event data and outputs the event data. The optical detection device according to any one of (1) to (11). (14) an event data generating unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event; and a gradation data generating unit that generates gradation data corresponding to the luminance of the incident light, Identifying a strength of interference caused by an overlap between the generation of the gradient data and the detection of the event; Correcting the event data according to the intensity of the identified interference. Correction method. (15) A computer controls a light detection device having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, Identifying a strength of interference caused by an overlap between the generation of the gradient data and the detection of the event; Correcting the event data according to the intensity of the identified interference. A program for executing a process. [Explanation of symbols]
[0251] 1 photodetector, 10 pixel array, 11 signal line, 12 signal line, 13 pixel, 14 pixel, 15 pixel, 16 pixel, 17 pixel, 18 pixel, 20 access control circuit, 21 signal line, 30 event signal output circuit, 40 gradation signal output circuit, 50 timing control section, 60 event signal processing section, 70 gradation signal processing section, 80 overlapping predicted row detection section, 81 overlapping row prediction section, 90 time stamp generation section, 100 pixel, 110 gradation signal generation section, 111 MOS transistor, 120 event signal generation section, 121 signal line, 130 photoelectric conversion section, 140 current-voltage conversion circuit, 150 differentiation circuit, 160 luminance change detection section, 170 output section, 201 photoelectric conversion section, 202 photoelectric conversion unit, 203 charge storage unit, 204 capacitor, 205 capacitor, 231 constant current circuit, 500 overlapping predicted row, 501 shutter, 502 exposure, 503 readout, 510 frame, 551 area, 601 correction unit, 602 application processor, 612 interference correction unit, 613 interference intensity identification unit, 621 interference correction unit, 651 HOPB area, 652 effective pixel area, 671 table
Claims
1. a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generator that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generator that generates gradation data that is data according to the luminance of the incident light; an interference intensity identifying unit that identifies an intensity of interference occurring due to an overlap between a period of generating the gradation data and a period of detecting the event; a correction unit that corrects the event data in accordance with the intensity of the interference identified by the interference intensity identification unit; A light detection device comprising:
2. the pixel array portion has a light-shielded region, The interference intensity identifying unit counts the number of events detected by the event data generating unit located in the light-shielded area, and identifies the intensity of the interference according to the counted number of events.
2. The optical detection device according to claim 1.
3. The interference intensity identification unit is obtaining a first number of events by counting events in a row of interest that specifies a strength of interference in the pixel array unit; Obtaining a second number of events in a non-interfering row located above the row of interest and free from interference; Obtaining a third number of events in a non-interfering row that is located below the row of interest and has no interference; The intensity of the interference is identified by determining whether or not a predetermined condition is satisfied using the first number of events and the larger number of the second number of events and the third number of events.
2. The optical detection device according to claim 1.
4. obtaining an interference occurrence flag indicating that an overlap predicted row, which is a row in which an overlap between a period of the generation of the gradation data and a period of the detection of the event is predicted, has been detected; The non-interfering row is a row in which the interference occurrence flag is not set.
4. The optical detection device according to claim 3.
5. a table in which information for identifying a frame, a row of interest for identifying an intensity of interference of the pixel array unit, and the intensity of interference are associated with each other; The interference intensity identification unit refers to the table and identifies the intensity of the interference in the target row.
2. The optical detection device according to claim 1.
6. When the interference intensity identifying unit identifies that the intensity of the interference is weak, the correction unit does not perform correction.
2. The optical detection device according to claim 1.
7. When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correcting unit corrects the event data to indicate that no event has occurred in the pixel of interest that is the correction target.
2. The optical detection device according to claim 1.
8. When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correction unit: obtain a first event of a pixel located above the pixel of interest to be corrected and directly above the pixel of interest in a non-interfering row where there is no interference, and obtain a second event of a pixel located below the pixel of interest and directly below the pixel of interest in a non-interfering row where there is no interference; If the first event and the second event match, correcting the event of the pixel of interest to event data indicating that the first event has occurred; If the first event and the second event do not match, the event data is corrected to indicate that no event has occurred in the pixel of interest.
2. The optical detection device according to claim 1.
9. When the interference intensity identifying unit identifies that the intensity of the interference is strong, the correction unit: obtain a first event of a pixel located above the pixel of interest to be corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and obtain a second event of a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference; If the first event and the second event match, correcting the event of the pixel of interest to event data indicating that the first event has occurred; If the first event and the second event do not match, the event data is corrected to indicate that no event has occurred in the pixel of interest.
2. The optical detection device according to claim 1.
10. When the interference intensity identifying unit identifies the intensity of the interference as medium, the correction unit: A pixel located above the pixel of interest to be corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference are set as search pixels; maintain the event data of the event of the pixel of interest if the event of the pixel of interest and at least one of the events of the plurality of search pixels match; If an event that matches the event of the pixel of interest does not occur in any of the plurality of search pixels, the event data is corrected to indicate that no event has occurred in the pixel of interest.
2. The optical detection device according to claim 1.
11. The interference intensity identifying unit outputs the intensity of the interference as a continuous value, The correction unit is converting said continuous values into correction factors; Generate random numbers, If the random number is equal to or smaller than the correction factor, correcting the event data of the pixel of interest being processed; If the random number is not equal to or less than the correction factor, the event data of the pixel of interest being processed is maintained.
2. The optical detection device according to claim 1.
12. The input and output of the correction unit are event data with the same number of bits, When the input event data is corrected, the corrected event data is output.
2. The optical detection device according to claim 1.
13. The correction unit adds corrected event data to the input event data and outputs the event data.
2. The optical detection device according to claim 1.
14. an event data generating unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event; and a gradation data generating unit that generates gradation data corresponding to the luminance of the incident light, Identifying a strength of interference caused by an overlap between the generation of the gradient data and the detection of the event; Correcting the event data according to the identified intensity of the interference. Correction method.
15. A computer controls a light detection device having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, Identifying a strength of interference caused by an overlap between the generation of the gradient data and the detection of the event; Correcting the event data according to the identified intensity of the interference. A program for executing a process.
Citation Information
Patent Citations
Sensor device and read-out method
JP2021129265A
Cited By
System and methods for processing an optical fiber preform
US12358830B2
