Light detection device, correction method, and program

The photodetection device addresses interference challenges by incorporating units for event and gradation data generation and correction, ensuring accurate event signal detection by mitigating interference, thus improving the reliability of event signal processing.

WO2024232284A1PCT designated stage expired Publication Date: 2024-11-14SONY SEMICON SOLUTIONS CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/016193
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-04-25
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Conventional photodetection devices, such as the EVS, face interference issues between grayscale signal generation and event detection, leading to potential errors in event signals due to overlapping processes, which affect the accuracy of event signal detection.

Method used

A photodetection device with an event data generation unit, gradation data generation unit, and interference intensity specification section, along with a correction mechanism to identify and correct event data based on interference intensity, ensuring accurate event signal detection by mitigating interference effects.

Benefits of technology

The solution improves the accuracy of event signal detection by effectively identifying and correcting for interference, thereby enhancing the reliability of event signal processing in photodetection systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016193_14112024_PF_FP_ABST
    Figure JP2024016193_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present technology relates to a light detection device, a correction method, and a program that make it possible to reduce the influence of interference between a gradation signal and an event signal. The light detection device is provided with: a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix pattern, the pixel array unit being provided with an event data generation unit that detects, as an event, a change in the same direction of the luminance of incident light and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data corresponding to the luminance of the incident light; an interference intensity determination unit that determines the intensity of interference that occurs due to an overlap between the periods of the gradation data generation and the event detection; and a correction unit that corrects the event data according to the intensity of the interference determined by the interference intensity determination unit. The present technology can be applied to, for example, a light detection device that acquires a gradation signal and an event signal.
Need to check novelty before this filing date? Find Prior Art

Description

Photodetection device, correction method, and program

[0001] The present technology relates to a light detection device, a correction method, and a program, and, for example, to a light detection device, a correction method, and a program that improve the accuracy of event detection.

[0002] For example, there is a sensor device known as an Event Based Vision Sensor (EVS) that detects an event signal representing a change in the amount of received light for each pixel. Some sensor devices of this type are equipped with both pixels for detecting event signals and pixels for detecting gradation signals so that a gradation signal (a signal representing the intensity of the amount of received light) of a pixel in which an event is detected can be obtained (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-129265

[0004] In the above-described conventional technology, the generation of the grayscale signal and the processing of the event detection overlap in time, which may cause interference due to control signals, etc. If interference occurs, there is a possibility that errors may occur in the event signal. It is desirable to reduce the effects of interference and improve the accuracy of the event signal.

[0005] The present technology has been made in view of such circumstances, and makes it possible to improve the accuracy of event signals.

[0006] An optical detection device according to one aspect of the present technology is an optical detection device including: an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event; a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light; an interference intensity identification unit that identifies an intensity of interference that occurs due to an overlap between a period in which the gradation data is generated and a period in which the event is detected; and a correction unit that corrects the event data according to the intensity of interference identified by the interference intensity identification unit.

[0007] A correction method according to one aspect of the present technology is a correction method in which a photodetector includes an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including: an event data generation unit that detects a change in luminance of incident light in the same direction; and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light; the photodetector identifies an intensity of interference that occurs due to an overlapping period between the generation of the gradation data and the detection of the event, and corrects the event data according to the identified intensity of interference.

[0008] A program according to one aspect of the present technology is a program for causing a computer that controls a photodetector device having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, to execute a process of identifying the intensity of interference that occurs due to an overlap between the generation of the gradation data and the detection of the event, and correcting the event data according to the identified intensity of interference.

[0009] According to one aspect of the present technology, a photodetector, a correction method, and a program include a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generator that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generator that generates gradation data corresponding to the luminance of the incident light. The intensity of interference that occurs due to an overlap between a period in which the gradation data is generated and a period in which the event is detected is identified, and the event data is corrected according to the identified intensity of interference.

[0010] The photodetector may be an independent device or an internal block constituting a single device.

[0011] The program can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0012] 1 is a diagram illustrating an example configuration of a photodetector according to an embodiment of the present disclosure; FIG. 1 is a diagram illustrating an example configuration of a pixel; FIG. 2 is a diagram illustrating an example configuration of a gradation signal generation unit; FIG. 3 is a diagram illustrating an example configuration of an event signal generation unit; FIG. 4 is a circuit diagram illustrating an example configuration of the event signal generation unit; FIG. 5 is a circuit diagram illustrating an example configuration of the event signal generation unit; FIG. 6 is a diagram illustrating an example configuration of a predicted overlapping row detection unit; FIG. 7 is a diagram illustrating an example of event data; FIG. 8 is a diagram illustrating an example processing procedure for overlapping row detection processing; FIG. 9 is a diagram illustrating an example configuration of a correction unit; FIG. 10 is a diagram illustrating another example configuration of the correction unit; FIG. 11 is a diagram illustrating event data before and after correction; FIG. 12 is a diagram illustrating event data before and after correction; FIG. 13 is a flowchart illustrating processing related to correction of event data; FIG. 14 is a flowchart illustrating a first interference intensity identification process; FIG. 15 is a diagram illustrating a HOPB region; FIG. 16 is a diagram illustrating conditions related to identification of interference intensity; FIG. 17 is a flowchart illustrating a second interference intensity identification process; FIG. 18 is a diagram illustrating conditions related to identification of interference intensity; FIG. 19 is a flowchart illustrating a third interference intensity identification process; FIG. 1 is a diagram showing an example of an interference intensity LUT 621. FIG. 2 is a flowchart for explaining a first event data correction process. FIG. 3 is a diagram for explaining removal of a noise event. FIG. 4 is a diagram for explaining a first correction rule. FIG. 5 is a diagram for explaining a second correction rule. FIG. 6 is a diagram for explaining a third correction rule. FIG. 7 is a flowchart for explaining a second event data correction process. FIG. 8 is a diagram showing the relationship between interference intensity and a correction rate. FIG. 9 is a diagram showing an example of the configuration of a PC. FIG. 10 is a block diagram showing an example of the schematic configuration of a vehicle control system. FIG. 11 is an explanatory diagram showing an example of the installation positions of an outside vehicle information detection unit and an imaging unit.

[0013] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described.

[0014] <Configuration of Photodetector> FIG. 1 is a diagram showing a configuration according to one embodiment of a photodetector to which the present technology is applied.

[0015] The photodetector 1 includes a pixel array unit 10, an access control circuit 20, an event signal output circuit 30, a gradation signal output circuit 40, a timing control unit 50, an event signal processing unit 60, a gradation signal processing unit 70, and an overlapping predicted row detection unit 80. The photodetector 1 further includes a timestamp generation unit 90 and an image processing unit 2.

[0016] The pixel array unit 10 is configured by arranging a plurality of pixels 100 in a two-dimensional matrix. Here, the pixels 100 generate grayscale signals that correspond to the luminance of incident light. The pixels 100 also detect changes in the luminance of the incident light in the same direction as described above as events, and generate event signals that are signals based on the detected events. The pixels 100 include a photoelectric conversion unit that performs photoelectric conversion of the incident light, and generate the grayscale signals and event signals described above based on the results of the photoelectric conversion.

[0017] The photoelectric conversion unit may be, for example, a photodiode. Signal lines 11, 12, and 21 are wired to each pixel 100. The pixel 100 generates a grayscale signal and an event signal under the control of a control signal transmitted by the signal line 21. The pixel 100 outputs the generated grayscale signal via the signal line 12 and outputs the generated event signal via the signal line 11. The signal line 21 is arranged in each row of a two-dimensional matrix, and is wired in common to the multiple pixels 100 arranged in one row. The signal lines 11 and 12 are arranged in each column of the two-dimensional matrix, and are wired in common to the multiple pixels 100 arranged in one column.

[0018] 2, each pixel 100 includes a grayscale signal generating unit 110 that generates a grayscale signal and an event signal generating unit 120 that generates an event signal. In the pixel array unit 10, multiple pixels 100 arranged in the same row simultaneously generate grayscale signals. The generation of the grayscale signals is performed sequentially for each row with a shift in timing.

[0019] The access control circuit 20 outputs a control signal to the pixel 100. The access control circuit 20 in the figure outputs a control signal for each row of the two-dimensional matrix of the pixel array unit 10 via a signal line 12.

[0020] The event signal output circuit 30 processes the event signal for each pixel 100 output from the pixel array unit 10 and outputs the processed event signal. The processing by the event signal output circuit 30 corresponds to, for example, the process of converting the analog event signal output from the pixel 100 into a digital data signal.

[0021] The gradation signal output circuit 40 processes the gradation signals generated by the pixels 100 and outputs the processed gradation signals. The gradation signal output circuit 40 in the same figure simultaneously processes gradation signals from a plurality of pixels 100 arranged in one row of the pixel array unit 10. The processing by the gradation signal output circuit 40 corresponds to, for example, a process of converting analog gradation signals output from the pixels 100 into digital signals.

[0022] The event signal processing unit 60 processes the event signal from the event signal output circuit 30. For example, the event signal output circuit 30 converts the event signal into data in a predetermined format. The event signal output circuit 30 outputs the processed event signal as event data to an external device.

[0023] The gradation signal processing unit 70 processes the gradation signal from the gradation signal output circuit 40. For example, the gradation signal processing unit 70 converts the gradation signal into data in a predetermined format. The gradation signal processing unit 70 outputs the processed gradation signal as gradation data.

[0024] The timing control unit 50 controls the pixels 100, etc. The timing control unit 50 controls the pixels 100, etc. by generating and outputting control signals for the pixels 100, etc. This control corresponds to control for causing the gradation signal generation unit 110 to generate a gradation signal, control for causing the event signal generation unit 120 to detect an event, and control for causing the event signal generation unit 120 to generate an event signal based on the detected event.

[0025] Control signals used for these controls are output to the pixels 100 for each row of the pixel array unit 10 via the access control circuit 20. The timing control unit 50 also generates control signals for the event signal processing unit 60 and the gradation signal processing unit 70. The generated control signals are output to the access control circuit 20, the event signal processing unit 60, the gradation signal processing unit 70, and the overlapping predicted row detection unit 80. The timing control unit 50 and the access control circuit 20 are an example of a row control unit.

[0026] The timestamp generating section 90 generates a timestamp and supplies it to the event signal processing section 60 .

[0027] The overlapping predicted row detection unit 80 detects overlapping predicted rows, which are rows where the periods for generating grayscale signals and detecting events are predicted to overlap. As described above, the pixel 100 includes the grayscale signal generation unit 110 and the event signal generation unit 120, and can generate grayscale signals and event signals separately. Therefore, the periods for generating grayscale signals and detecting events may overlap. In this case, interference occurs in the generation of grayscale signals and the detection of events.

[0028] This occurs because the gradation signal generation unit 110 and the event signal generation unit 120 are coupled by stray capacitance or the like, and a change in the control signal of one of the gradation signal generation unit 110 and the event signal generation unit 120 affects the signal level of the other. When this interference occurs, errors occur in the gradation signal and the event signal. Therefore, the above-mentioned overlapping predicted row is detected and used to reduce the influence of interference. The overlapping predicted row detection unit 80 in the same figure generates an interference occurrence flag indicating the occurrence of interference based on the detected overlapping predicted row, and outputs it to the event signal processing unit 60.

[0029] The image processing unit 2 processes grayscale data, which is data of grayscale signals. It is also possible to adopt a configuration including an event data processing unit that processes event data, which is data of event signals.

[0030] <Pixel Configuration> FIG. 2 is a diagram showing an example of the configuration of a pixel. This figure is a block diagram showing an example of the configuration of a pixel 100. The pixel 100 includes a gradation signal generation unit 110 and an event signal generation unit 120. The gradation signal generation unit 110 generates a gradation signal based on a control signal supplied from the access control circuit 20 via a signal line 21. The generated gradation signal is transmitted to the gradation signal output circuit 40 via a signal line 12. The event signal generation unit 120 generates an event signal based on the control signal supplied from the access control circuit 20 via the signal line 21. The generated event signal is transmitted to the event signal output circuit 30 via a signal line 11.

[0031] <Configuration of Gradation Signal Generator> Fig. 3 is a diagram showing an example of the configuration of the gradation signal generator. This diagram is a circuit showing an example of the configuration of the gradation signal generator 110. The gradation signal generator 110 includes a photoelectric conversion unit 201, a charge holding unit 203, and MOS transistors 211 to 214. N-channel MOS transistors can be used for the MOS transistors 211 to 214. The signal line 21 connected to the pixel 100 includes a signal line TRG, a signal line RST, and a signal line SEL. A power supply line Vdd that supplies power is wired to the pixel 100.

[0032] The anode of the photoelectric conversion unit 201 is grounded, and the cathode is connected to the source of the MOS transistor 211. The drain of the MOS transistor 211 is connected to the source of the MOS transistor 212, the gate of the MOS transistor 213, and one end of the charge holding unit 203. The other end of the charge holding unit 203 is grounded. The drain of the MOS transistor 213 is connected to the power supply line Vdd, and the source is connected to the drain of the MOS transistor 214. The source of the MOS transistor 214 is connected to the signal line 12. The signal lines TRG, RST, and SEL are connected to the gates of the MOS transistors 211, 212, and 214, respectively.

[0033] The photoelectric conversion unit 201 is an element that performs photoelectric conversion of incident light, generating and holding electric charges through photoelectric conversion.

[0034] The MOS transistor 211 transfers the charge held in the photoelectric conversion unit 201 to the charge holding unit 203. The MOS transistor 211 is controlled by a control signal transmitted by a signal line TRG.

[0035] The charge holding portion 203 is an element that holds electric charges, and can be configured by a semiconductor region formed on a semiconductor substrate.

[0036] The MOS transistor 212 resets the charge holding unit 203. The MOS transistor 212 is controlled by a control signal transmitted through a signal line RST.

[0037] The MOS transistor 213 is an element that generates a gray scale signal according to the charge held in the charge holding unit 203. The generated gray scale signal is output to a source terminal.

[0038] The MOS transistor 214 is an element that outputs the grayscale signal generated by the MOS transistor 213 to the signal line 12. The MOS transistor 214 is controlled by a control signal transmitted through a signal line SEL.

[0039] <Generation of Gradation Signals> FIG. 4 is a diagram showing an example of generation of a gradation signal. This figure is a timing diagram showing an example of generation of a gradation signal in the gradation signal generation unit 110. In this figure, "SEL" represents a selection signal transmitted by a signal line SEL. "RST" represents a reset signal transmitted by a signal line RST. "TRG" represents a transfer signal transmitted by a signal line TRG. The value "1" in the binarized waveform of these control signals represents an ON signal, which is a signal that turns on a MOS transistor. The dashed lines in this figure represent a 0V level. As shown in this figure, the gradation signal is generated by a shutter 501, an exposure 502, and a readout 503. In the initial state, the selection signal, the reset signal, and the transfer signal have values ​​of "0," "1," and "0," respectively.

[0040] The shutter 501 is a period corresponding to an electronic shutter, during which the reset signal and transfer signal have a value of "1" and the MOS transistors 211 and 212 are conductive, thereby discharging and resetting the charges in the photoelectric conversion unit 201 and the charge holding unit 203.

[0041] Exposure 502 is a period in which the transfer signal has a value of “0” and charges generated by photoelectric conversion in the photoelectric conversion unit 201 are accumulated in the photoelectric conversion unit 201 .

[0042] Readout 503 is a period during which the selection signal has a value of "1", the reset signal has a value of "0", and the gradation signal generated by the MOS transistor 111 is output to the signal line 12. During this period, the transfer signal has a value of "1", and the charge in the photoelectric conversion unit 201 is transferred to the charge holding unit 203. The MOS transistor 111 generates a gradation signal according to the charge in the charge holding unit 203 and outputs it to the signal line 12.

[0043] In this way, a grayscale signal is generated by the three periods of shutter 501, exposure 502, and readout 503, and is output from the grayscale signal generation unit 110. As described above, the grayscale signal is generated for each row. At this time, the shutter 501, exposure 502, and readout 503 are applied sequentially while shifting the timing for each row. This will be described later with reference to FIG. 9.

[0044] <Configuration of Event Signal Generator> Fig. 5 is a diagram showing an example configuration of the event signal generator. The figure is a block diagram showing an example configuration of the event signal generator 120. The event signal generator 120 in the figure includes a photoelectric conversion unit 130, a current-voltage conversion circuit 140, a differentiation circuit 150, a luminance change detection unit 160, and an output unit 170.

[0045] The photoelectric conversion unit 130 performs photoelectric conversion of incident light, similar to the photoelectric conversion unit 201. The photoelectric conversion unit 130 can be configured by a photodiode.

[0046] The current-voltage conversion circuit 140 converts the photocurrent from the photoelectric conversion unit 130 into a voltage signal. During this conversion, the current-voltage conversion circuit 140 performs logarithmic compression on the voltage signal. The converted voltage signal is output to the differentiation circuit 150. The configuration of the current-voltage conversion circuit 140 will be described in detail later.

[0047] The differentiating circuit 150 extracts the change in the voltage signal output from the current-voltage conversion circuit 140 and integrates the extracted change to generate a signal corresponding to the amount of change in the voltage signal. This signal corresponds to a signal corresponding to a change in the luminance of the incident light. This signal is referred to as an optical signal. The differentiating circuit 150 outputs the generated optical signal to the luminance change detection unit 160 via a signal line 121. The differentiating circuit 150 also receives an input of a control signal from the access control circuit 20. This control signal is a signal that resets the circuit that detects the amount of change in the voltage signal. The configuration of the differentiating circuit 150 will be described in detail later.

[0048] The luminance change detection unit 160 detects changes in the luminance of incident light. The luminance change detection unit 160 in the same figure detects changes in the optical signal output from the differentiation circuit 150 based on a threshold value. That is, when the change in the optical signal exceeds the threshold value, the change in the optical signal is detected as an event. Here, an event in the direction in which the optical signal increases is called an ON event, and an event in the direction in which the optical signal decreases is called an OFF event. The luminance change detection unit 160 detects ON events and OFF events using the voltages of the ON event detection signal and OFF event detection signal supplied from the access control circuit 20 as threshold values. The detection results are output to the output unit 170. The configuration of the luminance change detection unit 160 will be described in detail later.

[0049] The output section 170 outputs the ON event and OFF event detected by the brightness change detection section 160 based on the control signal from the access control circuit 20 as an event signal.

[0050] <Circuit Configuration of Event Signal Generator> Figures 6 and 7 are circuit diagrams showing examples of the configuration of the event signal generator. Figure 6 is a circuit diagram showing examples of the configuration of the current-voltage conversion circuit 140 and the differentiation circuit 150. Note that this figure also shows a photoelectric conversion unit 202. Figure 7 is a circuit diagram showing examples of the configuration of the luminance change detection unit 160 and the output unit 170.

[0051] The current-voltage conversion circuit 140 in the figure includes MOS transistors 215 to 217. In the figure, Vdd represents a power supply line Vdd that supplies power. Vb1 represents a signal line Vb1 that supplies a bias voltage. The MOS transistors 215 and 217 can be n-channel MOS transistors. The MOS transistor 216 can be a p-channel MOS transistor.

[0052] The anode of the photoelectric conversion unit 202 is grounded, and the cathode is connected to the source of the MOS transistor 215 and the gate of the MOS transistor 217. The sources of the MOS transistors 215 and 216 are connected to the power supply line Vdd, and the gate of the MOS transistor 216 is connected to a signal line Vb1. The source of the MOS transistor 217 is grounded, and the drain is connected to the gate of the MOS transistor 215, the drain of the MOS transistor 216, and an output signal line of the current-voltage conversion circuit 140. One end of the capacitor of the differentiation circuit 150 is connected to this output signal line.

[0053] The MOS transistor 215 is a MOS transistor that supplies a current to the photoelectric conversion unit 202. A sink current (photocurrent) corresponding to incident light flows through the photoelectric conversion unit 202. The MOS transistor 215 supplies this sink current. At this time, the gate of the MOS transistor 215 is driven by the output voltage of a MOS transistor 217 (described later), and outputs a source current equal to the sink current of the photoelectric conversion unit 202. Because the gate-source voltage Vgs of the MOS transistor is a voltage corresponding to the source current, the source voltage of the MOS transistor 215 is a voltage corresponding to the current of the photoelectric conversion unit 202. As a result, the photocurrent of the photoelectric conversion unit 202 is converted into a voltage signal.

[0054] MOS transistor 217 is a MOS transistor that amplifies the source voltage of MOS transistor 215. MOS transistor 216 constitutes a constant current load for MOS transistor 217. An amplified voltage signal is output to the drain of MOS transistor 217. This voltage signal is output to differentiation circuit 150 and also fed back to the gate of MOS transistor 215. When Vgs of MOS transistor 215 is equal to or lower than the threshold voltage, the source current changes exponentially with respect to changes in Vgs. Therefore, the output voltage of MOS transistor 217 that is fed back to the gate of MOS transistor 215 is a voltage signal that is logarithmically compressed photocurrent of photoelectric conversion unit 202, which is equal to the source current of MOS transistor 215.

[0055] <Configuration of Differentiating Circuit> The differentiating circuit 150 in the figure includes capacitors 204 and 205, MOS transistors 218 and 219, and a constant current circuit 231. The MOS transistors 218 and 219 can be p-channel MOS transistors.

[0056] As described above, one end of capacitor 204 is connected to the output of current-voltage conversion circuit 140, and the other end of capacitor 204 is connected to the gate of MOS transistor 218, the drain of MOS transistor 219, and one end of capacitor 205. The other end of capacitor 205 is connected to the drain of MOS transistor 218, the drain of MOS transistor 219, the sink-side terminal of constant current circuit 231, and signal line 121. The source of MOS transistor 218 is connected to power supply line Vdd, and the gate of MOS transistor 219 is connected to signal line AZ. The sink-side terminal of constant current circuit 231 is grounded.

[0057] Capacitor 204 corresponds to a coupling capacitor. This capacitor 204 blocks the DC component of the output voltage of current-voltage conversion circuit 140 and passes only the AC component. Furthermore, a current based on a change in the output voltage of current-voltage conversion circuit 140 is supplied to the gate of MOS transistor 218 via capacitor 204. The AC component of the output voltage of current-voltage conversion circuit 140 corresponds to a change in photocurrent.

[0058] The MOS transistor 218 and the constant current circuit 231 form an inverting amplifier circuit. The change in the output voltage of the current-voltage conversion circuit 140 is input to the gate of the MOS transistor 218 via the capacitor 204, inverted and amplified by the MOS transistor 218, and output to the drain. As a result, a current based on the change in the output voltage of the current-voltage conversion circuit 140 flows through the capacitor 205, and the capacitor 205 is charged and discharged. In other words, the change in the output voltage of the current-voltage conversion circuit 140 is accumulated (integrated). An optical signal, which is a signal corresponding to the amount of change in the voltage signal output by the current-voltage conversion circuit 140, is output to the signal line 121.

[0059] The MOS transistor 219 resets the differentiation circuit 150. By making this MOS transistor 219 conductive, both ends of the capacitor 205 are shorted. The integrated change in the output voltage of the current-voltage conversion circuit 140 is discharged and reset. This resetting causes the output voltage of the differentiation circuit 150 to become, for example, the voltage at the midpoint between the power supply line Vdd and the ground line. This resetting is controlled by an AZ control signal transmitted by a signal line AZ. Hereinafter, the resetting of the differentiation circuit 150 will be referred to as the AZ operation.

[0060] 7, the brightness change detection unit 160 includes MOS transistors 220 to 223. P-channel MOS transistors can be used for the MOS transistors 220 and 222. N-channel MOS transistors can be used for the MOS transistors 221 and 223. Signal lines ON and OFF from the access control circuit 20 are connected to the brightness change detection unit 160. The signal line ON is a signal line that transmits an ON-event detection signal. The signal line OFF is a signal line that transmits an OFF-event detection signal.

[0061] The signal line 121 is connected to the gate of the MOS transistor 220 and the gate of the MOS transistor 222. The source of the MOS transistor 220 is connected to the power supply line Vdd, and the drain is connected to the drain of the MOS transistor 221 and the gate of the MOS transistor 225 in the output section 170. The gate of the MOS transistor 221 is connected to the signal line ON, and the source is grounded. The source of the MOS transistor 222 is connected to the power supply line Vdd, and the drain is connected to the drain of the MOS transistor 223 and the gate of the MOS transistor 227 in the output section 170. The gate of the MOS transistor 223 is connected to the signal line OFF, and the source is grounded.

[0062] The circuit of MOS transistors 220 and 221 constitutes a comparison circuit. The output of this comparison circuit changes depending on the magnitude relationship between the drain current on the sink side of MOS transistor 221 and the drain current on the source side of MOS transistor 220. When the output voltage of differentiation circuit 150 is lower than a threshold based on the voltage of the on-event detection signal, specifically, a voltage obtained by subtracting this threshold voltage from the power supply voltage Vdd, the source current of MOS transistor 220 becomes smaller than the sink current of MOS transistor 221.

[0063] Therefore, the output voltage becomes L level. On the other hand, when the output voltage of the differentiation circuit 150 becomes higher than the threshold voltage (the voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd), the sink current of the MOS transistor 221 becomes smaller than the source current of the MOS transistor 220. Therefore, the output voltage shifts to H level.

[0064] In this way, the comparator circuit made up of MOS transistors 220 and 221 compares the output voltage of the differentiating circuit 150 with the threshold voltage of the on-event detection signal to detect an on-event, which is a change in the direction of increasing the luminance of incident light. Note that when the on-event detection signal is at a voltage higher than the threshold voltage, for example, the power supply voltage of the power supply line Vdd, the output of the comparator is always at the L level. In other words, by applying the threshold voltage as the on-event detection signal, on-events can be detected.

[0065] The circuit of MOS transistors 222 and 223 also constitutes a comparison circuit. When the output voltage of differentiating circuit 150 is lower than a threshold based on the voltage of the off-event detection signal, specifically, the voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd, the output voltage becomes L level. On the other hand, when the output voltage of differentiating circuit 150 becomes higher than the threshold voltage (the voltage obtained by subtracting the threshold voltage from the power supply voltage Vdd), the output voltage transitions to H level.

[0066] By setting the threshold voltage of the off-event detection signal to a lower voltage than the threshold voltage of the on-event detection signal, the comparison circuit of the MOS transistors 222 and 223 detects an off-event, which is a change in the direction of decreasing the luminance of incident light. Note that when the off-event detection signal is at a voltage lower than the threshold voltage, for example, the ground potential, the output of the comparator is always at the H level. In other words, by applying the threshold voltage as the off-event detection signal, off-events can be detected.

[0067] <Configuration of Output Section> The output section 170 includes MOS transistors 224 to 227. N-channel MOS transistors can be used for the MOS transistors 224 to 227. The drain of the MOS transistor 224 is connected to one side of the signal line 11, and the drain of the MOS transistor 226 is connected to the other side of the signal line 11. The gates of the MOS transistors 224 and 226 are commonly connected to a signal line OUT. The source of the MOS transistor 224 is connected to the drain of the MOS transistor 225, and the source of the MOS transistor 225 is grounded. The source of the MOS transistor 226 is connected to the drain of the MOS transistor 227, and the source of the MOS transistor 227 is grounded.

[0068] When an event read signal is input to the signal line OUT, the MOS transistors 224 and 226 become conductive. As a result, the drain voltages of the MOS transistors 225 and 227 are output to the signal line 11. The on-event detection signal and the off-event detection signal of the luminance change detection unit 160 are applied to the gates of the MOS transistors 225 and 227, and therefore an event signal including the on-event signal and the off-event signal is output to the signal line 11.

[0069] In this way, the gray scale signal generating unit 110 and the event signal generating unit 120 respectively generate the gray scale signal and the event signal. When generating the gray scale signal, the timing control unit 50 generates a gray scale address signal which is an address signal of the row for which the gray scale signal is to be generated, and outputs the generated address signal to the access control circuit 20.

[0070] The access control circuit 20 outputs a selection signal, a reset signal, and a transfer signal to the pixels 100 in a row based on the gradation address signal. Meanwhile, when generating an event signal, the timing control unit 50 outputs an on-event detection signal, an off-event detection signal, an AZ control signal, an output signal, and an address signal for an Event-based Vision Sensor (EVS), which is an address signal when reading out the event signal, to the access control circuit 20. The access control circuit 20 sequentially outputs the on-event detection signal, the off-event detection signal, and the AZ control signal to all pixels 100 in the pixel array unit 10. Thereafter, the output signal is sequentially output for each row of the pixel array unit 10, thereby outputting (reading out) the event signal. This process will be described with reference to FIG. 8 .

[0071] 2 illustrates an example in which the gradation signal generation unit 110 and the event signal generation unit 120 each include a photoelectric conversion unit. Alternatively, the pixel 100 may be configured in such a way that the gradation signal generation unit 110 and the event signal generation unit 120 share one photoelectric conversion unit.

[0072] <Generation of Grayscale Signals and Event Signals> Fig. 8 is a diagram showing an example of generation of grayscale signals and event signals. The figure is a timing diagram showing an example of generation of grayscale signals and event signals. The horizontal axis of the figure represents time, and the vertical axis represents row addresses. Note that the figure shows an example of generation of grayscale signals during a frame period, which is a period during which grayscale signals for all rows of the pixel array unit 10 are generated, and generation of event signals during that period.

[0073] The rectangles in the figure represent the periods of shutter 501, exposure 502, and readout 503 described in Figure 4. As shown in the figure, shutter 501, exposure 502, and readout 503 are performed sequentially while shifting the timing for each row, thereby generating one frame of grayscale signal. This method of generating grayscale signals is called a rolling shutter method.

[0074] The solid lines in the figure represent the timing of on-event detection ("ON" in the figure), off-event detection ("OFF" in the figure), AZ operation ("AZ" in the figure), and event signal output (readout, "RD" in the figure). As shown in the figure, on-event detection, off-event detection, and AZ operation are performed simultaneously for the pixels 100 in all rows.

[0075] As a result, overlap occurs between the row where the grayscale signal is generated and the row where the on-event detection, off-event detection, and AZ operation are performed. The aforementioned interference occurs in the pixels 100 in such a row. Therefore, a predicted overlap row is detected, which is a row where overlap is predicted during the period of grayscale signal generation and event detection (for example, the period from on-event detection and off-event detection to AZ operation). A predicted overlap row 500 is shown in the figure.

[0076] <Configuration of Overlapping Predicted Line Detector> Fig. 9 is a diagram showing an example of the configuration of the overlapping predicted line detector. The figure is a block diagram showing an example of the configuration of the overlapping predicted line detector 80. The overlapping predicted line detector 80 in the figure includes an overlapping line predictor 81, a memory unit (#1) 82, and a memory unit (#2) 83.

[0077] The overlapping row prediction unit 81 detects overlapping predicted rows based on control signals from the timing control unit 50. The overlapping row prediction unit 81 receives inputs of a gradation address signal, an on-event detection signal, an off-event detection signal, an AZ control signal, an output signal, and an EVS address signal. The overlapping row prediction unit 81 detects overlapping predicted rows from these control signals. For example, it can detect overlapping predicted rows from the gradation address signal when the on-event detection signal, the off-event detection signal, and the AZ control signal are input. In this case, an error due to interference occurs in either the gradation signal or the event signal.

[0078] The overlapping row prediction unit 81 can also detect the overlapping predicted row from the gradation address signal and the EVS address signal. In this case, the overlapping predicted row can be detected before generating an event signal or the like, and therefore the generation of the gradation signal or the event signal for the overlapping predicted row can be avoided, thereby reducing the influence of interference.

[0079] The overlapping row prediction unit 81 stores information about the overlapping predicted rows detected from the on-event detection signal, the off-event detection signal, and the gradation address signal in a memory unit (#1) 82. The overlapping row prediction unit 81 generates an interference occurrence flag based on the detected information about the overlapping predicted rows, and outputs it to the event signal processing unit 60.

[0080] The overlapping row prediction unit 81 stores information about the overlapping predicted rows detected from the AZ control signal and the gradation address signal in a memory unit (#2) 83. If the AZ operation is affected by interference, the event signal for the next frame period will be affected. Therefore, the information about the overlapping predicted rows in this case is stored in a memory unit (#2) 83 different from the memory unit (#1) 82. When moving to the next frame period, the overlapping row prediction unit 81 generates an interference occurrence flag based on the information about the overlapping predicted rows stored in the memory unit (#2) 83 and outputs it to the event signal processing unit 60.

[0081] <Event Data> FIG. 10 is a diagram showing an example of event data. This diagram shows an example of event data generated by the event signal processing unit 60. This diagram shows a frame 510 of event data for one frame period. In this diagram, "FS" is a block indicating the start of the frame. "FE" is a block indicating the end of the frame. "PH" is a block indicating the header of the packet. "PF" is a block indicating the footer of the packet. "EBD" is a block indicating embedded data. "Event" is a block in which an event signal for each row is held. This "Event" is placed between "PH" and "PF". Note that an area 551 in this diagram represents the area of ​​data for the row corresponding to the overlapping predicted row 500.

[0082] In the figure, an "Event" corresponding to area 551 stores an event signal affected by interference. Therefore, an interference occurrence flag is added to the "PH" of the "Event." The thick-lined rectangle in the figure represents the "PH" to which the interference occurrence flag is added. This makes it possible to identify data affected by interference. The identified data can be removed in a device that uses the event data. For example, the event signal processing unit 60 adds mask information to the "PH" of a row based on the interference occurrence flag output from the overlapping predicted row detection unit 80. The interference occurrence flag can also be placed next to the "PF." The interference occurrence flag is an example of information indicating an overlapping predicted row.

[0083] <Duplicate Line Detection Processing> FIG. 11 shows an example of the processing procedure for the duplicate line detection processing. First, the duplicate line prediction unit 81 determines whether a vertical synchronization signal indicating the start of a frame period has been input (step S100). If a vertical synchronization signal has not been input (step S110, No), the duplicate line prediction unit 81 proceeds to processing in step S103. On the other hand, if a vertical synchronization signal has been input (step S100, Yes), the duplicate line prediction unit 81 initializes the storage unit (#1) 82 (step S101). Next, the duplicate line prediction unit 81 transfers the information in the storage unit (#2) to the storage unit (#1) (step S102) and proceeds to processing in step S103. In step S103, the duplicate line prediction unit 81 initializes the storage unit (#2) 83 (step S103).

[0084] Next, the overlapping line prediction unit 81 determines whether an event is being detected (step S104). If an event is being detected (step S104, Yes), the overlapping line prediction unit 81 stores the line generated by the gradation signal in the storage unit (#1) 82 (step S105) and proceeds to step S106. On the other hand, if an event is not being detected (step S104, No), the overlapping line prediction unit 81 determines whether AZ operation is being performed (step S107). If AZ operation is not being performed (step S107, No), the overlapping line prediction unit 81 proceeds to step S106. On the other hand, if AZ operation is being performed (step S107, Yes), the overlapping line prediction unit 81 stores the line generated by the gradation signal in the storage unit (#2) 82 (step S108) and proceeds to step S106.

[0085] In step S106, the duplicated line prediction unit 81 determines whether an event signal is being read (step S106). If an event signal is not being read (step S106, No), the duplicated line prediction unit 81 ends the process. On the other hand, if an event signal is being read (step S106, Yes), the duplicated line prediction unit 81 determines whether line data is stored in the memory unit (#1) 82 (step S109). If line data is not stored in the memory unit (#1) 82 (step S109, No), the duplicated line prediction unit 81 ends the process.

[0086] On the other hand, if the row data is stored in the memory unit (#1) 82 (step S109, Yes), the overlapping row predictor 81 outputs an interference occurrence flag to the event signal processor 60 (step S110). Next, the event signal processor 60 adds interference information to the output data (step S111).

[0087] In this way, the photodetector 1 detects predicted overlap rows, which are rows where the generation of grayscale signals and the detection of events are predicted to overlap, and adds information indicating the occurrence of interference to the event data including the event signals of those rows. This makes it possible to avoid using event signals affected by interference and reduce the effects of interference.

[0088] <Configuration Related to Correction> The above-described processing can detect event data that may have been affected by interference and output an interference occurrence flag. For example, by correcting the event data for which an interference occurrence flag has been output, the influence of interference can be reduced and the accuracy of the event data can be improved.

[0089] It is also possible to uniformly correct event data for which an interference occurrence flag has been output. In this case, the same correction is performed on event data with strong interference intensity and event data with weak interference intensity. There is a possibility that the correction will be weak for event data with strong interference intensity and strong for event data with weak interference intensity, resulting in inappropriate correction. Therefore, we will explain how correction according to interference intensity can be performed to enable appropriate correction.

[0090] Fig. 12 is a diagram showing an example configuration of a correction unit 601 that performs correction according to interference intensity. The correction unit 601 shown in Fig. 12 includes an event data acquisition unit 611, an interference correction unit 621, and an interference intensity identification unit 613. The correction unit 601 shown in Fig. 12 is provided within a sensor, for example, within the photodetector 1 (Fig. 1). The correction unit 601 can be provided within the event signal processing unit 60 (Fig. 1), or can be provided outside the event signal processing unit 60 at a position where it processes event data from the event signal processing unit 60.

[0091] An event data acquisition unit 611 in the correction unit 601 acquires event data and supplies the acquired event data to an interference correction unit 612 and an interference intensity identification unit 613. The interference correction unit 612 corrects the event data based on information related to interference intensity supplied from the interference intensity identification unit 613, and outputs the corrected event data to, for example, a downstream application 602.

[0092] 13 is a diagram showing another example of the configuration of the correction unit 601 that performs correction according to interference intensity. The correction unit 601 may be configured to be provided in the application (application processor) 602. In this case, for example, an interference correction unit 612 and / or an interference intensity identification unit 613 are provided in the application 602 that acquires and processes event data output from the event signal processing unit 60 (FIG. 1).

[0093] Fig. 14 is a diagram showing yet another example configuration of the correction unit 601 that performs correction according to interference intensity. The correction unit 601 shown in Fig. 14 differs from the correction unit 601 shown in Fig. 13 in that event data and an interference occurrence flag are input to an interference correction unit 612, but has a similar configuration in other respects. When the correction unit 601 shown in Fig. 14 is provided, for example, in the photodetector 1 (Fig. 1), it can also be configured to include an event data acquisition unit 611. The correction unit 601 shown in Fig. 14 may be provided in the sensor, or may be provided in an application processor 602 outside the sensor.

[0094] Fig. 15 is a diagram showing yet another example configuration of the correction unit 601 that performs correction according to interference intensity. The correction unit 601 shown in Fig. 15 is similar to the correction unit 601 shown in Fig. 14 in that event data and an interference occurrence flag are input to an interference correction unit 612, but differs in that it includes an interference intensity look-up table (LTU) 321.

[0095] 15 may be provided in the sensor, or may be provided in an application processor 602 outside the sensor. As will be described later, the interference intensity LUT 621 is a table in which a predetermined row in one frame is associated with the interference intensity, and the interference correction unit 612 refers to the interference intensity LUT 621 and performs correction according to the interference intensity.

[0096] In a configuration provided with the interference intensity LUT 621, it is also possible to adopt a configuration in which the interference occurrence flag is not input, or in which the interference occurrence flag is not generated, and the portion related to the generation of the interference occurrence flag is omitted. In other words, the interference intensity LUT 621 can include the interference occurrence flag, i.e., information on pixels where interference occurs, and in this way, it is possible to omit processing and functions related to the interference occurrence flag.

[0097] <Regarding Event Data Before and After Correction> Event data before and after correction will be described with reference to Fig. 16. The upper part of Fig. 16 shows an example of event data in a predetermined row before correction, and the lower part of Fig. 16 shows an example of event data in a predetermined row after correction.

[0098] 16, one square represents one pixel (one event signal generator 120). P indicates that the event data is an on-event, and is represented as "01" as the event data. N indicates that the event data is an off-event, and is represented as "10" as the event data. A blank space indicates that there is no event, and is represented as "00" as the event data. The numbers 1 to 20 shown at the top of the figure are numbers added for explanation purposes.

[0099] Pixel 1 has no event and the event data is "00". Pixel 2 has an on-event and the event data is "01". Pixel 3 has no event and the event data is "00". Pixel 4 has an off-event and the event data is "10". Pixel 5 has no event and the event data is "00". Pixels 6 and 7 have on-events and the event data is "01".

[0100] Pixels 8 to 11 have no event and the event data is "00". Pixels 12 to 14 have an off event and the event data is "10". Pixels 15 and 16 have no event and the event data is "00". Pixel 17 has an on event and the event data is "01". Pixels 18 to 20 have no event and the event data is "00".

[0101] When the correction unit 601 acquires such event data, the interference correction unit 612 performs correction according to the interference intensity, as will be described in detail later, and outputs the corrected event data as shown in the lower part of Fig. 16. The following describes the parts of the event data that have been replaced by the correction.

[0102] Pixels 2, 7, and 17 had event data of an on event, but are corrected to "00" data indicating no event through correction. Pixels 12 and 13 had event data of an off event, but are corrected to "00" data indicating no event through correction. In this way, correction is performed on pixels where an on event or an off event has occurred, and the pixels may be corrected to "00" data indicating no event.

[0103] Pixel 9 had event data without an event, but is corrected to on-event event data "01" through correction. Pixel 10 had event data without an event, but is corrected to off-event event data "10" through correction. In this way, by performing correction on a pixel detected as having no event, it may be corrected to a pixel in which an on-event or an off-event has occurred.

[0104] A data string such as that shown in the lower part of Fig. 16 is output to a subsequent stage as corrected event data. In this example, the event data for one pixel before correction is represented by two bits, and the event data after correction is also represented by two bits. As shown in Fig. 17, the two-bit event data before correction may be converted to four-bit event data after correction and output to a subsequent stage.

[0105] 17 is the same as the event data before correction shown in the upper part of Fig. 16, and represents the event data before correction input to the interference correction unit 612. Pixel 1, pixel 3, pixel 5, pixel 8, pixel 11, pixel 15, pixel 16, and pixels 18 to 20 are event data without an event, and remain event data without an event even after correction.

[0106] These pixels are pixels that have the same event data before and after correction and are treated as event data that does not need to be corrected, or pixels that have become the same event data as a result of correction. In such cases, the 2-bit event data "00" before correction is concatenated with the 2-bit event data "00" after correction, in other words, the event data before correction is converted into 4-bit event data "0000" that is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.

[0107] Pixels 4 and 14 are OFF-event event data, and remain OFF-event event data even after correction. These pixels are the same event data before and after correction, and are pixels that were treated as event data that did not require correction, or pixels that became the same event data as a result of correction.

[0108] In such a case, the corrected 2-bit event data "10" is concatenated to the pre-correction 2-bit event data "10." In other words, the pre-correction event data is converted into 4-bit event data "1010" that is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.

[0109] Pixel 6 is an ON event, and remains an ON event even after correction. This pixel has the same event data before and after correction, and is a pixel that was treated as event data that does not need to be corrected, or a pixel that became the same event data as a result of correction.

[0110] In such a case, the corrected 2-bit event data "01" is concatenated to the pre-correction 2-bit event data "01." In other words, the pre-correction event data is converted into 4-bit event data "0101" that is repeated twice, and this converted event data is output to the subsequent stage as corrected event data.

[0111] Pixels 2, 7, and 17 have on-event event data, but are corrected to no-event event data after correction. In such cases, the pre-correction 2-bit event data "01" is concatenated with the corrected 2-bit event data "00," converting them into 4-bit event data "0100," and this converted event data is output to the subsequent stage as corrected event data.

[0112] Pixels 12 and 13 have off-event event data, but are corrected to no-event event data after correction. In this case, the pre-correction 2-bit event data "10" is concatenated with the corrected 2-bit event data "00," converting them into 4-bit event data "1000." This converted event data is then output to the subsequent stage as corrected event data.

[0113] Pixel 9 is a pixel with no-event event data that has been corrected to on-event event data after correction. In this case, the 2-bit event data "00" before correction is concatenated with the 2-bit event data "01" after correction, converting it to 4-bit event data "0001," and this converted event data is output to the subsequent stage as corrected event data.

[0114] Pixel 10 has no-event event data, but after correction it has been corrected to off-event event data. In this case, the pre-correction 2-bit event data "00" is concatenated with the post-correction 2-bit event data "10," converting it into 4-bit event data "0010." This converted event data is then output to the subsequent stage as corrected event data.

[0115] 17 is output as corrected event data to the subsequent stage. For example, if the subsequent stage is an application 602, the application 602 may perform processing using the corrected 4-bit event data, or may perform processing using the latter 2 bits of event data (the 2 bits added after the correction processing).

[0116] When the application 602 uses the corrected 4-bit event data, it can be configured to compare the first 2 bits (event data before correction) with the last 2 bits (event data after correction) and perform different processing depending on whether they are the same, i.e., no correction has been made, or different, i.e., whether correction has been made.

[0117] <Processing of Correction Unit> The processing of the correction unit 601 will be described with reference to the flowchart of Fig. 18. Here, the description will continue using the processing of the correction unit 601 in the configuration shown in Fig. 12 as an example, but the basic processing can be performed in the same way in any of the correction units 601 shown in Figs. 13 to 15.

[0118] In step S201, the event data acquisition unit 611 acquires event data.

[0119] In step S202, the interference intensity identifying unit 613 identifies the interference intensity. The process in step S202 will be described later with reference to FIGS.

[0120] In step S203, the interference correction unit 612 performs correction on the event data according to the interference intensity to generate corrected event data. The processing in step S203 will be described later with reference to FIGS.

[0121] In step S204, the corrected event data is output to a subsequent processing unit.

[0122] <Regarding the First Interference Intensity Identification Process> The interference intensity identification process in step S202 will be described with reference to the flowchart of Fig. 19. The process based on the flowchart shown in Fig. 19 is an interference intensity identification process (referred to as the first interference intensity identification process) performed in a photodetector 1 including a pixel array unit 10 having an optical black region, as shown in Fig. 20.

[0123] In step S221, the number of events in the HOPB (Horizontal Optical Black) region of the target row is counted. The HOPB region will now be described with reference to FIG. 20 . The pixel region of the pixel array unit 10 is configured to include an effective pixel region 652 (the visible region in the figure) that actually receives light, amplifies signal charges generated by photoelectric conversion, and reads them out to a column signal processing circuit, and a HOPB region 651 that outputs optical black, which serves as a reference for the black level.

[0124] 20 shows an example in which an optical black area is provided in the vertical direction (the longitudinal direction in the figure), and this area is referred to as a HOPB area 651. The optical black area may also be provided in the horizontal direction. While FIG. 20 shows an example in which the HOPB area 651 is provided on both the left and right sides of the figure, a configuration in which the HOPB area 651 is provided only on the left side or only on the right side is also possible.

[0125] In step S221 (FIG. 19), events occurring in a row of interest in the HOPB area 651 are counted. The HOPB area 651 is a light-shielded area where there is no change in illuminance and where it is expected that no events will occur. If interference occurs, there is a possibility that an event will occur in such an area. If interference is strong, the number of events occurring in the HOPB area 651 will also increase. This fact is used to detect the interference intensity.

[0126] In step S221, the number of events occurring in the HOPB area 651 of the row for which interference intensity is to be measured (the row of interest) is counted. In step S222, the number of counted events is converted into interference intensity. For example, a table such as that shown in FIG. 21 is provided, and this table is referenced to convert the number of events into interference intensity.

[0127] 21, table 671 is a table in which the number of counted events is associated with interference intensity. Table 671 shown in Fig. 21 shows, for example, a case in which the number of event data in one row of HOPB area 651 is 500. This is just one example, and the number of event data in HOPB area 651 may be set to a value other than 500, and the values ​​described below may of course be values ​​other than those shown below.

[0128] If the number of counted events is "10 or less", the interference intensity is converted to "weak". If the number of counted events is "11 or more and 100 or less", the interference intensity is converted to "medium". If the number of counted events is "101 or more", the interference intensity is converted to "strong".

[0129] Note that a configuration other than the configuration in which the table 671 is provided and referenced to convert the number of events into interference intensity may be used, for example, a configuration in which interference intensity is calculated from the number of events using a predetermined arithmetic expression.

[0130] The number of events may be counted separately for ON events and OFF events, or may be counted together without distinguishing between ON events and OFF events.

[0131] On events and off events may be counted separately, and the interference strength of the on events and the interference strength of the off events may be estimated separately, or on events and off events may be counted together without distinction, and the interference strength may be estimated using the count value obtained by counting them together.

[0132] Here, the interference intensity is classified into three categories, namely, strong, medium, and weak, in other words, an example is shown in which the interference intensity is classified discretely, but it may also be estimated and output as a continuous value, for example, interference intensity = 0.8.

[0133] In step S 223 , the interference intensity converted from the number of events in step S 222 is output from the interference intensity identifying unit 613 to the interference correcting unit 612 .

[0134] In this way, when a light-shielded area is provided in the pixel array unit 10, the interference intensity identification unit 613 can be configured to identify the interference intensity using event data obtained from within this light-shielded area.

[0135] <Second Interference Intensity Identification Process> The interference intensity identification process (hereinafter referred to as second interference intensity identification process) in step S202 will be described with reference to the flowchart of FIG.

[0136] The first interference intensity identification process described with reference to Fig. 19 has been described using an example of a process for counting the number of events and identifying the interference intensity using the HOPB region 651 of the pixel array unit 10. The second interference intensity identification process described with reference to Fig. 22 has been described using an example of a process for counting the number of events and identifying the interference intensity using the effective pixel region 652 (Fig. 20) of the pixel array unit 10.

[0137] In step S241, the number of events occurring in the row of interest within the effective pixel area 652 is counted. The value of the counted number of events is set to a value a.

[0138] In step S242, the larger value of the number of events between the non-interfering rows above and below the row of interest is obtained. The obtained value is designated as value b. In step S242, the number of events obtained from the row above the row of interest in the effective pixel region 652 and determined not to be interfering with each other, and the number of events obtained from the row below the row of interest in the effective pixel region 652 and determined not to be interfering with each other are obtained and compared, and the larger value is obtained as value b.

[0139] Although the explanation will be continued here by taking as an example a case where the larger value of the number of events in the non-interfering rows above and below the row of interest is obtained and used, other values ​​may also be used. For example, the average value of the number of events in the non-interfering rows above and below the row of interest may be calculated and used.

[0140] An example will be given in which the number of events in the non-interfering rows above and below the row of interest is used, but it is also possible to configure the system to use either the number of events in the non-interfering row above the row of interest or the number of events in the non-interfering row below the row of interest.

[0141] In the effective pixel region 652, normal events also exist; in other words, unlike the HOPB region 651, events that occur in a normal state where no interference occurs also exist. Therefore, the intensity of interference cannot be estimated simply by counting the number of events, as with the HOPB region 651. Therefore, the interference intensity of the row of interest is identified by comparing the number of events in the row of interest with the number of events in the non-interfering rows above and below the row of interest, in other words, the number of events in rows where events are normally detected. The interference intensity is estimated based on how much more the number of events in the row of interest is compared to the number of events in the non-interfering rows.

[0142] To perform this comparison, in step S242, the larger value b of the number of events between the non-interfering rows above and below the row of interest is obtained. Whether the rows above and below are non-interfering rows can be determined using, for example, an interference occurrence flag. For example, a row located above the row of interest and for which the interference occurrence flag is not set is treated as the upper non-interfering row, and a row located below the row of interest and for which the interference occurrence flag is not set is treated as the lower non-interfering row.

[0143] A non-interfering row is a row for which the interference occurrence flag is not set, and is therefore not necessarily a row adjacent to the row of interest. For example, if the interference occurrence flag is set for the row immediately above the row of interest, that row is avoided, and it is determined whether the interference occurrence flag is set for the row above that row. If the interference occurrence flag is set, it is determined whether the interference occurrence flag is set for the row above that row. This process is repeated until a row is found for which it is determined that the interference occurrence flag is not set.

[0144] In step S243, the interference intensity is identified by comparing the value a with the value b. For example, the conditions shown in FIG. 23 can be set so that the interference intensity that meets the conditions is identified. The conditions shown in FIG. 23 show an example of the conditions when the number of event data in one row is 1920. The number of event data in one row is not limited to 1920, and may be a number of event data other than 1920. Furthermore, the conditions are set appropriately depending on the number of event data. The numerical values ​​in the conditions shown in FIG. 23, for example, 1.5, 10, and 200, are merely examples and are not limiting, and appropriate values ​​can be set appropriately depending on the number of event data, etc.

[0145] Referring to FIG. 23, when the condition "value a≦1.5×value b+10" is met, the interference intensity is estimated to be "weak." When the condition "value a≧1.5×value b+200" is met, the interference intensity is estimated to be "strong." When the condition "the above two conditions are not met" is met, the interference intensity is estimated to be "medium."

[0146] The number of events may be counted separately for ON events and OFF events, or may be counted together without distinguishing between ON events and OFF events.

[0147] On events and off events may be counted separately, and the interference strength of the on events and the interference strength of the off events may be estimated separately, or on events and off events may be counted together without distinction, and the interference strength may be estimated using the number of counts counted together.

[0148] Here, the interference intensity is classified into three categories, strong, medium, and weak, in other words, a discrete classification is shown as an example, but it may also be estimated and output as a continuous value, for example, interference intensity = 0.8. In this case, the above conditions can be changed, and the interference intensity can be calculated using a predetermined arithmetic expression in which, for example, values ​​a and b are substituted.

[0149] In this way, the interference intensity can be determined using event data obtained from within the effective pixel area 652 of the pixel array unit 10.

[0150] <Third Interference Intensity Identification Process> The interference intensity identification process (referred to as third interference intensity identification process) in step S202 will be described with reference to the flowchart in FIG.

[0151] The third interference intensity identification process is performed when the correction unit 601 is configured to include an interference intensity LUT 621 as shown in FIG. 15 . In step S261, a timestamp and a row of interest for an event are set. The timestamp is information that can be treated equivalently to a frame number, and is information for identifying a frame (information for identifying the point in time at which the frame was acquired). In step S261, a frame to be processed and a row (row of interest) for identifying the interference intensity in that frame are set.

[0152] In step S262, the interference intensity corresponding to the timestamp and the row of interest is acquired from the interference intensity LUT 621. Fig. 25 is a diagram showing an example of the interference intensity LUT 621. The interference intensity LUT 621 is a table in which the timestamp, the row of interest, and the interference intensity are associated with each other.

[0153] The interference intensity LUT 621 is a table in which a timestamp, a target row, and an interference intensity are associated. In the interference intensity LUT 621 shown in Fig. 25, for example, "00000000" is associated as a timestamp, "0" as a target row, and "weak" as an interference intensity, and "00000000" is associated as a timestamp, "1" as a target row, and "medium" as an interference intensity. In the interference intensity LUT 621 shown in Fig. 25, for example, "00000015" is associated as a timestamp, "100" as a target row, and "strong" as an interference intensity.

[0154] Such an interference intensity LUT 621 is created, for example, when a pattern of how the interference intensity changes can be obtained, and is provided in the correction unit 601. Frames and rows where interference is likely to occur may depend on the device, and the interference intensity LUT 621 can be created when similar patterns exist depending on the device.

[0155] If the transition of the interference intensity is periodic, it is sufficient that the LUT for one period is stored in the interference intensity LUT 621. The interference intensity LUT 621 may be provided with a mechanism for learning and updating by acquiring an interference occurrence flag.

[0156] The table of the interference intensity LUT 621 can be configured to have a table for each of two types of events, ON events and OFF events, or to have a table that collectively includes ON events and OFF events without distinguishing between them.

[0157] Here, the interference intensity is classified into three categories, namely, strong, medium, and weak, in other words, an example is shown in which the interference intensity is classified discretely, but it may also be estimated as a continuous value, for example, interference intensity = 0.8, and the value of the continuous value may be stored in the interference intensity LUT 621.

[0158] In step S262, the interference intensity corresponding to the timestamp and the target row set as the frame to be processed in step S261 is read from the interference intensity LUT 621. In step S263, the interference intensity read from the interference intensity LUT 621 is output from the interference intensity LUT 621 to the interference correction unit 612.

[0159] In this way, when there is a specific pattern in the way interference intensity appears, it is possible to create an interference intensity LUT 621 in advance and configure it so that the interference intensity is specified by referring to the interference intensity LUT 621.

[0160] Returning to the flowchart of Fig. 18, in step S202, an interference intensity identification process is executed. The interference intensity identification process is performed by executing any one of the first to third interference intensity identification processes described above, or by executing a combination of these processes. When the interference intensity is identified by executing the interference intensity identification process, in step S203, an event data correction process is executed.

[0161] <First Event Data Correction Process> The event data correction process executed in step S203 will now be described with reference to a flowchart of FIG.

[0162] In step S301, the interference correction unit 612 acquires the event data of the target row (the row to be processed) output from the event data acquisition unit 611. In step S302, the interference correction unit 612 acquires the interference intensity of the target row identified by the interference intensity identification unit 613.

[0163] Here, the explanation will be continued by taking as an example a case where the interference intensity identifying unit 613 identifies three types of interference intensity, weak, medium, and strong, and outputs them to the interference correction unit 612. The event data correction method is switched depending on the interference intensity. In step S303, the interference correction unit 612 determines whether the interference intensity is weak.

[0164] If it is determined in step S303 that the interference intensity is weak, the process proceeds to step S304. If the interference intensity is weak, it is determined that the number of false events due to interference is small, and no correction process is performed (step S304), and the input event data is output to a subsequent processing unit. Note that, as described with reference to FIG. 17 , when converting pre-correction 2-bit data into 4-bit data for output, in step S304, the pre-correction 2-bit data is repeated and output as 4-bit data.

[0165] On the other hand, if it is determined in step S303 that the interference intensity is not weak, the process proceeds to step S305. In step S305, it is determined whether the interference intensity is medium. If it is determined in step S305 that the interference intensity is medium, the process proceeds to step S306. In step S306, noise events are removed.

[0166] The removal of noise events performed in step S306 will be described with reference to Fig. 27. When the interference intensity is medium, normal events are retained, and false events due to interference are considered to have been added as noise, and processing is performed to remove such noise events.

[0167] The left diagram in A of Fig. 27 shows the arrangement of pixels (pixels including the event signal generator 120) before correction, and the event data of an event that occurred in the event signal generator 120. Of the pixels 120-1 to 120-7 shown in A of Fig. 27, the pixel that is located in the target row determined to have medium interference intensity and that is the pixel to be corrected is designated as pixel 120-4.

[0168] The pixels located above the pixel of interest, pixel 120-4, and in the non-interfering row are referred to as pixels 120-1 to 120-3. Pixel 120-1 is located on the upper left side of the pixel of interest 120-4, pixel 120-2 is located directly above the pixel of interest 120-4, and pixel 120-3 is located on the upper right side of the pixel of interest 120-4. Note that, although A in FIG. 27 shows pixels 120-1 to 120-3 as being located in rows adjacent to the pixel of interest 120-4, if the non-interfering row is not adjacent, pixels 120-1 to 120-3 may be located in non-interfering rows several rows away.

[0169] The pixels located below the pixel 120-4 of interest and in the non-interfering row are designated as pixels 120-5 to 120-7. Pixel 120-5 is located to the lower left of the pixel of interest 120-4, pixel 120-6 is located directly below the pixel of interest 120-4, and pixel 120-7 is located to the lower right of the pixel of interest 120-4. Note that although A in FIG. 27 shows pixels 120-5 to 120-7 as being located in rows adjacent to the pixel of interest 120-4, if the non-interfering row is not adjacent, pixels 120-5 to 120-7 may be located in non-interfering rows several rows away.

[0170] The following description will be continued using such a pixel arrangement as an example. Here, pixels in which an event has occurred are shown with diagonal lines, and the description will continue assuming that pixel of interest 120-4 is a pixel in which an ON event or an OFF event has occurred. In the example shown in FIG. 27A, pixel 120-1 is also a pixel in which an ON event or an OFF event has occurred, and is shown with diagonal lines. Here, pixels with dense diagonal lines, such as pixel 120-1, are pixels in which an ON event has occurred, and pixels with thin diagonal lines, such as pixel 120-4, are pixels in which an OFF event has occurred.

[0171] A total of six pixels are set as search target pixels: pixels 120-1 to 120-3, which are ±1 pixel in the x direction in the non-interfering row located above the pixel of interest 120-4 in the interfering row, and pixels 120-5 to 120-7, which are ±1 pixel in the x direction in the non-interfering row located below the pixel of interest 120-4. If no event of the same polarity as the event of pixel of interest 120-4 exists among these six pixels set as search target pixels, it is determined to be an isolated point and is to be deleted as a noise event.

[0172] In the example shown in A of Figure 27, no OFF event has occurred in the six search pixels 120-1 to 120-3 and 120-5 to 120-7, compared to the OFF event of the target pixel 120-4. In this case, it is determined that there is no event of the same polarity (OFF event) as the target pixel 120-4 among the search target pixels, and they are processed as isolated points. As shown in the right diagram of A of Figure 27, the target pixel 120-4 in which the OFF event was detected is corrected to a pixel 120-4 in which no event has occurred.

[0173] In this way, when the event data of the pixel of interest does not match the event data of any of the multiple (six in this case) search pixels, the event data of the pixel of interest is corrected to data without an event.

[0174] See B of Fig. 27. The left diagram of B of Fig. 27 shows an example of event occurrence in a pixel arrangement before correction. In the example shown in the left diagram of B of Fig. 27, an OFF event occurs in the pixel of interest 120-4, an ON event occurs in the pixel 120-1 to be searched, and an OFF event occurs in the pixel 120-7 to be searched.

[0175] In the event occurrence situation shown in B of Figure 27, an OFF event has occurred in the pixel of interest 120-4, and an OFF event has also occurred in pixel 120-7, which is the pixel to be searched. In this case, the pixel to be searched is determined to have an event of the same polarity (OFF event) as the pixel of interest 120-4, and is held as a normal event. As shown in the right diagram of B of Figure 27, the pixel of interest 120-4 in which an OFF event has been detected is treated as the pixel 120-4 in which an OFF event has occurred.

[0176] In this way, if the event data of the pixel of interest matches the event data of at least one of the multiple (six in this case) search pixels, the event data of the pixel of interest is maintained as is.

[0177] Here, an example has been described in which six pixels are searched for, but fewer or more than six pixels may be set as search target pixels.

[0178] In this way, when the interference intensity is medium, correction is performed to remove isolated points.

[0179] 26 , if it is determined in step S305 that the interference intensity is not medium, i.e., in this case, if it is determined that the interference intensity is strong, the process proceeds to step S307. In step S307, correction is performed on all event data. If the interference intensity is strong, there is a possibility that many erroneous events have occurred due to interference, so correction is performed on all event data in the target row.

[0180] The correction of all event data executed in step S307 will be described with reference to Figures 28 to 30. In Figures 28 to 30, the upper rows show the pixel layout and an example of event occurrence before correction, and the lower rows show the pixel layout and an example of event occurrence after correction. The pixel layout is the same as in Figure 27, and the pixel to be corrected is the pixel of interest 120-4. The event data of the pixel of interest 120-4 is corrected according to the event occurrence status of the six pixels surrounding it, namely, pixels 120-1 to 120-7 (excluding pixel 120-4).

[0181] The correction rule for when the interference intensity is strong, which will be described with reference to Fig. 28, is referred to as correction rule 1. Correction rule 1 is a correction that performs masking and deletes the event of the pixel of interest.

[0182] An example will be described in which an OFF event occurs in pixel of interest 120-4. Before correction, as shown in the upper part of A in Fig. 28, ON events occur in search pixels 120-2 and 120-6 located directly above and below pixel of interest 120-4. As shown in the lower part of A in Fig. 28, since the event of the pixel of interest is deleted under correction rule 1, pixel of interest 120-4 is corrected to event data in which the event has been deleted and no event has occurred, and is then output.

[0183] Before correction shown in the upper part of B in Fig. 28, an ON event occurs in both pixel 120-2 and pixel 120-7, which are search pixels located directly above and to the lower right of pixel of interest 120-4. As shown in the lower part of B in Fig. 28, since the event of the pixel of interest is deleted under correction rule 1, the event of pixel of interest 120-4 is deleted and corrected to event data in which no event has occurred, and the event is output.

[0184] Before correction shown in the upper part of C in Fig. 28, an ON event occurs in pixel 120-2, which is a search pixel located directly above pixel of interest 120-4. As shown in the lower part of C in Fig. 28, since the event of the pixel of interest is deleted under correction rule 1, pixel of interest 120-4 is corrected to event data in which the event has been deleted and no event has occurred, and is then output.

[0185] Before correction shown in the upper part of D in Fig. 28, an ON event occurs in pixel 120-2, a search pixel located directly above pixel of interest 120-4, and an OFF event occurs in pixel 120-6, a search pixel located directly below pixel of interest 120-4. As shown in the lower part of D in Fig. 28, since the event of the pixel of interest is deleted under correction rule 1, the event of pixel of interest 120-4 is deleted and corrected to event data in which no event has occurred, and the event is output.

[0186] 29, correction rule 2 for when the interference intensity is strong will be explained. Correction rule 2 is a rule that overwrites the event data of the pixel of interest 120-4 when events occurring in search pixels located in non-interfering rows above and below the pixel of interest 120-4 and directly above and below the pixel of interest 120-4 match, and deletes the event of the pixel of interest when they do not match.

[0187] The examples shown in the upper rows of A to D in FIG. 29 show pixel arrangements and event occurrence examples before correction, and are similar to the examples shown in the upper rows of A to D in FIG.

[0188] Referring to the upper row of A in Fig. 29, before correction, ON events occur in both pixel 120-2 and pixel 120-6, which are search pixels located directly above and below pixel of interest 120-4. In this way, when events of the pixels located above and below pixel of interest 120-4 match, the event of pixel of interest 120-4 is rewritten as the matching event. Therefore, as shown in the lower row of A in Fig. 29, pixel of interest 120-4 was an OFF event, but this OFF event is rewritten as the ON events occurring in the pixels above and below it, and output.

[0189] Before correction shown in the upper part of B in Fig. 29, an ON event occurs in both pixel 120-2 and pixel 120-7, which are search pixels located directly above and to the lower right of pixel of interest 120-4, but no event occurs in pixel 120-6, which is a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, the events of the pixels located above and below pixel of interest 120-4 do not match, so the event of pixel of interest 120-4 is deleted and rewritten as no event, and output, as shown in the lower part of B in Fig. 29.

[0190] Before correction shown in the upper part of C in Fig. 29, an ON event occurs in pixel 120-2, a search pixel located directly above pixel of interest 120-4, but no event occurs in pixel 120-6, a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, the events of the pixels located above and below pixel of interest 120-4 do not match, so the event of pixel of interest 120-4 is deleted and rewritten as no event, and output, as shown in the lower part of C in Fig. 29.

[0191] Before correction shown in the upper part of D in Fig. 29, an ON event occurs in pixel 120-2, a search pixel located directly above pixel of interest 120-4, and an OFF event occurs in pixel 120-6, a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 2, the events of the pixels located above and below pixel of interest 120-4 do not match, so the event of pixel of interest 120-4 is deleted and rewritten as no event, and output, as shown in the lower part of D in Fig. 29.

[0192] 30, correction rule 3 for when the interference intensity is strong will be explained. Correction rule 3 is a rule that overwrites the event data of pixel of interest 120-4 when the events occurring in the search pixels directly above and below the pixel of interest 120-4 match, and when the events occurring in the search pixels directly above and below the pixel of interest 120-4 do not match but an event occurring in a search pixel diagonally in place of the pixel directly above or below matches, and deletes the event of the pixel of interest when there is no match.

[0193] The examples shown in the upper rows of A to D in FIG. 30 show pixel arrangements and event occurrence examples before correction, and are similar to the examples shown in the upper rows of A to D in FIG.

[0194] Referring to the upper row of A in Fig. 30, before correction, ON events occur in both pixel 120-2 and pixel 120-6, which are search pixels located directly above and below pixel of interest 120-4. In this way, when events of the pixels located above and below pixel of interest 120-4 match, the event of pixel of interest 120-4 is rewritten as the matching event. Therefore, as shown in the lower row of A in Fig. 30, pixel of interest 120-4 is an OFF event, but this OFF event is rewritten as the ON events occurring in the pixels above and below it, and output.

[0195] Before correction shown in the upper part of B in Fig. 30, an ON event occurs in both pixel 120-2 and pixel 120-7, which are search pixels located directly above and to the lower right of pixel of interest 120-4, but no event occurs in pixel 120-6, which is a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below pixel of interest 120-4 do not match, but the events of the pixel located above and diagonally below pixel of interest 120-4 match as ON events, so the event of pixel of interest 120-4 is corrected from an OFF event to an ON event and output, as shown in the lower part of B in Fig. 30.

[0196] Before correction shown in the upper part of C in Fig. 30, an ON event has occurred in pixel 120-2, a search pixel located directly above pixel of interest 120-4, but no event has occurred in pixel 120-6, a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below pixel of interest 120-4 do not match, and they also do not match with the events of pixels located diagonally, so as shown in the lower part of C in Fig. 30, the event of pixel of interest 120-4 is deleted and rewritten as no event, and is output.

[0197] Before correction shown in the upper part of D in Fig. 30, an ON event occurs in pixel 120-2, a search pixel located directly above pixel of interest 120-4, and an OFF event occurs in pixel 120-6, a search pixel located directly below pixel of interest 120-4. In such a case, when correction is performed based on correction rule 3, the events of the pixels located above and below pixel of interest 120-4 do not match, and they also do not match with the events of pixels located diagonally, so as shown in the lower part of D in Fig. 30, the event of pixel of interest 120-4 is deleted and rewritten as no event and output.

[0198] When the interference intensity is strong, the event data of the pixel of interest is corrected by applying one of the correction rules 1 to 3. The correction rules 1 to 3 can also be configured to be used in combination.

[0199] In the above-described embodiment, the non-interfering rows may include corrected interfering rows. In other words, the corrected interfering rows may be treated as non-interfering rows when processing the row of interest. In such a case, an interfering row with medium or weak interference intensity and for which correction has been completed may be treated as a non-interfering row, but an interfering row with strong interference intensity and for which correction has been completed may not be treated as a non-interfering row.

[0200] If there is a non-interfering line located more than a predetermined number of lines away from the line of interest, the corrected interfering line close to the line of interest may be treated as a non-interfering line, and the above-mentioned correction process may be performed.

[0201] <Correction Process of Second Event Data> The correction process of the event data executed in step S203 will now be described. Fig. 31 is a flowchart for explaining the correction process of the second event data. The correction process of the second event data is a correction process when the interference intensity is output as a continuous value.

[0202] In step S321, the interference correction unit 612 acquires the event data of the target row (the row to be processed) output from the event data acquisition unit 611. In step S322, the interference correction unit 612 acquires the interference intensity of the target row identified by the interference intensity identification unit 613. The interference intensity acquired here is a continuous value, for example, a value such as 0.8.

[0203] In step S323, the interference intensity is converted into a correction factor. For example, the interference intensity is converted into a correction factor based on the graph shown in FIG. 32. In the graph shown in FIG. 32, the horizontal axis represents the interference intensity, and the vertical axis represents the correction factor. s0 and s1 shown on the horizontal axis each represent a control parameter. When the interference intensity is converted into a correction factor based on the graph shown in FIG. 32, if the interference intensity is between 0 and s0, the correction factor is converted to "0", and if the interference intensity is equal to or greater than s1, the correction factor is converted to "1". If the interference intensity is between s0 and s1, the correction factor is converted into a value expressed by a linear function according to the value of the interference intensity.

[0204] Returning to the flowchart shown in FIG. 31, in step S324, a pixel to be corrected in the target row, for example, a pixel corresponding to the target pixel 120-4 in the case described with reference to FIGS. 28 to 30, is set.

[0205] In step S325, a uniform random number in the range [0.0, 1.0] is generated. In step S326, the generated random number is compared with the converted correction factor to determine whether the value of the random number is equal to or less than the correction factor. This determination is made to control the number of event data to be processed according to the correction factor, for example, by reducing the number of event data to be corrected when the correction factor is small and increasing the number of event data to be corrected when the correction factor is large.

[0206] If it is determined in step S326 that the value of the random number is equal to or less than the correction factor, the process proceeds to step S327, where the pixel of interest is corrected.

[0207] In step S326, the random number is determined to be equal to or less than the correction factor when the interference intensity is determined to be strong or medium in the first event data correction process. Therefore, the correction of the pixel of interest performed in step S327 can be performed using the processes performed in steps S306 and S307 (FIG. 26).

[0208] On the other hand, if it is determined in step S326 that the value of the random number is not equal to or less than the correction factor, the process proceeds to step S328, where no correction process is performed on the pixel of interest, and the event data of the pixel of interest is output.

[0209] In step S326, the random number is determined to be greater than or equal to the correction factor when the interference intensity is determined to be weak in the first event data correction process. Therefore, the correction performed in step S328 can be the same as the process performed in step S304 (FIG. 26), i.e., no correction process is performed in this case.

[0210] When the processing in step S327 or step S328 is completed, the process proceeds to step S329. In step S329, it is determined whether or not one row of pixels has been processed. If it is determined in step S328 that there are pixels in the target row that have not yet been processed, the process returns to step S324, and the processing from step S324 onwards is repeated, with the unprocessed pixels in the target row being the processing target.

[0211] On the other hand, if it is determined in step S329 that all pixels in the target row have been processed, the process of the flowchart shown in FIG. 31 ends, and the process proceeds to step S204 (FIG. 18).

[0212] In step S204, the event data of the target row after correction is output to a subsequent processing unit.

[0213] In this way, the event data is corrected according to the interference intensity. By correcting the event data according to the interference intensity, appropriate correction can be performed, thereby improving the accuracy of the event data.

[0214] <Regarding Input / Output Response in Correction Unit> A description will now be given of the input / output response in the correction unit 601. By changing the event data input to the correction unit 601 and observing the event data output from the correction unit 601, it is possible to determine whether the correction unit 601 applies the present technology or not.

[0215] Event data that has not been interfered with is prepared, and an interference occurrence flag is assigned to this event data that has not been interfered with. An erroneous event is assigned to the row to which the interference occurrence flag is assigned. This assigned erroneous event may be an ON event or an OFF event.

[0216] The event data with the false event attached is input to the correction unit 601, and the output of the correction unit 601 is observed. At this time, the false event attachment rate is changed between 0 and 100%, and the output is observed at each change. In other words, the interference strength is changed between weak (none) and strong, and the output is observed at each change.

[0217] As described above, when correction is performed by the correction unit 601 to which the present technology is applied, if the false event assignment rate is changed between 0 and 100%, correction is performed according to the false event assignment rate, in other words, correction is performed according to the interference intensity, so that the output event data differs for each assignment rate. Specifically, the lower the false event assignment rate, the more the original event data is retained in the output.

[0218] If the correction unit 601 performs correction uniformly instead of performing correction according to interference intensity, the output event data will not change and will remain the same even if the false event assignment rate changes.

[0219] As described above, the output results are different between the correction unit 601 to which the present technology is applied and the correction unit to which the present technology is not applied. With the correction unit 601 to which the present technology is applied, it can be confirmed by such a confirmation method that appropriate correction is performed.

[0220] <Regarding the Recording Medium> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0221] 33 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes using a program. In the computer, a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, and a RAM (Random Access Memory) 2003 are interconnected by a bus 2004. An input / output interface 2005 is also connected to the bus 2004. An input unit 2006, an output unit 2007, a storage unit 2008, a communication unit 2009, and a drive 2010 are connected to the input / output interface 2005.

[0222] The input unit 2006 includes a keyboard, a mouse, a microphone, etc. The output unit 2007 includes a display, a speaker, etc. The storage unit 2008 includes a hard disk, a non-volatile memory, etc. The communication unit 2009 includes a network interface, etc. The drive 2010 drives removable media 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0223] In a computer configured as described above, the CPU 2001 performs the above-described series of processes by, for example, loading a program stored in the memory unit 2008 into the RAM 2003 via the input / output interface 2005 and the bus 2004 and executing it.

[0224] The program executed by the computer (CPU 2001) can be provided by being recorded on, for example, a removable medium 2011 such as a package medium. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0225] In the computer, the program can be installed in the storage unit 2008 via the input / output interface 2005 by inserting the removable medium 2011 into the drive 2010. The program can also be received by the communication unit 2009 via a wired or wireless transmission medium and installed in the storage unit 2008. Alternatively, the program can be installed in advance in the ROM 2002 or the storage unit 2008.

[0226] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0227] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0228] FIG. 34 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0229] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 34, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0230] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0231] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0232] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0233] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0234] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0235] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0236] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0237] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0238] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 34, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0239] FIG. 35 is a diagram showing an example of the installation position of the imaging unit 12031.

[0240] In FIG. 35, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0241] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0242] 35 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0243] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0244] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0245] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0246] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0247] In this specification, a system refers to an entire device made up of multiple devices.

[0248] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0249] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0250] Note that the present technology may also be configured as follows: (1) A photodetector comprising: a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, an interference intensity determiner that determines an intensity of interference occurring due to an overlap between a period in which the gradation data is generated and a period in which the event is detected, and a correction unit that corrects the event data according to the intensity of interference determined by the interference intensity determiner. (2) The photodetector according to (1), wherein the pixel array unit has a light-shielded region, and the interference intensity determiner counts the number of events detected by the event data generation unit located in the light-shielded region and determines the intensity of the interference according to the number of counted events. (3) The photodetector according to (1), wherein the interference intensity identifier acquires a first number of events counted by counting events in a row of interest that identifies the intensity of interference in the pixel array unit, acquires a second number of events in a non-interfering row located above the row of interest and where there is no interference, acquires a third number of events in a non-interfering row located below the row of interest and where there is no interference, and identifies the intensity of interference by determining whether a predetermined condition is met using the larger of the first number of events, the second number of events, and the third number of events. (4) The photodetector according to (3), wherein the interference occurrence flag is acquired to indicate that a predicted overlap row is detected, the row being a row where the generation of the gradation data and the detection of the event are predicted to overlap, the non-interfering row being a row for which the interference occurrence flag is not set. (5) The photodetector according to (1), further comprising: a table in which information for identifying a frame, a row of interest that identifies the intensity of interference in the pixel array unit, and the intensity of the interference are associated; and the interference intensity identifier refers to the table to identify the intensity of interference in the row of interest. (6) The photodetector according to any one of (1) to (5), wherein the correction unit does not perform correction when the interference intensity specifying unit specifies that the intensity of the interference is weak.(7) The photodetector according to any one of (1) to (6), wherein, when the interference intensity specifying unit specifies that the intensity of interference is strong, the correction unit corrects the event data to indicate that no event has occurred at the pixel of interest that is the correction target. (8) When the interference intensity specifying unit specifies that the intensity of interference is strong, the correction unit acquires a first event of a pixel that is located above the pixel of interest that is the correction target and immediately above the pixel of interest in a non-interference row where there is no interference, and a second event of a pixel that is located below the pixel of interest and immediately below the pixel of interest in a non-interference row where there is no interference, and when the first event and the second event match, corrects the event of the pixel of interest to event data indicating that the first event has occurred, and when the first event and the second event do not match, corrects the event to event data indicating that no event has occurred at the pixel of interest. (9) When the interference intensity identification unit identifies that the intensity of the interference is strong, the correction unit acquires a first event of a pixel located above the pixel of interest to be corrected and located directly above or diagonally above the pixel of interest in a non-interference row where there is no interference, and a second event of a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interference row where there is no interference; if the first event and the second event match, corrects the event of the pixel of interest to event data indicating that the first event has occurred; and if the first event and the second event do not match, corrects the event of the pixel of interest to event data indicating that no event has occurred at the pixel of interest.(10) When the interference intensity identification unit identifies the intensity of interference as medium, the correction unit sets, as search pixels, a pixel located above the pixel of interest being the correction target and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference; when an event of the pixel of interest matches an event of at least one pixel among the events of the plurality of search pixels, maintains the event data of the event of the pixel of interest; and when an event matching the event of the pixel of interest has not occurred in any of the plurality of search pixels, corrects the event data to indicate that no event has occurred in the pixel of interest. (11) The photodetector according to any one of (1) to (9), wherein the interference intensity specifying unit outputs the intensity of interference as a continuous value, and the correction unit converts the continuous value into a correction factor, generates a random number, and if the random number is equal to or less than the correction factor, corrects the event data of a pixel of interest being processed, and if the random number is not equal to or less than the correction factor, maintains the event data of the pixel of interest being processed. (12) The photodetector according to any one of (1) to (11), wherein the input and output of the correction unit are event data with the same number of bits, and when the input event data is corrected, outputs the corrected event data. (13) The photodetector according to any one of (1) to (11), wherein the correction unit adds the corrected event data to the input event data and outputs the event data. (14) A correction method in which a photodetector includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, identifies the intensity of interference that occurs due to an overlap between the generation of the gradation data and the detection of the event, and corrects the event data according to the identified intensity of interference.(15) A program for causing a computer that controls a photodetector having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit including an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, to execute the following process: identify the intensity of interference that occurs due to an overlap between the generation of the gradation data and the detection of the event; and correct the event data according to the identified intensity of interference.

[0251] REFERENCE SIGNS LIST 1 Photodetector, 10 Pixel array section, 11 Signal line, 12 Signal line, 13 Pixel, 14 Pixel, 15 Pixel, 16 Pixel, 17 Pixel, 18 Pixel, 20 Access control circuit, 21 Signal line, 30 Event signal output circuit, 40 Gradation signal output circuit, 50 Timing control section, 60 Event signal processing section, 70 Gradation signal processing section, 80 Overlapping predicted row detection section, 81 Overlapping row prediction section, 90 Time stamp generation section, 100 Pixel, 110 Gradation signal generation section, 111 MOS transistor, 120 Event signal generation section, 121 Signal line, 130 Photoelectric conversion section, 140 Current-voltage conversion circuit, 150 Differentiation circuit, 160 Brightness change detection section, 170 Output section, 201 Photoelectric conversion section, 202 Photoelectric conversion unit, 203 Charge storage unit, 204 Capacitor, 205 Capacitor, 231 Constant current circuit, 500 Overlapping predicted row, 501 Shutter, 502 Exposure, 503 Readout, 510 Frame, 551 Area, 601 Correction unit, 602 Application processor, 612 Interference correction unit, 613 Interference intensity identification unit, 621 Interference correction unit, 651 HOPB area, 652 Effective pixel area, 671 Table

Claims

1. A photodetection device comprising: a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit having an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event; and a gradation data generation unit that generates gradation data that is data corresponding to the luminance of the incident light; an interference intensity determination unit that determines the intensity of interference that occurs due to an overlap in the period of generation of the gradation data and the period of detection of the event; and a correction unit that corrects the event data according to the intensity of interference determined by the interference intensity determination unit.

2. The optical detection device according to claim 1, wherein the pixel array section has a light-shielded area, and the interference intensity determination section counts the number of events detected in the event data generation section located in the light-shielded area, and determines the intensity of the interference according to the counted number of events.

3. The optical detection device of claim 1, wherein the interference intensity identification unit: obtains a first number of events by counting events in a row of interest that identifies the intensity of interference in the pixel array unit; obtains a second number of events in a non-interfering row located above the row of interest and where there is no interference; obtains a third number of events in a non-interfering row located below the row of interest and where there is no interference; and identifies the intensity of interference by determining whether or not a predetermined condition is met using the larger of the first number of events, the second number of events, and the third number of events.

4. The optical detection device according to claim 3, further comprising: an interference occurrence flag indicating that an overlap predicted row, which is a row in which the generation of the gradation data and the detection of the event are predicted to overlap, has been detected; and the non-interfering row is a row in which the interference occurrence flag is not set.

5. The optical detection device according to claim 1, further comprising a table in which information for identifying a frame, a row of interest that identifies the intensity of interference in the pixel array section, and the intensity of the interference are associated, and the interference intensity identification section refers to the table to identify the intensity of the interference in the row of interest.

6. The optical detection device according to claim 1, wherein when the interference intensity determination section determines that the intensity of the interference is weak, the correction section does not perform correction.

7. The optical detection device according to claim 1, wherein, when the interference intensity identification unit identifies the intensity of the interference as being strong, the correction unit corrects the event data to indicate that no event has occurred in the pixel of interest that is the subject of correction.

8. The optical detection device of claim 1, wherein, when the interference intensity identification unit identifies that the intensity of interference is strong, the correction unit acquires a first event of a pixel located above the pixel of interest being corrected and directly above the pixel of interest in a non-interfering row where there is no interference, and a second event of a pixel located below the pixel of interest and directly below the pixel of interest in a non-interfering row where there is no interference; when the first event and the second event match, the correction unit corrects the event of the pixel of interest to event data indicating that the first event has occurred; and when the first event and the second event do not match, the correction unit corrects the event of the pixel of interest to event data indicating that no event has occurred at the pixel of interest.

9. The optical detection device of claim 1, wherein when the interference intensity identification unit identifies the intensity of interference as strong, the correction unit acquires a first event of a pixel located above the pixel of interest being corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and a second event of a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference; when the first event and the second event match, corrects the event of the pixel of interest to event data indicating that the first event has occurred; and when the first event and the second event do not match, corrects the event of the pixel of interest to event data indicating that no event has occurred at the pixel of interest.

10. The optical detection device of claim 1, wherein when the interference intensity identification unit identifies the intensity of interference as medium, the correction unit sets as search pixels a pixel located above the pixel of interest being corrected and located directly above or diagonally above the pixel of interest in a non-interfering row where there is no interference, and a pixel located below the pixel of interest and located directly below or diagonally below the pixel of interest in a non-interfering row where there is no interference, maintains the event data of the event of the pixel of interest when an event of the pixel of interest matches an event of at least one pixel among the events of the multiple search pixels, and corrects the event data to indicate that no event has occurred at the pixel of interest when an event matching the event of the pixel of interest has occurred in any of the multiple search pixels.

11. The optical detection device of claim 1, wherein the interference intensity identification unit outputs the intensity of the interference as a continuous value, and the correction unit converts the continuous value into a correction factor, generates a random number, corrects the event data of the pixel of interest being processed if the random number is less than or equal to the correction factor, and maintains the event data of the pixel of interest being processed if the random number is not less than or equal to the correction factor.

12. The optical detection device according to claim 1, wherein the input and output of the correction unit are event data having the same number of bits, and when the input event data is corrected, the corrected event data is output.

13. The photodetector according to claim 1, wherein the correction section adds corrected event data to the input event data and outputs the corrected event data.

14. A correction method for a photodetection device having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit having an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data corresponding to the luminance of the incident light, the photodetection device identifying the intensity of interference that occurs due to an overlap in the periods of generation of the gradation data and detection of the event, and correcting the event data in accordance with the identified intensity of interference.

15. A program for causing a computer to execute a process of controlling a photodetection device having a pixel array unit in which a plurality of pixels are arranged in a two-dimensional matrix, the pixel array unit having an event data generation unit that detects a change in the luminance of incident light in the same direction as an event and generates event data based on the detected event, and a gradation data generation unit that generates gradation data that is data according to the luminance of the incident light, the computer executing a process of identifying the intensity of interference that occurs due to an overlap in the period of generation of the gradation data and the period of detection of the event, and correcting the event data according to the identified intensity of interference.

Citation Information

Patent Citations

  • Sensor device and read-out method

    JP2021129265A

  • Solid-state imaging element and imaging apparatus

    JP2022108423A

  • Solid-state imaging device and imaging device that combine dynamic vision sensor and imaging function

    JP2022548199A

  • Solid-state imaging element and imaging device

    WO2021117350A1

  • Image sensor, data processing device, and image sensor system

    WO2023058670A1