Imaging device and imaging method

Abstract

[Problem] There is no risk of generating unnecessary events. [Solution] This imaging device comprises: a photoelectric conversion unit that has a plurality of photoelectric conversion elements for performing photoelectric conversion to generate an electric signal; a setting unit that sets a threshold value according to a noise level in a predetermined region among the plurality of photoelectric conversion elements; and a first detection unit that detects a detection signal when the amount of change in the electric signal generated by the plurality of photoelectric conversion elements exceeds the threshold value.

Description

[Technical field]

[0001] The present disclosure relates to an imaging device and an imaging method. [Background technology]

[0002] As one of the event-driven imaging devices, an asynchronous imaging device called DVS (Dynamic Vision Sensor) has been proposed (see, for example, Patent Document 1). In the asynchronous imaging device, only when some event (for example, movement) occurs in the scene, data of a part where the luminance level has changed due to the event is acquired. Therefore, the asynchronous imaging device can acquire image data much faster than a general synchronous imaging device that unnecessarily acquires all the data of an image at a fixed frame rate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-535999 Summary of the Invention [Problem to be solved by the invention]

[0004] Because the DVS has excellent sensitivity, it reacts to changes in electrical signals caused by noise in the photoelectric conversion elements in the imaging device, and there is a risk that events will be generated even in scenes where there is essentially no movement. In addition, the generation of unnecessary events that are actually necessary increases the power consumption of the DVS.

[0005] Therefore, the present disclosure provides an imaging device and an imaging method that are free from the risk of causing unnecessary events. [Means for solving the problem]

[0006] In order to solve the above problems, according to the present disclosure, a photoelectric conversion unit having a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal; a setting unit that sets a threshold value according to a noise level of a predetermined region of the plurality of photoelectric conversion elements; a first detection unit that detects a detection signal when a change amount of the electrical signal generated by the plurality of photoelectric conversion elements exceeds the threshold; An imaging device is provided, comprising:

[0007] the photoelectric conversion elements in the predetermined region are shielded from light, The photoelectric conversion elements other than the predetermined region may each perform photoelectric conversion on incident light to generate an electrical signal.

[0008] The multiple photoelectric conversion elements may be arranged in a two-dimensional matrix, and the light-shielded area may correspond to at least one of an arrangement of the photoelectric conversion elements in rows and an arrangement of the photoelectric conversion elements in columns.

[0009] The row-by-row arrangement of the photoelectric conversion elements and the column-by-column arrangement of the photoelectric conversion elements may include ends of the plurality of photoelectric conversion elements arranged in the two-dimensional matrix.

[0010] The setting unit may set the threshold value based on the number of times that an absolute value of an amount of change in the electrical signal generated by a photoelectric conversion element in the predetermined region exceeds the threshold value within a predetermined period.

[0011] the first detection unit detects a first detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal increases exceeds a first threshold, and detects a second detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal decreases exceeds a second threshold; The setting unit is setting the first threshold based on the number of times that an absolute value of a change in a direction in which a signal level of the electrical signal generated by the electrical conversion element in the predetermined region increases exceeds the first threshold in a predetermined period; The second threshold may be set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electrical signal generated by the electrical conversion element in the specified region decreases exceeds the second threshold in a specified period.

[0012] The setting unit may set the threshold in stages based on the number of times the threshold is exceeded in the predetermined period.

[0013] The setting unit may decrease a rate of change of the threshold value over time.

[0014] The setting unit may reduce a rate of change of the threshold value so as to gradually approach an initial setting.

[0015] The setting unit may set the threshold to a first-stage threshold, and then reduce the rate of change to gradually approach a predetermined set value.

[0016] The setting unit may not change the threshold value when the number of times that the threshold value is exceeded during the predetermined period is less than a predetermined value.

[0017] The setting unit may set the threshold value in accordance with temperatures corresponding to the plurality of photoelectric conversion elements.

[0018] The setting unit may increase a rate of change of the threshold value as the temperature change increases.

[0019] The first detection unit sequentially reads out the electrical signals of the photoelectric conversion elements in the predetermined region, The setting unit may count the number of the detection signals that exceed the threshold value over a predetermined period of time.

[0020] In order to solve the above problem, according to the present disclosure, a threshold is set according to a noise level of a light-shielded photoelectric conversion element; The imaging method detects a detection signal when an absolute value of a change in an electrical signal generated by a plurality of photoelectric conversion elements, each of which performs photoelectric conversion on incident light to generate an electrical signal, exceeds the threshold value. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing an example of a system configuration of an imaging system to which the technology according to the present disclosure is applied. [Diagram 2] 1 is a block diagram showing an example of the configuration of an imaging device according to a first configuration example of the present disclosure. [Diagram 3] FIG. 2 is a block diagram showing an example of the configuration of a pixel array unit. [Figure 4] FIG. 2 is a circuit diagram showing an example of a circuit configuration of a pixel. [Diagram 5] FIG. 4 is a block diagram showing a first example of the configuration of an address event detection unit. [Figure 6] FIG. 4 is a block diagram showing a first configuration example of an address event detection unit of a light-shielded pixel. [Figure 7] FIG. 4 is a circuit diagram showing an example of the configuration of a current-voltage conversion unit. [Figure 8] FIG. 4 is a circuit diagram showing an example of a configuration of a subtractor and a quantizer in the address event detection unit. [Figure 9] FIG. 13 is a block diagram showing a second example of the configuration of the address event detection unit. [Figure 10] FIG. 13 is a block diagram showing a second example configuration of the address event detection unit of the light-shielded pixel. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of an imaging device according to a second configuration example of the present disclosure. [Figure 12] FIG. 2 is an exploded perspective view showing an outline of a stacked chip structure of the imaging device. [Figure 13] FIG. 2 is a block diagram showing an example of the configuration of a column processing unit of an imaging device according to a first configuration example. [Figure 14] FIG. 1 is a schematic diagram showing the relationship between noise events and temperature. [Figure 15] 10 is a flowchart showing an example of processing by a threshold control unit. [Figure 16] 10 is a flowchart showing an example of processing by a threshold control unit in parallel processing. [Figure 17] 17 is a flowchart showing an example of the process of step S1000 in FIGS. 15 and 16. [Figure 18]FIG. 18 is a diagram illustrating the example of the process shown in FIG. 17. [Figure 19] 18 is a diagram showing an example of processing in which the processing shown in FIG. 17 is performed for three periods. [Figure 20] 10 is a flowchart showing an example of a process for independently adjusting a first threshold value and a second threshold value. [Figure 21] FIG. 11 is a diagram illustrating an example of a process for independently adjusting a first threshold value and a second threshold value. [Figure 22] 11 is a flowchart showing an example of a process for independently adjusting a first threshold value and a second threshold value in parallel processing. [Figure 23] 11 is a flowchart showing an example of changing a threshold value in stages. [Figure 24] FIG. 24 is a diagram showing a schematic diagram of the processing example shown in FIG. 23. [Diagram 25] 11 is a flowchart showing an example of changing the first threshold value in stages. [Figure 26] FIG. 11 is a diagram illustrating an example in which the first threshold value is changed in stages. [Figure 27] 10 is a flowchart showing an example of changing the second threshold value in stages. [Figure 28] FIG. 11 is a diagram illustrating an example in which the second threshold value is changed in a stepwise manner. [Figure 29] 13 is a flowchart showing an example in which a dead zone is provided. [Diagram 30] FIG. 13 is a diagram illustrating an example in which a second threshold value having a dead zone is changed in a stepwise manner. [Diagram 31] 10 is a flowchart showing an example of changing the change amount of a threshold value in response to a temperature change. [Diagram 32] FIG. 11 is a diagram showing an example in which the change amount of a threshold is changed in response to a temperature change. [Diagram 33] FIG. 11 is a block diagram showing the configuration of an imaging device according to a modified example of the first embodiment. [Diagram 34] 1 is a block diagram showing a schematic configuration example of a vehicle control system that is an example of a mobile object control system to which the technology according to the present disclosure can be applied. [Diagram 35] 4A and 4B are diagrams showing examples of installation positions of an imaging unit and an outside-vehicle information detection unit; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an embodiment of an imaging device and an imaging method will be described with reference to the drawings. The following description will focus on the main components of the imaging device, but the imaging device may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0023] (First embodiment) FIG. 1 is a block diagram showing an example of a system configuration of an imaging system to which the technology according to the present disclosure is applied.

[0024] 1, an imaging system 10 to which the technology according to the present disclosure is applied includes an imaging lens 11, a recording unit 12, a control unit 13, and an imaging device 20. This imaging system 10 is an example of an electronic device according to the present disclosure, and examples of such electronic device include a camera system mounted on an industrial robot and an in-vehicle camera system.

[0025] In the imaging system 10 configured as above, the imaging lens 11 captures incident light from a subject and forms an image on the imaging surface of the imaging device 20. The imaging device 20 performs photoelectric conversion on a pixel-by-pixel basis on the incident light captured by the imaging lens 11 to obtain imaging data. An imaging device according to the present disclosure, which will be described later, is used as this imaging device 20.

[0026] The imaging device 20 executes predetermined signal processing such as image recognition processing on the captured image data, and outputs the processing result and data indicating an address event detection signal (hereinafter, may be simply referred to as a "detection signal") described later to the recording unit 12. A method for generating an address event detection signal will be described later. The recording unit 12 stores data supplied from the imaging device 20 via a signal line 14. The control unit 13 is, for example, configured by a microcomputer, and controls the imaging operation in the imaging device 20.

[0027] [Imaging device according to the first configuration example (arbiter system)] FIG. 2 is a block diagram showing an example of the configuration of an imaging device according to a first configuration example, which is used as the imaging device 20 in the imaging system 10 to which the technology according to the present disclosure is applied.

[0028] 2, an imaging device 20 according to a first configuration example of an imaging device of the present disclosure is an asynchronous imaging device called DVS, and includes a pixel array unit 21, a driving unit 22, an arbiter unit (arbitration unit) 23, a column processing unit 24, a signal processing unit 25, and a setting unit 26. The setting unit 26 also includes a counting unit 26a and a threshold control unit 26b.

[0029] In the imaging device 20 having the above configuration, a plurality of pixels 30 are two-dimensionally arranged in a matrix (array) in the pixel array section 21. Vertical signal lines VSL, which will be described later, are wired for each pixel column in this matrix-like pixel arrangement.

[0030] Each of the pixels 30 generates an analog signal of a voltage corresponding to the photocurrent as a pixel signal. Each of the pixels 30 detects the presence or absence of an address event based on whether or not the amount of change in the photocurrent exceeds a predetermined threshold. When an address event occurs, the pixel 30 outputs a request to the arbiter unit 23.

[0031] A light-shielding region 21a that blocks light is provided in the pixel array section 21. Meanwhile, a light-receiving region 21b is provided in the pixel array section 21 other than the light-shielding region 21a.

[0032] The light-shielding region 21a is provided, for example, in correspondence with a plurality of pixels 30 arranged in rows at the upper end. Alternatively, it may be provided in correspondence with a plurality of pixels 30 arranged in columns. In this way, the light-shielding region 21a is provided at least at the upper end, lower end, left end, or right end of the pixel array unit 21 so as to cover the plurality of pixels 30. The configuration of the pixels 30 in the light-shielding region 21a is the same as that of the pixels 30 outside the light-shielding region 21a, except that no light is incident thereon. As a result, the plurality of pixels 30 in the light-shielding region 21a detect the presence or absence of an address event, for example, depending on whether or not the amount of change in current due to dark noise exceeds a predetermined threshold. In addition, in this embodiment, the pixels 30 in the light-shielding region 21a may be referred to as light-shielding pixels 30. The temperature sensor 21c acquires temperatures corresponding to the plurality of pixels 30 in the pixel array unit 21, and supplies a signal including temperature information to the threshold control unit 26b of the setting unit .

[0033] The drive unit 22 drives each of the pixels 30 arranged in the light receiving region 21b other than the light blocking region 21a, and causes the pixels 30 to output pixel signals generated by the pixels 30 to the column processing unit .

[0034] The arbiter unit 23 arbitrates requests from each of the multiple pixels 30 and transmits a response based on the arbitration result to the pixel 30. Upon receiving the response from the arbiter unit 23, the pixel 30 supplies a detection signal indicating the detection result (address event detection signal) to the drive unit 22 and the signal processing unit 25. The detection signal from the pixel 30 can also be read out from multiple rows.

[0035] The column processing unit 24 is composed of, for example, an analog-to-digital converter, and performs processing for converting analog pixel signals output from the pixels 30 in each pixel column of the pixel array unit 21 into digital signals. Then, the column processing unit 24 supplies the digital signals after the analog-to-digital conversion to the signal processing unit 25.

[0036] The signal processing unit 25 executes predetermined signal processing such as CDS (Correlated Double Sampling) processing and image recognition processing on the digital signal supplied from the column processing unit 24. Then, the signal processing unit 25 supplies data indicating the processing result and the detection signal supplied from the arbiter unit 23 to the recording unit 12 (see FIG. 1) via the signal line 14.

[0037] The setting unit 26 sets a threshold value for detecting the presence or absence of an address event. The setting unit 26 sets the threshold value according to, for example, the noise level of the plurality of light-shielded pixels 30. The counting unit 26a counts the address events that occur in the light-shielded pixels 30.

[0038] The threshold control unit 26b sets a threshold for detecting the presence or absence of an address event in the address event detection unit 33 described later according to the number of address events counted by the counting unit 26a. The threshold control unit 26b has, for example, a register, and can set the address event threshold and the threshold for the number of occurrences of address events by the register. The details of the threshold control unit 26b will be described later.

[0039] [Pixel array configuration example] FIG. 3 is a block diagram showing an example of the configuration of the pixel array unit 21. As shown in FIG.

[0040] In the pixel array section 21 in which a plurality of pixels 30 are two-dimensionally arranged in a matrix, each of the plurality of pixels 30 includes a light receiving section 31, a pixel signal generating section 32, and an address event detecting section 33.

[0041] In the pixel 30 having the above configuration, the light receiving unit 31 in the light receiving region 21b performs photoelectric conversion on the incident light to generate a photocurrent. Then, the light receiving unit 31 supplies the photocurrent generated by the photoelectric conversion to either the pixel signal generating unit 32 or the address event detecting unit 33 under the control of the driving unit 22 (see FIG. 2).

[0042] On the other hand, the light receiving unit 31 of the light-shielded pixel outputs an output current corresponding to a noise component such as a dark current, and supplies the current corresponding to the noise component to the address event detection unit 33 under the control of the drive unit 22 (see FIG. 2).

[0043] The pixel signal generation unit 32 generates a pixel signal SIG, which is a voltage signal corresponding to the photocurrent supplied from the light receiving unit 31 of the light receiving region 21b, and supplies this generated pixel signal SIG to the column processing unit 24 (see Figure 2) via the vertical signal line VSL.

[0044] The address event detector 33 detects the presence or absence of an address event based on whether or not the amount of change in photocurrent from each of the light-receiving sections 31 in the light-receiving region 21b has exceeded a predetermined threshold. An address event (hereinafter sometimes simply referred to as an "event") is made up of, for example, an ON event indicating that the amount of change in photocurrent has exceeded a first threshold, which is the upper limit, and an OFF event indicating that the amount of change has fallen below a second threshold, which is the lower limit. The address event detection signal is made up of, for example, one bit indicating the detection result of an ON event and one bit indicating the detection result of an OFF event. The address event detector 33 may also be configured to detect only ON events.

[0045] Meanwhile, the address event detector 33 of the light-shielded pixel detects the presence or absence of an address event based on whether the amount of change in the output current output by each of the light-receiving sections 31 of the light-shielded pixel exceeds a predetermined threshold. The address event is, for example, an ON event indicating that the amount of change in the output current exceeds a first threshold, which is an upper limit, and an OFF event indicating that the amount of change falls below a second threshold, which is a lower limit. The address event detection signal is, for example, one bit indicating the detection result of an ON event and one bit indicating the detection result of an OFF event. The address event detector 33 may also be configured to detect only ON events or only OFF events.

[0046] When an address event occurs, the address event detection unit 33 in the light-receiving region 21b supplies a request for transmitting an address event detection signal to the arbiter unit 23 (see FIG. 2). Then, upon receiving a response to the request from the arbiter unit 23, the address event detection unit 33 supplies an address event detection signal to the drive unit 22 and the signal processing unit 25.

[0047] On the other hand, when an address event occurs, the address event detection unit 33 of the light-shielded pixel supplies a detection signal of the address event to the count unit 26a of the setting unit 26. The count unit 26a then counts the number of ON events and OFF events, and supplies the count number to the threshold control unit 26b. With this configuration, the address event detection unit 33 of the light-shielded pixel can supply a detection signal including event information to the setting unit 26 via a route different from that of the address event detection unit 33 of the light-receiving region 21b.

[0048] [Pixel circuit configuration example] 4 is a circuit diagram showing an example of a circuit configuration of the pixel 30. As described above, each of the pixels 30 includes a light receiving unit 31, a pixel signal generating unit 32, and an address event detecting unit 33.

[0049] In the pixel 30 having the above configuration, the light receiving section 31 includes a light receiving element (photoelectric conversion element) 311, a transfer transistor 312, and an OFG (Over Flow Gate) transistor 313. For example, an N-type MOS (Metal Oxide Semiconductor) transistor is used as the transfer transistor 312 and the OFG transistor 313. The transfer transistor 312 and the OFG transistor 313 are connected in series with each other.

[0050] The light receiving element 311 in the light receiving region 21b is connected between a common connection node N1 of the transfer transistor 312 and the OFG transistor 313 and ground, and performs photoelectric conversion on the incident light to generate a charge whose amount corresponds to the amount of the incident light.

[0051] On the other hand, the light receiving element 311 of the shaded pixel is connected between a common connection node N1 of the transfer transistor 312 and the OFG transistor 313 and ground, and generates an amount of charge corresponding to the noise component. The transfer transistor 312 of the light receiving element 311 of the shaded pixel is always in an OFF state, and the supply of charge to the pixel signal generation unit 32 is stopped. For this reason, the shaded pixel 30 may be configured without the transfer transistor 312 and the pixel signal generation unit 32.

[0052] A transfer signal TRG is supplied to a gate electrode of the transfer transistor 312 in the light receiving region 21b from the drive unit 22 shown in Fig. 2. In response to the transfer signal TRG, the transfer transistor 312 supplies the charge photoelectrically converted by the light receiving element 311 to the pixel signal generation unit 32.

[0053] A control signal OFG is supplied to a gate electrode of the OFG transistor 313 from the driving unit 22. In response to the control signal OFG, the OFG transistor 313 supplies an electrical signal generated by the light receiving element 311 to the address event detection unit 33. The electrical signal supplied to the address event detection unit 33 is a photocurrent consisting of electric charges.

[0054] The pixel signal generating unit 32 includes a reset transistor 321, an amplifying transistor 322, a selecting transistor 323, and a floating diffusion layer 324. As the reset transistor 321, the amplifying transistor 322, and the selecting transistor 323, for example, an N-type MOS transistor is used.

[0055] The pixel signal generation unit 32 is supplied with charges photoelectrically converted by the light receiving element 311 from the light receiving unit 31 via the transfer transistor 312. The charges supplied from the light receiving unit 31 are accumulated in the floating diffusion layer 324. The floating diffusion layer 324 generates a voltage signal with a voltage value according to the amount of accumulated charge. In other words, the floating diffusion layer 324 converts the charges into a voltage.

[0056] The reset transistor 321 is connected to the power supply voltage V DDThe reset transistor 321 is connected between the power supply line and the floating diffusion layer 324. A reset signal RST is supplied to a gate electrode of the reset transistor 321 from the driving unit 22. The reset transistor 321 initializes (resets) the amount of charge in the floating diffusion layer 324 in response to the reset signal RST.

[0057] The amplifier transistor 322 is connected to the power supply voltage V DD Between the power supply line and the vertical signal line VSL, the amplifier transistor 322 is connected in series with the selection transistor 323. The amplifier transistor 322 amplifies the voltage signal converted from the charge to the voltage by the floating diffusion layer 324.

[0058] A selection signal SEL is supplied to a gate electrode of the selection transistor 323 from the driving unit 22. In response to the selection signal SEL, the selection transistor 323 outputs the voltage signal amplified by the amplification transistor 322 as a pixel signal SIG to the column processing unit 24 (see FIG. 2) via the vertical signal line VSL.

[0059] In an imaging device 20 having a pixel array section 21 in which pixels 30 having the above-mentioned configuration are arranged two-dimensionally, when the control section 13 shown in FIG. 1 instructs the drive section 22 to start detecting an address event in the light-receiving area 21b, the drive section 22 supplies a control signal OFG to the OFG transistor 313 of the light-receiving section 31, thereby driving the OFG transistor 313 to supply a photocurrent to the address event detection section 33.

[0060] On the other hand, when the drive unit 22 is instructed by the control unit 13 shown in Figure 1 to start detecting an address event of a shaded pixel, the drive unit 22 drives the OFG transistor 313 of the light receiving unit 31 of the shaded pixel by supplying a control signal OFG to the OFG transistor 313, thereby causing the OFG transistor 313 to supply an output current to the address event detection unit 33.

[0061] Then, when an address event is detected in a pixel 30 in the light receiving region 21b, the drive unit 22 turns off the OFG transistor 313 of that pixel 30 to stop the supply of photocurrent to the address event detection unit 33. Next, the drive unit 22 drives the transfer transistor 312 by supplying a transfer signal TRG to the transfer transistor 312, and causes the charge photoelectrically converted in the light receiving element 311 to be transferred to the floating diffusion layer 324.

[0062] In this way, the imaging device 20 having the pixel array unit 21 in which the pixels 30 having the above configuration are arranged two-dimensionally outputs only pixel signals of the pixels 30 in the light receiving region 21b in which an address event has been detected to the column processing unit 24. On the other hand, the light-shielded pixels 30 are used only for detecting an address event. This makes it possible to reduce the power consumption of the imaging device 20 and the amount of image processing compared to a case in which pixel signals of all pixels are output regardless of the presence or absence of an address event.

[0063] It should be noted that the configuration of the pixel 30 illustrated here is merely an example, and is not limited to this configuration example. For example, a pixel configuration may be adopted that does not include the pixel signal generating unit 32. In the case of this pixel configuration, the OFG transistor 313 may be omitted in the light receiving unit 31, and the function of the OFG transistor 313 may be provided to the transfer transistor 312.

[0064] [First configuration example of the address event detector] Fig. 5 is a block diagram showing a first configuration example of the address event detection unit 33 of the light receiving region 21b. Fig. 6 is a block diagram showing a first configuration example of the address event detection unit 33 of the light-shielded pixel. As shown in Figs. 5 and 6, the address event detection unit 33 in this configuration example has a current-voltage conversion unit 331, a buffer 332, a subtractor 333, a quantizer 334, and a transfer unit 335.

[0065] The current-voltage converter 331 converts the photocurrent from the light receiving unit 31 of the pixel 30 into a logarithmic voltage signal. The current-voltage converter 331 supplies the converted voltage signal to a buffer 332. The buffer 332 buffers the voltage signal supplied from the current-voltage converter 331 and supplies it to a subtractor 333.

[0066] The subtractor 333 is supplied with a row drive signal from the drive unit 22. The subtractor 333 reduces the level of the voltage signal supplied from the buffer 332 in accordance with the row drive signal. The subtractor 333 then supplies the voltage signal after the level reduction to the quantizer 334. The quantizer 334 quantizes the voltage signal supplied from the subtractor 333 into a digital signal and outputs it to the transfer unit 335 as an address event detection signal.

[0067] 5, the transfer unit 335 of the light-receiving region 21b transfers the address event detection signal supplied from the quantizer 334 to the arbiter unit 23 and the like. When an address event is detected, the transfer unit 335 supplies a request to the arbiter unit 23 to request transmission of the address event detection signal. Then, when the transfer unit 335 receives a response to the request from the arbiter unit 23, it supplies the address event detection signal to the drive unit 22 and the signal processing unit 25.

[0068] On the other hand, as shown in FIG. 6, the transfer unit 335 of the light-shielded pixel transfers the address event detection signal supplied from the quantizer 334 to the setting unit 26.

[0069] Next, a configuration example of the current-voltage converter 331, the subtractor 333, and the quantizer 334 in the address event detector 33 will be described.

[0070] (Example of current-voltage conversion section configuration) Fig. 7 is a circuit diagram showing an example of the configuration of the current-voltage conversion section 331 in the address event detection section 33. As shown in Fig. 7, the current-voltage conversion section 331 according to this example has a circuit configuration including an N-type transistor 3311, a P-type transistor 3312, and an N-type transistor 3313. As these transistors 3311 to 3313, for example, MOS transistors are used.

[0071] The N-type transistor 3311 is connected to the power supply voltage V DD The P-type transistor 3312 and the N-type transistor 3313 are connected between the power supply line of the power supply voltage V DD 5 and 6. The P-type transistor 3312 and the N-type transistor 3313 are connected in series between the power supply line and the ground. A common connection node N2 of the P-type transistor 3312 and the N-type transistor 3313 is connected to the gate electrode of the N-type transistor 3311 and the input terminal of the buffer 332 shown in FIG.

[0072] A predetermined bias voltage V bias is applied to the P-type transistor 3312. As a result, the P-type transistor 3312 supplies a constant current to the N-type transistor 3313. A photocurrent is input to the gate electrode of the N-type transistor 3313 from the light receiving section 31 through the signal input line 3314.

[0073] The drain electrodes of the N-type transistor 3311 and the N-type transistor 3313 are connected to the power supply side, and such a circuit is called a source follower. The photocurrent from the light receiving section 31 is converted into a logarithmic voltage signal by these two source followers connected in a loop.

[0074] (Example of subtractor and quantizer configuration) FIG. 8 is a circuit diagram showing an example of the configuration of the subtractor 333 and the quantizer 334 in the address event detection unit 33. As shown in FIG.

[0075] The subtractor 333 according to the present example includes a capacitance element 3331 , an inverter circuit 3332 , a capacitance element 3333 , and a switch element 3334 .

[0076] One end of the capacitance element 3331 is connected to the output terminal of the buffer 332 shown in Fig. 5 and Fig. 6, and the other end is connected to the input terminal of the inverter circuit 3332. The capacitance element 3333 is connected in parallel to the inverter circuit 3332. The switch element 3334 is connected between both ends of the capacitance element 3333. A row drive signal is supplied from the drive unit 22 to the switch element 3334 as an opening / closing control signal. The switch element 3334 opens and closes a path connecting both ends of the capacitance element 3333 in response to the row drive signal. The inverter circuit 3332 inverts the polarity of a voltage signal input via the capacitance element 3331.

[0077] In the subtractor 333 having the above configuration, when the switch element 3334 is turned on (closed), a voltage signal V init is input, and the terminal on the opposite side becomes a virtual ground terminal. For convenience, the potential of this virtual ground terminal is set to zero. At this time, the charge Q init is expressed by the following equation (1), where C1 is the capacitance of the capacitance element 3331. On the other hand, since both ends of the capacitance element 3333 are short-circuited, the stored charge therein is zero. Q init =C1×V init (1)

[0078] Next, the switch element 3334 is turned off (open), and the voltage of the terminal of the capacitance element 3331 on the buffer 332 side changes to V after If we consider the case where the charge Q after is expressed by the following equation (2). Q after =C1×V after (2)

[0079] On the other hand, the charge Q2 stored in the capacitive element 3333 is expressed as C2, and the output voltage is V out Then, it is expressed by the following equation (3). Q2=-C2×V out (3)

[0080] At this time, since the total charge amount of the capacitance element 3331 and the capacitance element 3333 does not change, the following equation (4) is established. Q init =Q after +Q2···(4)

[0081] By substituting equations (1) to (3) into equation (4) and rearranging it, the following equation (5) is obtained. V out =-(C1 / C2)×(V after -V init ) ···(5)

[0082] Equation (5) represents the subtraction operation of the voltage signal, and the gain of the subtraction result is C1 / C2. Since it is usually desired to maximize the gain, it is preferable to design C1 large and C2 small. On the other hand, if C2 is too small, kTC noise increases and noise characteristics may deteriorate, so the reduction in the capacitance of C2 is limited to a range in which noise can be tolerated. In addition, since an address event detection unit 33 including a subtractor 333 is mounted on each pixel 30, there are area restrictions on the capacitance elements 3331 and 3333. Taking these factors into consideration, the capacitance values ​​C1 and C2 of the capacitance elements 3331 and 3333 are determined.

[0083] 8, the quantizer 334 includes a comparator 3341. The comparator 3341 receives the output signal of the inverter circuit 3332, i.e., the voltage signal from the subtractor 430, as a non-inverting (+) input, and calculates a predetermined threshold voltage V th The comparator 3341 receives the voltage signal from the subtractor 430 and a predetermined threshold voltage V th and outputs a signal indicating the comparison result to the transfer unit 335 as an address event detection signal.

[0084] [Second configuration example of the address event detector] Fig. 9 is a block diagram showing a second configuration example of the address event detection unit 33 of the light receiving region 21b. Fig. 10 is a block diagram showing a second configuration example of the address event detection unit 33 of the light-shielded pixel. As shown in Figs. 9 and 10, the address event detection unit 33 according to this configuration example has a storage unit 336 and a control unit 337 in addition to a current-voltage conversion unit 331, a buffer 332, a subtractor 333, a quantizer 334, and a transfer unit 335.

[0085] The storage unit 336 is provided between the quantizer 334 and the transfer unit 335, and accumulates the output of the quantizer 334, i.e., the comparison result of the comparator 3341, based on a sample signal supplied from the control unit 337. The storage unit 336 may be a sampling circuit such as a switch, a plastic, or a capacitor, or may be a digital memory circuit such as a latch or a flip-flop.

[0086] The control unit 337 applies a predetermined threshold voltage V th The threshold voltage V supplied from the control unit 337 to the comparator 3341 is th For example, the control unit 337 may set a threshold voltage V corresponding to an ON event indicating that the amount of change in the photocurrent has exceeded the upper threshold. th1 , and a threshold voltage V corresponding to an off-event indicating that the change amount falls below the lower threshold th2 By supplying these at different timings, a single comparator 3341 can detect multiple types of address events.

[0087] For example, the storage unit 336 receives a threshold voltage V corresponding to an off event from the control unit 337 to an inverting (-) input terminal of the comparator 3341. th2 During the period when the on-event is being th1may be stored. The storage unit 336 may be provided inside the pixel 30 or outside the pixel 30. The storage unit 336 is not an essential component of the address event detection unit 33. That is, the storage unit 336 may be omitted.

[0088] [Imaging device according to the second configuration example (scanning method)] The imaging device 20 according to the first configuration example described above is an asynchronous imaging device that reads out events by an asynchronous readout method. However, the event readout method is not limited to the asynchronous readout method, and may be a synchronous readout method. An imaging device to which the synchronous readout method is applied is a scanning imaging device, the same as a normal imaging device that captures images at a predetermined frame rate.

[0089] FIG. 11 is a block diagram showing an example of the configuration of an imaging device according to the second configuration example, that is, a scanning type imaging device, which is used as the imaging device 20 in the imaging system 10 to which the technology according to the present disclosure is applied.

[0090] As shown in FIG. 11, an imaging device 20 according to a second configuration example of an imaging device of the present disclosure includes a pixel array section 21, a driving section 22, a signal processing section 25, a setting section 26, a readout area selection section 27, and a signal generation section 28.

[0091] The pixel array unit 21 includes a plurality of pixels 30. The plurality of pixels 30 output an output signal in response to a selection signal from the readout region selection unit 27. Each of the plurality of pixels 30 may be configured to have a quantizer within the pixel, for example, as shown in FIG. 6. The plurality of pixels 30 output an output signal corresponding to the amount of change in light intensity. The plurality of pixels 30 may be two-dimensionally arranged in a matrix, as shown in FIG. 11.

[0092] 2, the pixel array section 21 is provided with a light-shielding region 21a that blocks light, and a light-receiving region 21b other than the light-shielding region 21a. The light-shielding region 21a corresponds to at least one of the arrangement of the pixels 30 in rows and the arrangement of the pixels 30 in columns. The light-shielding region 21a is provided at least in one of the upper end, the lower end, the left end, and the right end of the pixel array section 21. In this way, the light-shielding region 21a is provided so as to cover the pixels 30 at the end of the pixel array section 21. The configuration of the pixels 30 in the light-shielding region 21a may be the same as that of the pixels 30 other than the light-receiving region 21b, except that light is not incident thereon. As a result, the pixels 30 in the light-shielding region 21a detect the presence or absence of an address event based on whether or not the amount of change in current due to noise exceeds a predetermined threshold.

[0093] The driving unit 22 drives each of the multiple pixels 30 to output a pixel signal generated in each pixel 30 to the signal processing unit 25. The driving unit 22 and the signal processing unit 25 are circuit units for acquiring gradation information. Therefore, when only event information is acquired, the driving unit 22 and the signal processing unit 25 may not be required.

[0094] The readout region selection unit 27 selects a portion of the multiple pixels 30 included in the pixel array unit 21. For example, the readout region selection unit 27 selects one or more rows among the rows included in the two-dimensional matrix structure corresponding to the pixel array unit 21. The readout region selection unit 27 sequentially selects one or more rows according to a preset period. The readout region selection unit 27 may also determine the selection region in response to a request from each pixel 30 in the pixel array unit 21.

[0095] The signal generating unit 28 generates an event signal corresponding to an active pixel that has detected an event among the selected pixels, based on the output signal of the pixel selected by the readout region selecting unit 27. An event is an event in which the intensity of light changes. An active pixel is a pixel in which the amount of change in the intensity of light corresponding to the output signal exceeds a first threshold value or falls below a second threshold value that is set in advance. For example, the signal generating unit 28 detects a first detection signal when the absolute value of the amount of change in the direction in which the signal level of the electrical signal increases exceeds the first threshold value, and detects a second detection signal when the absolute value of the amount of change in the direction in which the signal level of the electrical signal decreases exceeds the second threshold value. Then, the signal generating unit 28 regards the pixel that has detected the first detection signal or the second detection signal as an active pixel, and generates an event signal corresponding to the active pixel.

[0096] Signal generation unit 28 may be configured to include, for example, a column selection circuit that arbitrates signals coming into signal generation unit 28. Signal generation unit 28 may also be configured to output not only information about active pixels that detect an event, but also information about inactive pixels that do not detect an event.

[0097] The signal generation unit 28 outputs, through the output line 15, address information and time stamp information (e.g., (X, Y, Ton), (X, Y, Toff) of the active pixel in which an event has been detected. However, the data output from the signal generation unit 28 may be not only address information and time stamp information, but also frame format information (e.g., (0, 0, 1, 0, . . .)). Ton indicates the time when an on-event was detected, and Toff indicates the time when an off-event was detected.

[0098] The counting section 26a of the setting section 26 counts the number of on-events and off-events within a predetermined period using address information and timestamp information (e.g., (X, Y, Ton) and (X, Y, Toff) of the active pixels, and supplies the count numbers to the threshold control section 26b. The counting section 29a may count either the on-events or the off-events, and supply the count number to the threshold control section 26b.

[0099] [Example of chip structure] The chip (semiconductor integrated circuit) structure of the imaging device 20 according to the first or second configuration example described above may be, for example, a stacked chip structure. Fig. 12 is an exploded perspective view showing an outline of the stacked chip structure of the imaging device 20.

[0100] As shown in Fig. 12, the stacked chip structure, or so-called stacked structure, is a structure in which at least two chips, a light receiving chip 201 which is a first chip, and a detection chip 202 which is a second chip, are stacked. In the circuit configuration of the pixel 30 shown in Fig. 4, each light receiving element 311 is disposed on the light receiving chip 201, and all elements other than the light receiving element 311 and elements of other circuit parts of the pixel 30 are disposed on the detection chip 202. The light receiving chip 201 and the detection chip 202 are electrically connected through connecting parts such as vias (VIAs), Cu-Cu joints, and bumps.

[0101] Here, an example of a configuration in which the light receiving element 311 is arranged on the light receiving chip 201, and elements other than the light receiving element 311 and elements of other circuit parts of the pixel 30 are arranged on the detection chip 202 has been given, but the present invention is not limited to this example configuration.

[0102] 4, the elements of the light receiving unit 31 may be arranged on the light receiving chip 201, and elements other than the light receiving unit 31 and elements of other circuit parts of the pixel 30 may be arranged on the detection chip 202. Alternatively, the elements of the light receiving unit 31 and the reset transistor 321 and floating diffusion layer 324 of the pixel signal generation unit 32 may be arranged on the light receiving chip 201, and the other elements may be arranged on the detection chip 202. Furthermore, some of the elements constituting the address event detection unit 33 may be arranged on the light receiving chip 201 together with the elements of the light receiving unit 31.

[0103] [Example of column processing section configuration] 13 is a block diagram showing an example of the configuration of the column processing unit 24 of the imaging device 20 according to the first configuration example. As shown in FIG 13, the column processing unit 24 according to this example has a configuration including a plurality of analog-to-digital converters (ADC) 241 arranged for each pixel column of the pixel array unit 21.

[0104] Although the configuration example in which the analog-digital converters 241 are arranged in one-to-one correspondence with the pixel columns of the pixel array unit 21 has been exemplified here, the present invention is not limited to this configuration example. For example, the analog-digital converters 241 may be arranged for each of a plurality of pixel columns, and the analog-digital converters 241 may be used in a time-division manner between the plurality of pixel columns.

[0105] The analog-digital converter 241 converts the analog pixel signal SIG supplied via the vertical signal line VSL into a digital signal having a larger number of bits than the address event detection signal described above. For example, if the address event detection signal is 2 bits, the pixel signal is converted into a digital signal having 3 bits or more (e.g., 16 bits). The analog-digital converter 241 supplies the digital signal generated by the analog-digital conversion to the signal processing unit 25.

[0106] [About noise events] The imaging device 20 is an imaging device that includes, for each pixel 30, a detection unit (that is, an address event detection unit 33) that detects in real time, for each pixel address, that the amount of light at that pixel has exceeded a predetermined threshold value as an address event.

[0107] In this imaging device, when some event (i.e., a true event) occurs in a scene, data related to the occurrence of the true event is acquired. However, even in a scene in which no true event occurs, data may be acquired wastefully due to a noise event (false event) such as sensor noise. This not only results in a noise signal being read out, but also reduces the throughput of the signal output. The imaging device 20 of the present disclosure described below adjusts the sensitivity to true events and false events according to the number of events in the light-shielded pixels.

[0108] 14 is a schematic diagram showing the relationship between noise events and temperature. The horizontal axis indicates time, and the vertical axis indicates the number of events and temperature. Line L12A indicates the change in temperature over time, and line L12B indicates the change in the number of dark events, which are noise events in the dark (when light is blocked) over time. The pixel 30 outputs a dark current by performing photoelectric conversion even in the dark. This is counted as a dark event.

[0109] As shown in Fig. 14, the time series change in the number of dark events correlates with, for example, the time series change in temperature. For example, lowering the sensitivity to noise events reduces the number of noise events, but the detection sensitivity of true events also decreases. On the other hand, increasing the detection sensitivity of true events also increases the sensitivity to noise events.

[0110] For this reason, the threshold control unit 26b adjusts the sensitivity to true events and the sensitivity to false events caused by dark noise by adjusting the sensitivity according to the state of the imaging device 20. This will be described in more detail below.

[0111] 15 is a flowchart showing an example of processing by the threshold control unit 26b of the setting unit 26. Here, an example will be described in which the sensitivity is adjusted using the light-shielded pixels 30 in the light-shielded region 21a, and then an event is detected by the pixels 30 in the light-receiving region 21b. As described above, the asynchronous imaging device according to the first configuration example and the imaging device according to the second configuration example (scanning type) have different methods for reading out event detection signals, but the threshold setting method of the asynchronous imaging device according to the first configuration example and the threshold setting method of the imaging device according to the second configuration example (scanning type) can perform equivalent processing.

[0112] As shown in FIG. 15, first, the threshold control unit 26b sets a threshold in accordance with the noise level of the light-shielded pixels 30 in the light-shielded region 21a (step S1000).

[0113] Next, the address event detector 33 of the light-receiving region 21b detects a detection signal when the amount of change in the electrical signal generated by the pixel 30 of the light-receiving region 21b exceeds the threshold set in the threshold controller 26b (step S1100).

[0114] Next, threshold control unit 26b determines whether or not to end the overall process (step S1200), and if not (NO in step S1200), repeats the process from step S102. On the other hand, if to end (YES in step S1200), the overall process ends.

[0115] In this way, the threshold control section 26b sets the threshold in accordance with the noise level of the light-shielded pixels 30 in the light-shielded region 21a, and therefore it is possible to adjust the sensitivity to true events and dark noise events.

[0116] Fig. 16 is a flowchart showing an example of processing by the threshold controller 26b in parallel processing. As shown in Fig. 16, setting of a threshold according to the noise level of the light-shielded pixels 30 (step S1000) and detection processing by the address event detector 33 of the light-receiving region 21b (step S1100) are performed in parallel.

[0117] In this way, by performing the processes in parallel, it is possible to adjust the sensitivity to true events and dark noise events without stopping event detection by the address event detector 33 of the light receiving region 21b.

[0118] FIG. 17 is a flowchart showing an example of the process of step S1000 in FIGS. FIG. 18 is a diagram showing a typical example of the process shown in FIG. 17. The horizontal axis indicates time, and the vertical axis indicates thresholds. As shown in FIG. 18, the amount of change of the first threshold for an ON event and the second threshold for an OFF event relative to the reference level (AZ level) is shown typically. Here, the absolute values ​​of the amount of change of the first threshold for an ON event and the second threshold for an OFF event relative to the reference level (AZ level) are treated as being equal. For example, when the first threshold is increased by Vh1 volts from the initial setting, the second threshold is decreased by Vh1 volts from the initial setting.

[0119] As shown in FIG. 17, the threshold control section 26b controls the first threshold voltage V th1 , and the second threshold voltage V th2 is supplied to the inverting (-) input terminal of the comparator 3341 (see FIGS. 8 and 10) of the light-shielded pixel 30 and the light-receiving region 21b (step S100). th1 an event detection signal corresponding to an on-event indicating that the second threshold voltage V th2 The setting unit 26 outputs an event detection signal corresponding to an OFF event indicating that the threshold value has exceeded the threshold value to the counting unit 26a of the setting unit 26. The counting unit 26a counts the number of events that have occurred within a predetermined period and supplies the counted number to the threshold control unit 26b (step S102). In the case of the parallel processing shown in FIG. 16, the specified period of step S102 shown in FIG. 17 does not have to be the entire shooting period, and may alternate between a specified period in which the number of events of light-shielded pixels is counted and a period in which the number of events of light-shielded pixels is not counted.

[0120] In the case of the imaging device (scanning type) according to the second configuration example, the readout region selection unit 27 selects a part of the multiple pixels 30 included in the light-shielded region 21a. For example, the readout region selection unit 27 selects one or more rows among the rows included in the two-dimensional matrix structure corresponding to the light-shielded region 21a. The readout region selection unit 27 (see FIG. 11) sequentially selects one or more rows according to a preset period, reads out address information and time stamp information (e.g., (X, Y, Ton), (X, Y, Toff)) of the active pixels of the light-shielded pixels 30 in turn, and supplies the address information and time stamp information to the count unit 26a via the signal generation unit 28. The count unit 26a of the setting unit 26 counts the number of on-events and off-events within a predetermined period using the address information and time stamp information (e.g., (X, Y, Ton), (X, Y, Toff)) of the active pixels, and supplies the count number to the threshold control unit 26b.

[0121] Next, the threshold control unit 26b determines whether the number of counted events (number of ON events+number of OFF events) within a predetermined time is within a predetermined range (step S104). If the number of events is outside the predetermined range (YES in step S104), the threshold control unit 26b further determines whether the number of counted events is equal to or greater than a predetermined value (step S106). If the number of events is equal to or greater than the predetermined value (YES in step S106), the threshold control unit 26b lowers the sensitivity of the threshold (step S108). That is, the threshold control unit 26b applies the first threshold voltage V th1 1a volts to obtain the second threshold voltage V th2 The voltage is decreased by a predetermined value Th1a volts.

[0122] On the other hand, if the number of events is less than the predetermined value (NO in step S106), the threshold control unit 26b increases the sensitivity of the threshold. th1 The second threshold voltage V th2 The voltage is increased by a predetermined value Th2a volts (step S110).

[0123] On the other hand, if the number of events is less than the predetermined value (NO in step S104), the threshold sensitivity is maintained. That is, the threshold control unit 26b maintains the first threshold voltage V th1 and the second threshold voltage V th2 is not changed (step S112).

[0124] Fig. 19 is a diagram showing an example of a process in which the process shown in Fig. 17 is performed for three cycles. The horizontal axis indicates time, and the vertical axis indicates thresholds. As shown in Fig. 19, it is possible to vary the first threshold for an ON event and the second threshold for an OFF event over time.

[0125] In this way, the absolute value of the amount of change from the reference level of the electrical signal generated by the photoelectric conversion element 311 in the shielded region 21a is equal to or lower than the threshold (first threshold voltage V th1 or the second threshold voltage V th2 ) is exceeded, a threshold (first threshold voltage V th1 or the second threshold voltage V th2 ) is set. This enables threshold control according to the number of dark noise events occurring in the light-shielded pixels 30 in the light-shielded region 21a, and allows sensitivity adjustment to true events and dark noise events.

[0126] Fig. 20 is a flowchart showing a process example for independently adjusting the first threshold for an ON event and the second threshold for an OFF event. As shown in Fig. 20, first, the threshold control unit 26b sets the first threshold according to the ON event of the light-shielded pixel 30 in the light-shielded region 21a (step S1000a), and in parallel sets the second threshold according to the level of the OFF event of the light-shielded pixel 30 in the light-shielded region 21a (step S1000b).

[0127] Next, the address event detection unit 33 of the light receiving region 21b detects an on-event detection signal and an off-event detection signal when the change in the electrical signal generated in the pixel 30 of the light receiving region 21b exceeds a first threshold or a second threshold independently set in the threshold control unit 26b (step S1100).

[0128] 21 is a diagram illustrating an example of a process for independently adjusting a first threshold for an ON-event and a second threshold for an OFF-event, in which the horizontal axis indicates time and the vertical axis indicates thresholds.

[0129] In this way, the threshold control unit 26b independently sets the first threshold and the second threshold according to the noise level of the light-shielded pixels 30 in the light-shielded region 21a. This allows sensitivity adjustment to be performed for true events in which the absolute value of the amount of change in the direction in which the signal level of the electrical signal increases exceeds the first threshold and for dark noise events corresponding to ON events. At the same time, sensitivity adjustment can be performed for true events in which the absolute value of the amount of change in the direction in which the signal level of the electrical signal decreases exceeds the second threshold and for dark noise events corresponding to OFF events. In this way, since the first threshold and the second threshold can be independently adjusted, more appropriate sensitivity adjustment is possible even when there is a statistical bias in the occurrence distribution of dark noise events corresponding to ON events and dark noise events corresponding to OFF events.

[0130] Fig. 22 is a flowchart showing a processing example in which the first threshold and the second threshold in the parallel processing are independently adjusted. As shown in Fig. 21, the threshold control unit 26b performs a process of setting the first threshold according to the level of an ON event of the light-shielded pixel 30 in the light-shielded region 21a (step S1000a), a process of setting the second threshold according to the level of an OFF event of the light-shielded pixel 30 in the light-shielded region 21a (step S1000b), and a detection process (step S1100) of the address event detection unit 33 in the light-receiving region 21b in parallel.

[0131] In this way, by performing the processes in parallel, the first threshold value and the second threshold value can be adjusted independently without stopping event detection by the address event detector 33 of the light receiving region 21b.

[0132] FIG. 23 is a flow chart showing an example of gradually changing the threshold in the threshold setting process (step S1000). FIG. 24 is a diagram showing a schematic diagram of the processing example shown in FIG. 23. The horizontal axis indicates time, and the vertical axis indicates thresholds. As shown in FIG. 20, the amount of change of the first threshold for an ON event and the second threshold for an OFF event with respect to the reference level (AZ level) is changed stepwise in a time series. For example, the threshold is changed in several steps until it reaches the threshold that is the expected convergence value. Here, when the expected convergence value is the AZ level + a predetermined value Vhf, the predetermined value Th1 is changed to the AZ level + Vhf × 0.8, the predetermined value Th2 is changed to the AZ level + Thf × 0.96, and the like, so as to asymptotically approach the expected convergence value of the threshold with the passage of time.

[0133] As shown in FIG. 23, the threshold control unit 26b determines whether the number of events counted within a predetermined time period for the initially set threshold Th0 (the number of ON events+the number of OFF events) is equal to or greater than a predetermined value (step S104). If the number of events is equal to or greater than the predetermined value (YES in step S104), the threshold control unit 26b gradually lowers the sensitivity of the threshold (step S108a). That is, in the first period, the threshold control unit 26b applies the first threshold voltage V th1 is increased by a predetermined value Th1 volts from the reference level to obtain the second threshold voltage V th2 In the next, second period, the first threshold voltage V th1 is increased by a predetermined value Th2 volts from the reference level to obtain the second threshold voltage V th2 is decreased by a predetermined value Th2 volts from the reference level. In this case, the absolute value of (Th2-Th1) is smaller than the absolute value of (Th1-Th0).

[0134] On the other hand, if the number of events is less than the predetermined value (NO in step S104), threshold control unit 26b does not change the current threshold and ends the entire process.

[0135] In this way, since the threshold is changed stepwise, it is possible to set the sensitivity close to the target number of dark noise events. Also, since the threshold is changed stepwise, it is possible to change the threshold while suppressing the change in the number of events occurring per unit time in the pixels 30 of the light receiving region 21b.

[0136] Fig. 25 is a flow chart showing an example of gradually changing the first threshold in the threshold setting process (step S1000a). As shown in Fig. 25, the on-events of light-shielded pixels are counted (step S102a), and if the count is not within a predetermined range, the first threshold is gradually changed (step S110a). In this way, even when the first threshold and the second threshold are adjusted independently, the process of gradually changing the first threshold can be performed.

[0137] FIG. 26 is a diagram showing a schematic example of changing the first threshold in stages. The horizontal axis indicates time, and the vertical axis indicates the first threshold. As shown in FIG. 26, the initially set first threshold Th0a is changed to the first threshold Th1a in the first stage, and then changed to the first threshold Th2a in the second stage. In this way, the first threshold is roughly adjusted, and then gradually converged, thereby making it possible to hasten convergence while suppressing divergence. The process shown in FIG. 26 can also be applied to the threshold setting process (step S1000).

[0138] Fig. 27 is a flowchart showing an example of gradually changing the second threshold in the threshold setting process (step S1000b). As shown in Fig. 27, the off-events of the light-shielded pixels are counted (step S102b), and if they are not within a predetermined range, the second threshold is gradually changed (step S110b). In this way, even when the first threshold and the second threshold are adjusted independently, it is possible to perform the process of gradually changing the second threshold.

[0139] FIG. 28 is a diagram showing a schematic example of changing the second threshold in stages. The horizontal axis indicates time, and the vertical axis indicates the first threshold. As shown in FIG. 28, the initially set first threshold Th0b is changed to the first threshold Th1b in the first stage, and then changed to the first threshold Th2b in the second stage. In this way, the second threshold is roughly adjusted, and then gradually converged, thereby making it possible to hasten convergence while suppressing divergence. The process shown in FIG. 28 can also be applied to the threshold setting process (step S1000).

[0140] Fig. 29 is a flowchart showing an example in which a dead zone is provided in the threshold setting process (step S1000b). Fig. 30 is a diagram showing a schematic diagram of an example in which the second threshold in which the dead zone is provided is changed in stages. The horizontal axis indicates time, and the vertical axis indicates the second threshold.

[0141] Here, the dead zone means a threshold value set above and below the second threshold value Th0b, Th1b, Th2b, etc. For example, the dead zone is 10 percent before and after the absolute value of the difference between the AZ level and the second threshold value Th0b. More specifically, if the AZ level is, for example, 100 and the second threshold value Th0b is 50, the absolute value of the difference between the AZ level and the second threshold value Th0b is (100-50), and 10 percent before and after the absolute value of the difference is 50×0.1=±5. That is, the dead zone is in the range of 45 to 55. For example, when changing the threshold value of the second threshold value Th0b, if the second threshold value Th1b is within the range of the dead zone, the threshold value is not changed, and if it is outside the dead zone, the threshold value is changed. In FIG. 30, since the second threshold value Th1b is outside the range of the dead zone of the second threshold value Th0b, the value of the second threshold value Th1b is changed from the second threshold value Th0b. On the other hand, since the second threshold value Th2b is within the range of the dead band of the second threshold value Th1b, the value of the second threshold value Th2b is set to the same value as the second threshold value Th1b and is not changed.

[0142] 29, when the number of off events is within a predetermined range (NO in step S104b), the threshold control unit 26b determines whether the value of the second threshold Th1b to be changed is within the dead band of the current second threshold Th0b (step S112b), and when it is outside the dead band (NO in step S112b), changes the value of the second threshold Th1b from the second threshold Th0b. On the other hand, when it is within the dead band (YES in step S112b), the value of the second threshold Th1b is maintained at the value of the second threshold Th0b, and the process ends. In this way, if the amount by which the threshold should be changed is within the dead band, the threshold is not changed. Note that the process shown in Fig. 30 can also be applied to the threshold setting process (steps S1000 and S1000a).

[0143] For example, changing the threshold value may result in the observer perceiving image variation. For this reason, by increasing the range of the dead zone, the threshold value is not changed, and the observer is prevented from perceiving image variation. On the other hand, increasing the range of the dead zone suppresses sensitivity adjustment. For this reason, by adjusting the range of the dead zone, it is possible to balance sensitivity adjustment and image variation.

[0144] Fig. 31 is a flowchart showing an example of changing the amount of change in the threshold in response to temperature change in the threshold setting process (step S1000b). Fig. 32 is a diagram showing a schematic example of changing the amount of change in the threshold in response to temperature change. The horizontal axis indicates time, and the vertical axis indicates the second threshold. As shown in Fig. 32, the amount of change in the second threshold Th1b is increased in an area where the temperature time-series value Tma changes greatly over time.

[0145] 31, when the number of OFF events is within a predetermined range (NO in step S104b), the threshold control unit 26b calculates the amount of change in temperature over time using temperature information corresponding to the pixels 30 of the light receiving region 21b, which is recorded in chronological order in the recording unit 12. For example, the absolute value of the temperature difference over time is calculated (step S116b). The temperatures acquired from the temperature sensor 21c are recorded in chronological order in the recording unit 12.

[0146] Next, the threshold control unit 26b calculates the amount of change in the second threshold based on the amount of change in temperature, and changes the second threshold (step S118b). In this way, the amount of change in the second threshold is adjusted according to the amount of change in temperature over time. As shown in Fig. 14, since temperature change and noise change are correlated, the amount of change in the second threshold can be made large at the timing when the temperature change is large, making it possible to adjust the sensitivity more efficiently.

[0147] As described above, according to this embodiment, the threshold controller 26b sets the threshold in accordance with the noise level of the light-shielded pixels 30 in the light-shielded region 21a. This makes it possible to adjust the amount of dark noise events that occur, thereby making it possible to adjust the sensitivity to true events and dark noise events.

[0148] (Modification of the first embodiment) The imaging system 10 according to the modification of the first embodiment differs from the imaging system 10 according to the first embodiment in that the imaging system 10 has a calculation processing unit 15 that sets a threshold value outside the imaging device 20. The differences from the imaging system 10 according to the first embodiment will be described below.

[0149] Fig. 33 is a block diagram showing a configuration of an imaging device 20 according to a modified example of the first embodiment. In Fig. 33, the driver 22, the arbiter 23, and the column processor 24 are omitted. As shown in Fig. 33, the imaging device 20 has an arithmetic processor 15 outside the imaging device 20. The arithmetic processor 15 is a device that allows the user to change the arithmetic program and processing parameters. This allows the user to set a threshold value using an arbitrary algorithm.

[0150] <Application examples of the technology disclosed herein> The technology according to the present disclosure can be applied to various products. More specific application examples will be described below. For example, the technology according to the present disclosure may be realized as a distance measuring device mounted on any type of moving object such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor).

[0151] [Mobile object] Fig. 34 is a block diagram showing a schematic configuration example of a vehicle control system 7000 which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in Fig. 31, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

[0152] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle by wired communication or wireless communication. In FIG. 34, a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690 are illustrated as the functional configuration of the integrated control unit 7600. Other control units also include a microcomputer, a communication I / F, a storage unit, and the like.

[0153] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle according to various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle. The drivetrain control unit 7100 may also function as a control device such as an ABS (Antilock Brake System) or an ESC (Electronic Stability Control).

[0154] A vehicle state detection unit 7110 is connected to the drive system control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of an accelerator pedal, the amount of operation of a brake pedal, the steering angle of a steering wheel, the engine rotation speed, or the rotation speed of wheels, for example. The drive system control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0155] The body system control unit 7200 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 7200. The body system control unit 7200 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0156] The battery control unit 7300 controls the secondary battery 7310, which is a power supply source for the drive motor, according to various programs. For example, information such as battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs calculation processing using these signals, and controls the temperature regulation of the secondary battery 7310 or controls a cooling device or the like equipped in the battery device.

[0157] The outside-vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside-vehicle information detection unit 7420 is connected to the outside-vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside-vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, and the like around the vehicle equipped with the vehicle control system 7000.

[0158] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may be provided as independent sensors or devices, or may be provided as a device in which multiple sensors or devices are integrated.

[0159] Here, FIG. 35 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield in the vehicle interior of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0160] 35 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, image data captured by the imaging units 7910, 7912, 7914, and 7916 are superimposed to obtain an overhead image of the vehicle 7900.

[0161] The outside information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, and corners of the vehicle 7900 and on the upper part of the windshield inside the vehicle cabin may be, for example, ultrasonic sensors or radar devices. The outside information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and on the upper part of the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, and the like.

[0162] Returning to FIG. 34, the description will be continued. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle, and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the outside-vehicle information detection unit 7420 connected thereto. When the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves, and receives information on the received reflected waves. The outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface, based on the received information. The outside-vehicle information detection unit 7400 may perform environment recognition processing for recognizing rainfall, fog, road surface conditions, and the like, based on the received information. The outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

[0163] Furthermore, the outside vehicle information detection unit 7400 may perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface, based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or position adjustment on the received image data, and may generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410. The outside vehicle information detection unit 7400 may perform viewpoint conversion processing using image data captured by different imaging units 7410.

[0164] The in-vehicle information detection unit 7500 detects information inside the vehicle. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the bioinformation of the driver, or a microphone that collects sound in the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the bioinformation of a passenger sitting in the seat or a driver gripping the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected sound signal.

[0165] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. The input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by a device that can be operated by an occupant to input, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the occupant can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the occupant may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the occupant using the above-mentioned input unit 7800 and outputs the input signal to the integrated control unit 7600. A passenger or the like operates the input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0166] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device, etc.

[0167] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication between various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (registered trademark), WiMAX, LTE (Long Term Evolution), or LTE-Advanced (LTE-A), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or a carrier-specific network) via, for example, a base station or an access point. The general-purpose communication I / F 7620 may also connect to a terminal (e.g., a driver's, pedestrian's, or store's terminal, or a MTC (Machine Type Communication) terminal) present in the vicinity of the vehicle using, for example, a P2P (Peer To Peer) technology.

[0168] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in a vehicle. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE802.11p and a higher layer IEEE1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0169] The positioning unit 7640 performs positioning by receiving, for example, a GNSS signal from a Global Navigation Satellite System (GNSS) satellite (for example, a GPS signal from a Global Positioning System (GPS) satellite) and generates position information including the latitude, longitude, and altitude of the vehicle. The positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone having a positioning function.

[0170] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closure, required time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0171] The in-vehicle device I / F 7660 is a communication interface that mediates a connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). The in-vehicle device I / F 7660 may also establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include at least one of a mobile device or a wearable device owned by a passenger, or an information device carried or attached to a vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle equipment I / F 7660 exchanges control signals and data signals with these in-vehicle equipment 7760.

[0172] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0173] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate a control target value of a driving force generating device, a steering mechanism, or a braking device based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc. based on information acquired about the vehicle's surroundings.

[0174] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including peripheral information of the current position of the vehicle, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. The microcomputer 7610 may also predict dangers such as vehicle collisions, the approach of pedestrians, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0175] The audio / image output unit 7670 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of FIG. 34, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include at least one of an on-board display and a head-up display, for example. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may be other devices such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp, other than these devices. When the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, image, table, graph, etc. When the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced audio data or acoustic data into an analog signal and audibly outputs it.

[0176] In the example shown in FIG. 34, at least two control units connected via the communication network 7010 may be integrated into one control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by any control unit may be provided by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, a predetermined arithmetic processing may be performed by any control unit. Similarly, a sensor or device connected to any control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0177] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the imaging units 7910, 7912, 7914, 7916, and 7918, the outside vehicle information detection units 7920, 7922, 7924, 7926, 7928, and 7930, and the driver state detection unit 7510, among the configurations described above. Specifically, the imaging system 10 of FIG. 1 having the imaging device according to the present disclosure can be applied to these imaging units and detection units. Then, by applying the technology according to the present disclosure, the influence of a noise event such as a sensor noise can be mitigated, and the occurrence of a true event can be reliably and quickly detected, thereby making it possible to realize safe vehicle driving.

[0178] The present technology can be configured as follows.

[0179] (1) a photoelectric conversion unit having a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal; a setting unit that sets a threshold value according to a noise level of a predetermined region of the plurality of photoelectric conversion elements; a first detection unit that detects a detection signal when a change amount of the electrical signal generated by the plurality of photoelectric conversion elements exceeds the threshold; An imaging device comprising:

[0180] (2) the photoelectric conversion elements in the predetermined region are shielded from light; The imaging device according to (1), wherein the photoelectric conversion elements other than the specified region each convert incident light into an electric signal by photoelectric conversion.

[0181] (3) The imaging device described in (2), wherein the multiple photoelectric conversion elements are arranged in a two-dimensional matrix, and the light-shielded area corresponds to at least one of an arrangement of the photoelectric conversion elements in rows and an arrangement of the photoelectric conversion elements in columns.

[0182] (4) The imaging device described in (3), wherein the row-by-row arrangement of the photoelectric conversion elements and the column-by-column arrangement of the photoelectric conversion elements include ends of the multiple photoelectric conversion elements arranged in the two-dimensional matrix.

[0183] (5) The imaging device described in any one of (1) to (4), wherein the setting unit sets the threshold based on the number of times that the absolute value of the change in the electrical signal generated by the photoelectric conversion element in the specified region exceeds the threshold in a specified period.

[0184] (6) The first detection unit detects a first detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal increases exceeds a first threshold, and detects a second detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal decreases exceeds a second threshold; The setting unit is setting the first threshold based on the number of times that an absolute value of a change in a direction in which a signal level of the electrical signal generated by the electrical conversion element in the predetermined region increases exceeds the first threshold in a predetermined period; The imaging device according to any one of (1) to (5), wherein the second threshold is set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electrical signal generated by the electrical conversion element in the specified region decreases exceeds the second threshold in a specified period.

[0185] (7) The imaging device according to any one of (1) to (5), wherein the setting unit sets the threshold in stages based on the number of times the threshold is exceeded in the specified period.

[0186] (8) The imaging device according to (7), wherein the setting unit reduces a rate of change of the threshold value over time.

[0187] (9) The imaging device according to (8), wherein the setting unit reduces a rate of change of the threshold value to make the threshold value gradually approach an initial setting.

[0188] (10) The imaging device according to (8), wherein the setting unit reduces the rate of change after setting the threshold to a first-stage threshold, and causes the rate of change to gradually approach a predetermined set value.

[0189] (11) The imaging device according to any one of (1) to (10), wherein the setting unit does not change the threshold value when the number of times the threshold value is exceeded during the specified period is less than a specified value.

[0190] (12) The imaging device according to any one of (1) to (10), wherein the setting unit sets the threshold value in accordance with temperatures corresponding to the plurality of photoelectric conversion elements.

[0191] (13) The imaging device according to (12), wherein the setting unit increases a rate of change of the threshold as the temperature change increases.

[0192] (14) The first detection unit sequentially reads out the electrical signals of the photoelectric conversion elements in the predetermined region, The imaging device according to any one of (1) to (13), wherein the setting unit counts the number of the detection signals that exceed the threshold value over a predetermined period of time.

[0193] (15) setting a threshold value according to a noise level of the light-shielded photoelectric conversion element; An imaging method comprising: detecting a detection signal when an absolute value of a change in an electrical signal generated by a plurality of photoelectric conversion elements, each of which performs photoelectric conversion on incident light to generate an electrical signal, exceeds the threshold value.

[0194] The aspects of the present disclosure are not limited to the above-mentioned individual embodiments, but include various modifications that may be conceived by a person skilled in the art, and the effects of the present disclosure are not limited to the above-mentioned contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and intent of the present disclosure derived from the contents defined in the claims and their equivalents. [Explanation of symbols]

[0195] 10 imaging system, 20 imaging device, 21 pixel array section, 21a light-shielding region, 21b light-receiving region, 22 drive section, 23 arbiter section, 24 column processing section, 25 signal processing section, 26 setting section, 26a counting section, 26b threshold control section, 30 pixel, 31 light-receiving section, 33 address event detection section.

Claims

1. a photoelectric conversion unit having a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal; a setting unit that sets a threshold value according to a noise level of a predetermined region of the plurality of photoelectric conversion elements; a first detection unit that detects a detection signal when a change amount of the electrical signal generated by the plurality of photoelectric conversion elements exceeds the threshold; Equipped with The setting unit sets the threshold in stages based on the number of times that the absolute value of the amount of change in the electrical signal generated by the photoelectric conversion element in the specified region exceeds the threshold in a specified period.

2. the photoelectric conversion elements in the predetermined region are shielded from light, The imaging device according to claim 1 , wherein the photoelectric conversion elements other than the predetermined region each perform photoelectric conversion on incident light to generate the electrical signal.

3. 3. The imaging device according to claim 2, wherein the plurality of photoelectric conversion elements are arranged in a two-dimensional matrix, and the light-shielded region corresponds to at least one of an arrangement of the photoelectric conversion elements in rows and an arrangement of the photoelectric conversion elements in columns.

4. The imaging device according to claim 3 , wherein the row-by-row arrangement of the photoelectric conversion elements and the column-by-column arrangement of the photoelectric conversion elements include ends of the plurality of photoelectric conversion elements arranged in the two-dimensional matrix.

5. the first detection unit detects a first detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal increases exceeds a first threshold, and detects a second detection signal when an absolute value of an amount of change in a direction in which a signal level of the electrical signal decreases exceeds a second threshold; The setting unit is setting the first threshold based on the number of times that an absolute value of a change in a direction in which a signal level of the electrical signal generated by the electrical conversion element in the predetermined region increases exceeds the first threshold in a predetermined period; 2. The imaging device according to claim 1, wherein the second threshold is set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electrical signal generated by the electrical conversion element in the specified region decreases exceeds the second threshold in a specified period.

6. The imaging device according to claim 1 , wherein the setting unit reduces a rate of change of the threshold value over time.

7. The imaging device according to claim 6 , wherein the setting section reduces a rate of change of the threshold value so as to approach an initial setting.

8. The imaging device according to claim 6 , wherein the setting section reduces the rate of change after setting the threshold to a first-stage threshold, and causes the rate of change to gradually approach a predetermined set value.

9. The imaging device according to claim 1 , wherein the setting unit does not change the threshold value when the number of times the threshold value is exceeded in a predetermined period is less than a predetermined value.

10. The imaging device according to claim 1 , wherein the setting unit sets the threshold value in accordance with temperatures corresponding to the plurality of photoelectric conversion elements.

11. The imaging device according to claim 10 , wherein the setting section increases a rate of change of the threshold value as the temperature change increases.

12. The first detection unit sequentially reads out the electrical signals of the photoelectric conversion elements in the predetermined region, The imaging device according to claim 1 , wherein the setting unit counts the number of the detection signals that exceed the threshold value over a predetermined period of time.

13. A threshold value is set according to a noise level of the light-shielded photoelectric conversion element, An imaging method, comprising: detecting a detection signal when an absolute value of a change in an electrical signal generated by a plurality of photoelectric conversion elements, each of which generates an electrical signal by photoelectrically converting incident light, exceeds a threshold value, The imaging method includes setting the threshold in a stepwise manner based on the number of times that an absolute value of a change in the electrical signal generated by the light-shielded photoelectric conversion element exceeds the threshold in a predetermined period.

Images