Imaging Apparatus and Imaging Method

Abstract

[Problem] There is no possibility of causing an unnecessary event. [Solution] An imaging device includes: a photoelectric conversion unit including a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal; a setting unit that sets a threshold value based on a noise level of a predetermined area of the plurality of photoelectric conversion elements; and a first detection unit that detects a detection signal when an amount of change in the electrical signals generated by the plurality of photoelectric conversion elements exceeds the threshold value.

Description

Technical Field

[0001] The present disclosure relates to an imaging device and an imaging method. Background Art

[0002] As one type of event-driven imaging device, an asynchronous imaging device called a dynamic vision sensor (DVS) has been proposed (see, for example, Patent Document 1). In an asynchronous imaging device, only when a specific event (e.g., movement) occurs in a scene, data is acquired for the portion where the brightness level has changed due to the event. Therefore, compared to conventional synchronous imaging devices that unnecessarily acquire all image data at a fixed frame rate, asynchronous imaging devices can acquire image data at a much higher speed.

[0003] Reference List

[0004] Patent Literature

[0005] Patent Document 1: JP 2017-535999 A Summary of the Invention

[0006] Problems to be solved by the present invention

[0007] Due to its excellent sensitivity, the DVS also responds to changes in the electrical signal caused by noise in the photoelectric conversion elements in the imaging device, and can generate events even in scenes where there is initially no motion. In addition, generating unnecessary events that were initially necessary increases the power consumption of the DVS.

[0008] Therefore, the present disclosure provides an imaging apparatus and an imaging method that do not cause useless events.

[0009] Solution to the problem

[0010] In order to solve the above problems, according to the present disclosure, an imaging device is provided, comprising:

[0011] a photoelectric conversion unit including a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal;

[0012] a setting unit that sets a threshold value according to a noise level of a predetermined area of the plurality of photoelectric conversion elements; and

[0013] The first detection unit detects a detection signal when the amount of change in the electrical signals generated by the plurality of photoelectric conversion elements exceeds a threshold value.

[0014] The photoelectric conversion element in a predetermined area can be shielded from light, and

[0015] The photoelectric conversion elements other than the predetermined area can each photoelectrically convert incident light to generate an electric signal.

[0016] The plurality of photoelectric conversion elements may be arranged in a two-dimensional matrix, and the light shielding region may correspond to at least one of an arrangement in which the photoelectric conversion elements are arranged in row units or an arrangement in which the photoelectric conversion elements are arranged in column units.

[0017] The arrangement of the photoelectric conversion elements in row units and the arrangement of the photoelectric conversion elements in column units may include end portions of the plurality of photoelectric conversion elements arranged in a two-dimensional matrix.

[0018] The setting unit may set the threshold value based on the number of times that the absolute value of the amount of change in the electric signal generated by the photoelectric conversion element in the predetermined area exceeds the threshold value in the predetermined period of time.

[0019] The first detection unit may detect a first detection signal when an absolute value of an amount of change in a direction in which the signal level of the electric signal increases exceeds a first threshold value, and may detect a second detection signal when an absolute value of an amount of change in a direction in which the signal level of the electric signal decreases exceeds a second threshold value, and

[0020] Setting unit,

[0021] The first threshold value may be set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the photoelectric conversion element in the predetermined area increases exceeds the first threshold value in the predetermined period of time, and

[0022] The second threshold value may be set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the photoelectric conversion element in the predetermined area decreases exceeds the second threshold value in the predetermined period of time.

[0023] The setting unit may also set the threshold in stages based on the number of times the threshold is exceeded in a predetermined period of time.

[0024] The setting unit may reduce the rate of change of the threshold value as time passes.

[0025] The setting unit may reduce the rate of change of the threshold value in such a manner as to asymptotically approach the initial setting value.

[0026] After setting the threshold value to the threshold value of the first stage, the setting unit reduces the change rate in such a manner as to gradually approach a predetermined set value.

[0027] The setting unit does not change the threshold value when the number of times the threshold value is exceeded within the predetermined period of time is less than a predetermined value.

[0028] The setting unit may set the threshold value according to temperatures corresponding to the plurality of photoelectric conversion elements.

[0029] The setting unit may increase the rate of change of the threshold value as the temperature change increases.

[0030] The first detection unit may sequentially read the electrical signals of the photoelectric conversion elements in a predetermined area, and

[0031] The setting unit may count the number of detection signals exceeding the threshold value within a predetermined period of time.

[0032] In order to solve the above problems, according to the present disclosure, an imaging method is provided, comprising:

[0033] Setting a threshold value according to a noise level of the photoelectric conversion element that is shielded from light; and

[0034] A detection signal is detected when an absolute value of a change amount of an electric signal generated by a plurality of photoelectric conversion elements, each of which performs photoelectric conversion on incident light to generate an electric signal, exceeds a threshold value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a block diagram illustrating an example of a system configuration of an imaging system to which the technology according to the present disclosure is applied.

[0036] Figure 2 is a block diagram illustrating a configuration example of an imaging device according to a first configuration example of the present disclosure.

[0037] Figure 3 is a block diagram showing an example of the configuration of a pixel array unit.

[0038] Figure 4 is a circuit diagram showing an example of a circuit configuration of a pixel.

[0039] Figure 5 is a block diagram showing a first configuration example of the address event detection unit.

[0040] Figure 6 is a block diagram illustrating a first configuration example of an address event detection unit of a light-shielded pixel.

[0041] Figure 7 is a circuit diagram showing an example of the configuration of a current-voltage conversion unit.

[0042] Figure 8 is a circuit diagram showing an example of the configuration of a subtractor and a quantizer in an address event detection unit.

[0043] Figure 9 is a block diagram showing a second configuration example of the address event detection unit.

[0044] Figure 10 : is a block diagram showing a second configuration example of the address event detection unit of the light-shielded pixel.

[0045] Figure 11is a block diagram illustrating a configuration example of an imaging device according to a second configuration example of the present disclosure.

[0046] Figure 12 is an exploded perspective view schematically showing a laminated chip structure of an imaging device.

[0047] Figure 13 is a block diagram illustrating an example of the configuration of a column processing unit of an imaging device according to a first configuration example.

[0048] Figure 14 is a diagram showing the relationship between noise events and temperature.

[0049] Figure 15 is a flowchart illustrating a processing example of the threshold control unit.

[0050] Figure 16 is a flowchart illustrating a processing example of a threshold control unit in parallel processing.

[0051] Figure 17 It shows Figure 15 and Figure 16 Flowchart of a processing example of step S1000 in .

[0052] Figure 18 It is schematically shown in Figure 17 Schematic diagram of a processing example shown in .

[0053] Figure 19 It shows Figure 17 ] is a diagram of an example of executing a process of three cycles as shown in FIG.

[0054] Figure 20 is a flowchart illustrating an example of a process of independently adjusting a first threshold value and a second threshold value.

[0055] Figure 21 is a diagram schematically illustrating a processing example of independently adjusting the first threshold value and the second threshold value.

[0056] Figure 22 is a flowchart illustrating a processing example of independently adjusting a first threshold value and a second threshold value in parallel processing.

[0057] Figure 23 is a flowchart illustrating an example in which the threshold value is changed in stages.

[0058] Figure 24 It is schematically shown in Figure 23 Schematic diagram of the processing example shown in .

[0059] Figure 25 is a flowchart illustrating an example in which the first threshold value is changed in stages.

[0060] Figure 26 is a diagram schematically showing an example in which the first threshold value is changed in stages.

[0061] Figure 27 is a flowchart illustrating an example of changing the second threshold in stages.

[0062] Figure 28 is a diagram schematically showing an example in which the second threshold value is changed in stages.

[0063] Figure 29 is a flowchart showing an example in which a dead zone is provided.

[0064] Figure 30 : is a diagram schematically showing an example in which the second threshold value provided with a dead zone is changed in stages.

[0065] Figure 31 is a flowchart illustrating an example in which the amount of change in the threshold value changes according to temperature changes.

[0066] Figure 32 is a flowchart schematically illustrating an example in which the amount of change in the threshold value changes according to temperature change.

[0067] Figure 33 is a block diagram showing the configuration of an imaging apparatus according to a modification example of the first embodiment.

[0068] Figure 34 1 is a block diagram showing a schematic configuration example of a vehicle control system as an example of a mobile body control system to which the technology according to the present disclosure can be applied.

[0069] Figure 35 2 is a diagram showing an example of installation positions of the imaging unit and the vehicle exterior information detection unit. DETAILED DESCRIPTION

[0070] Hereinafter, embodiments of an imaging device and an imaging method will be described with reference to the accompanying drawings. Although the following will primarily describe the main components of the imaging device, the imaging device may have components and functions not shown or described. The following description does not exclude components and functions not shown or described.

[0071] (First embodiment)

[0072] Figure 1 is a block diagram illustrating an example of a system configuration of an imaging system to which the technology according to the present disclosure is applied.

[0073] like Figure 1As shown, an imaging system 10 to which the technology according to the present disclosure is applied includes an imaging lens 11, a recording unit 12, a control unit 13, and an imaging device 20. The imaging system 10 is an example of an electronic device of the present disclosure, and examples of electronic devices include a camera system mounted on an industrial robot, a vehicle-mounted camera system, and the like.

[0074] In the imaging system 10 having the above configuration, the imaging lens 11 captures incident light from a subject and forms an image on the imaging surface of the imaging device 20. The imaging device 20 photoelectrically converts the incident light captured by the imaging lens 11 in units of pixels to obtain imaging data. As the imaging device 20, the imaging device of the present disclosure described later is used.

[0075] The imaging device 20 performs predetermined signal processing, such as image recognition processing, on the captured image data and outputs data representing the processing results and a detection signal of an address event (hereinafter sometimes referred to as a "detection signal") to the recording unit 12. A method for generating the detection signal of the address event will be described later. The recording unit 12 stores the data supplied from the imaging device 20 via the signal line 14. The control unit 13 includes, for example, a microcomputer and controls the imaging operation of the imaging device 20.

[0076] [Imaging Apparatus (Discrimination Method) According to First Configuration Example]

[0077] Figure 2 is a block diagram showing a configuration example of an imaging device according to a first configuration example, which serves as the imaging device 20 in the imaging system 10 to which the technology according to the present disclosure is applied.

[0078] like Figure 2 As shown, an imaging device 20 as a first configuration example of an imaging device according to the present disclosure is an asynchronous imaging device called DVS, and includes a pixel array unit 21, a drive unit 22, a discrimination unit (discriminator unit) 23, a column processing unit 24, a signal processing unit 25, and a setting unit 26. In addition, the setting unit 26 has a counting unit 26a and a threshold control unit 26b.

[0079] In the imaging device 20 having the above configuration, a plurality of pixels 30 are two-dimensionally arranged in a matrix (array) in the pixel array unit 21. A vertical signal line VSL to be described later is wired for each pixel column relative to this matrix-like pixel arrangement.

[0080] Each of the plurality of pixels 30 generates an analog signal corresponding to a voltage of the photocurrent as a pixel signal. Furthermore, each of the plurality of pixels 30 detects the presence or absence of an address event based on whether the amount of change in the photocurrent exceeds a predetermined threshold. When an address event occurs, the pixel 30 outputs a request to the determination unit 23.

[0081] In addition, the pixel array unit 21 is provided with a light-shielding area 21 a that blocks light. On the other hand, in addition to the light-shielding area 21 a , a light-receiving area 21 b is provided in the pixel array unit 21 .

[0082] The light-shielding area 21a is set, for example, to correspond to a plurality of pixels 30 in rows at the upper end. Alternatively, the light-shielding area 21a can be set corresponding to a plurality of pixels 30 in columns. As described above, the light-shielding area 21a is set in at least one of the upper end, the lower end, the left end, or the right end of the pixel array unit 21 to cover the plurality of pixels 30. Except that light is not incident, the configuration of the pixels 30 in the light-shielding area 21a is the same as the configuration of the pixels 30 outside the light-shielding area 21a. Therefore, the plurality of pixels 30 in the light-shielding area 21a detects the presence or absence of an address event based on whether the amount of change in current caused by, for example, dark-time noise exceeds a predetermined threshold. In addition, in the present embodiment, the pixels 30 in the light-shielding area 21a can be referred to as light-shielding pixels 30.

[0083] The temperature sensor 21 c acquires temperatures corresponding to the plurality of pixels 30 in the pixel array unit 21 , and supplies a signal including temperature information to the threshold control unit 26 b of the setting unit 26 .

[0084] The driving unit 22 drives each of the plurality of pixels 30 arranged in the light receiving area 21 b excluding the light shielding area 21 a to output a pixel signal generated in each pixel 30 to the column processing unit 24 .

[0085] The discrimination unit 23 discriminates a request from each of the plurality of pixels 30 and transmits a response based on the discrimination result to the pixel 30. The pixel 30 that has received the response from the discrimination unit 23 supplies a detection signal indicating the detection result (a detection signal of an address event) to the drive unit 22 and the signal processing unit 25. Reading the detection signal from the pixel 30 can be performed by reading a plurality of rows.

[0086] The column processing unit 24 includes, for example, an A / D converter and performs processing for converting analog pixel signals output from the pixels 30 of the row into digital signals for each pixel column of the pixel array unit 21. The column processing unit 24 then supplies the A / D-converted digital signals to the signal processing unit 25.

[0087] The signal processing unit 25 performs predetermined signal processing, such as correlated double sampling (CDS) processing or image recognition processing, on the digital signal supplied from the column processing unit 24. Then, the signal processing unit 25 supplies data indicating the processing result and the detection signal supplied from the discrimination unit 23 to the recording unit 12 via the signal line 14 (see Figure 1 ).

[0088] The setting unit 26 sets a threshold value for detecting the presence or absence of an address event. For example, the setting unit 26 sets the threshold value based on the noise level of the plurality of light-shielded pixels 30. The counting unit 26a counts the address events generated in the light-shielded pixels 30.

[0089] The threshold control unit 26b sets a threshold value for detecting the presence or absence of an address event in the address event detection unit 33, which will be described later, based on the number of address events counted by the counting unit 26a. The threshold control unit 26b includes, for example, a register and can set a threshold value for an address event and a threshold value for the number of address events that have occurred in the register. The details of the threshold control unit 26b will be described later.

[0090] [Configuration example of pixel array unit]

[0091] Figure 3 2 is a block diagram showing an example of the configuration of the pixel array unit 21 .

[0092] In the pixel array unit 21 in which a plurality of pixels 30 are two-dimensionally arranged in a matrix, each of the plurality of pixels 30 includes a light receiving unit 31 , a pixel signal generating unit 32 , and an address event detecting unit 33 .

[0093] In the pixel 30 having the above configuration, the light receiving unit 31 in the light receiving area 21b photoelectrically converts the incident light to generate a photocurrent. Then, under the control of the driving unit 22, the light receiving unit 31 supplies the photocurrent generated by the photoelectric conversion to the pixel signal generating unit 32 or the address event detecting unit 33 (see Figure 2 ).

[0094] On the other hand, the light receiving unit 31 of the light shielded pixel outputs an output current corresponding to a noise component such as a dark current. Then, the light receiving unit 31 of the light shielded pixel supplies the current corresponding to the noise component to the address event detection unit 33 (see Figure 2 ).

[0095] The pixel signal generating unit 32 generates a signal of a voltage corresponding to the photocurrent supplied from the light receiving unit 31 in the light receiving area 21b as a pixel signal SIG, and supplies the generated pixel signal SIG to the column processing unit 24 via the vertical signal line VSL (see FIG. Figure 2 ).

[0096] The address event detection unit 33 detects the presence or absence of an address event based on whether the amount of change in the photocurrent from each of the light receiving cells 31 in the light receiving area 21b exceeds a predetermined threshold. Address events (hereinafter sometimes referred to as "events") include, for example, a turn-on event indicating that the amount of change in the photocurrent exceeds an upper first threshold and a turn-off event indicating that the amount of change is lower than a lower second threshold. In addition, the detection signal of the address event includes, for example, one bit indicating the detection result of the turn-on event and one bit indicating the detection result of the turn-off event. Note that the address event detection unit 33 can be configured to detect only the turn-on event.

[0097] On the other hand, the address event detection unit 33 of the light-shielded pixel detects the presence or absence of an address event based on whether the change in the current output from each of the light-receiving cells 31 of the light-shielded pixel exceeds a predetermined threshold. Address events include, for example, an on-event indicating that the change in the output current exceeds an upper first threshold and an off-event indicating that the change is below a lower second threshold. Furthermore, the detection signal for the address event includes, for example, one bit indicating the detection result of an on-event and one bit indicating the detection result of an off-event. Note that the address event detection unit 33 can be configured to detect only on-events or only off-events.

[0098] When an address event occurs, the address event detecting unit 33 in the light receiving area 21b supplies a request for requesting transmission of a detection signal of the address event to the discrimination unit 23 (see Figure 2 Then, upon receiving a request in response to the determination unit 23 , the address event detection unit 33 supplies a detection signal of the address event to the drive unit 22 and the signal processing unit 25 .

[0099] On the other hand, when an address event occurs, the address event detection unit 33 of the light-shielded pixel supplies a detection signal of the address event to the counting unit 26a of the setting unit 26. The counting unit 26a then counts the number of on-events and off-events and supplies the counted number to the threshold control unit 26b. With this configuration, the address event detection unit 33 of the light-shielded pixel can supply a detection signal including event information to the setting unit 26 via a different path from the address event detection unit 33 in the light-receiving area 21b.

[0100] [Circuit Configuration Example of Pixel]

[0101] Figure 4 is a circuit diagram showing an example of a circuit configuration of the pixel 30. As described above, each of the plurality of pixels 30 includes the light receiving unit 31, the pixel signal generating unit 32, and the address event detecting unit 33.

[0102] In the pixel 30 having the above configuration, the light receiving unit 31 includes a light receiving element (photoelectric conversion element) 311, a transfer transistor 312, and an overcurrent gate (OFG) transistor 313. For example, N-type metal oxide semiconductor (MOS) transistors are used as the transfer transistor 312 and the OFG transistor 313. The transfer transistor 312 and the OFG transistor 313 are connected in series.

[0103] The light receiving element 311 in the light receiving region 21 b is connected between the common connection node N1 of the transfer transistor 312 and the OFG transistor 313 and the ground, and photoelectrically converts incident light to generate charges of a charge amount corresponding to the light amount of the incident light.

[0104] On the other hand, the light-shielded pixel's light-receiving element 311 is connected between the common connection node N1 of the transfer transistor 312 and the OFG transistor 313 and the ground, and generates a charge corresponding to the charge amount of the noise component. The transfer transistor 312 of the light-shielded pixel's light-receiving element 311 is always in the off state, and the supply of charge to the pixel signal generating unit 32 is stopped. Therefore, the light-shielded pixel 30 does not need to include the transfer transistor 312 and the pixel signal generating unit 32.

[0105] Transmission signal TRG from Figure 2 The driving unit 22 shown in FIG supplies a signal to the gate electrode of the transfer transistor 312 in the light receiving area 21 b. In response to the transfer signal TRG, the transfer transistor 312 supplies the charge photoelectrically converted by the light receiving element 311 to the pixel signal generating unit 32.

[0106] The control signal OFG is supplied from the driving unit 22 to the gate electrode of the OFG transistor 313. In response to the control signal OFG, the OFG transistor 313 supplies the electric signal generated by the light receiving element 311 to the address event detection unit 33. The electric signal supplied to the address event detection unit 33 is a photocurrent containing charge.

[0107] The pixel signal generating unit 32 includes a reset transistor 321, an amplifying transistor 322, a selecting transistor 323, and a floating diffusion layer 324. For example, N-type MOS transistors are used as the reset transistor 321, the amplifying transistor 322, and the selecting transistor 323.

[0108] The charge photoelectrically converted by the light receiving element 311 is supplied from the light receiving unit 31 to the pixel signal generating unit 32 by the transfer transistor 312. The charge supplied from the light receiving unit 31 is accumulated in the floating diffusion layer 324. The floating diffusion layer 324 generates a voltage signal having a voltage value corresponding to the amount of accumulated charge. That is, the floating diffusion layer 324 converts the charge into a voltage.

[0109] The reset transistor 321 is connected to the power supply voltage V DD The power supply line and the floating diffusion layer 324 are connected. A reset signal RST is supplied from the driving unit 22 to the gate electrode of the reset transistor 321. The reset transistor 321 initializes (resets) the charge amount of the floating diffusion layer 324 in response to the reset signal RST.

[0110] The amplifier transistor 322 and the selection transistor 323 are connected to the power supply voltage V DD The amplifying transistor 322 amplifies the voltage signal subjected to charge-voltage conversion by the floating diffusion layer 324 .

[0111] The selection signal SEL is supplied from the drive unit 22 to the gate electrode of the selection transistor 323. In response to the selection signal SEL, the selection transistor 323 outputs the voltage signal amplified by the amplifier transistor 322 to the column processing unit 24 via the vertical signal line VSL (see Figure 2 ) as the pixel signal SIG.

[0112] In the imaging device 20 including the pixel array unit 21 in which the pixels 30 having the above-described configuration are two-dimensionally arranged, when Figure 1 When the control unit 13 shown in FIG gives an instruction to start detecting an address event of the light receiving area 21 b , the driving unit 22 supplies a control signal OFG to the OFG transistor 313 of the light receiving unit 31 , driving the OFG transistor 313 to supply a photocurrent to the address event detecting unit 33 .

[0113] On the other hand, when Figure 1 When the control unit 13 shown in FIG gives an instruction to start detecting the address event of the shielded pixel, the driving unit 22 supplies the control signal OFG to the OFG transistor 313 of the light receiving unit 31 of the shielded pixel, driving the OFG transistor 313 to supply the output current to the address event detection unit 33.

[0114] Then, when an address event is detected in the pixel 30 in the light receiving area 21 b, the drive unit 22 turns off the OFG transistor 313 of the pixel 30 and stops the supply of photocurrent to the address event detection unit 33. Next, the drive unit 22 drives the transfer transistor 312 by supplying the transfer signal TRG to the transfer transistor 312, and transfers the charge photoelectrically converted by the light receiving element 311 to the floating diffusion layer 324.

[0115] In this manner, the imaging device 20, which includes the pixel array unit 21 in which the pixels 30 having the above-described configuration are two-dimensionally arranged, outputs pixel signals from only the pixels 30 in the light-receiving region 21 b where an address event has been detected to the column processing unit 24. On the other hand, the light-shielded pixels 30 are used only to detect address events. Therefore, compared to the case where pixel signals for all pixels are output, regardless of the presence or absence of an address event, the power consumption of the imaging device 20 and the amount of image processing can be reduced.

[0116] Note that the configuration of the pixel 30 illustrated here is an example and is not limited to this configuration example. For example, the pixel configuration does not necessarily include the pixel signal generating unit 32. In the case of such a pixel configuration, in the light receiving unit 31, it is only necessary to omit the OFG transistor 313 and make the transfer transistor 312 have the function of the OFG transistor 313.

[0117] [First Configuration Example of Address Event Detection Unit]

[0118] Figure 5 is a block diagram showing a first configuration example of the address event detection unit 33 in the light receiving area 21 b. Figure 6 1 is a block diagram showing a first configuration example of the address event detection unit 33 for light-shielded pixels. Figure 5 and Figure 6 As shown, the address event detection unit 33 according to the present configuration example includes a current-voltage conversion unit 331 , a buffer 332 , a subtractor 333 , a quantizer 334 , and a transmission unit 335 .

[0119] The current-voltage conversion unit 331 converts the photocurrent from the light receiving unit 31 of the pixel 30 into a logarithmic voltage signal. The current-voltage conversion unit 331 supplies the converted voltage signal to the buffer 332. The buffer 332 buffers the voltage signal supplied from the current-voltage conversion unit 331 and supplies the voltage signal to the subtractor 333.

[0120] The subtractor 333 is supplied with a row drive signal from the driver unit 22. The subtractor 333 reduces the level of the voltage signal supplied from the buffer 332 according to the row drive signal. The subtractor 333 then supplies the voltage signal after the level reduction to the quantizer 334. The quantizer 334 quantizes the voltage signal supplied from the subtractor 333 into a digital signal and outputs the digital signal as a detection signal of the address event to the transmission unit 335.

[0121] like Figure 5As shown, the transmission unit 335 of the light receiving area 21b transmits the detection signal of the address event supplied from the quantizer 334 to the determination unit 23 and the like. When an address event is detected, the transmission unit 335 supplies a request for requesting the transmission of the detection signal of the address event to the determination unit 23. Then, when a response to the request is received from the determination unit 23, the transmission unit 335 supplies the detection signal of the address event to the drive unit 22 and the signal processing unit 25.

[0122] On the other hand, Figure 6 As shown, the light-shielded pixel transmission unit 335 transmits the detection signal of the address event supplied from the quantizer 334 to the setting unit 26 .

[0123] Next, a configuration example of the current-voltage conversion unit 331 , the subtractor 333 , and the quantizer 334 in the address event detection unit 33 will be described.

[0124] (Configuration Example of Current-Voltage Conversion Unit)

[0125] Figure 7 3 is a circuit diagram showing an example of the configuration of the current-voltage conversion unit 331 in the address event detection unit 33. Figure 7 As shown, the current-voltage conversion unit 331 according to this embodiment has a circuit configuration including an N-type transistor 3311, a P-type transistor 3312, and an N-type transistor 3313. As these transistors 3311 to 3313, for example, MOS transistors are used.

[0126] N-type transistor 3311 is connected to the power supply voltage V DD The P-type transistor 3312 and the N-type transistor 3313 are connected in series between the power supply voltage V DD Then, the common connection node N2 of the P-type transistor 3312 and the N-type transistor 3313 is connected to Figure 5 and 6 , which is a gate electrode of the N-type transistor 3311 and an input terminal of the buffer 332.

[0127] Predetermined bias voltage V bias is applied to the gate electrode of the P-type transistor 3312. Therefore, the P-type transistor 3312 supplies a constant current to the N-type transistor 3313. Photocurrent is input from the light receiving unit 31 to the gate electrode of the N-type transistor 3313 through the signal input line 3314.

[0128] The drain electrodes of the N-type transistor 3311 and the N-type transistor 3313 are connected to the power supply side, and such a circuit is called a source follower. The photocurrent from the light receiving unit 31 is converted into a logarithmic voltage signal by two source followers connected in a ring shape.

[0129] (Configuration example of subtractor and quantizer)

[0130] Figure 8 3 is a circuit diagram showing an example of the configuration of the subtractor 333 and the quantizer 334 in the address event detection unit 33 .

[0131] The subtractor 333 according to the present example includes a capacitance element 3331 , an inverter circuit 3332 , a capacitance element 3333 , and a switch element 3334 .

[0132] One end of the capacitor element 3331 is connected to Figure 5 and Figure 6 The output terminal of the buffer 332 is shown, and the other end thereof is connected to the input terminal of the inverter circuit 3332. The capacitor element 3333 is connected in parallel with the inverter circuit 3332. The switch element 3334 is connected between the two ends of the capacitor element 3333. The row drive signal is supplied from the driver unit 22 to the switch element 3334 as an open / close control signal. The switch element 3334 opens and closes the path connecting the two ends of the capacitor element 3333 according to the row drive signal. The inverter circuit 3332 inverts the polarity of the voltage signal input via the capacitor element 3331.

[0133] In the subtractor 333 having the above configuration, when the switch element 3334 is turned on (off), the voltage signal V init The terminal of the capacitor element 3331 on one side of the buffer 332 is input, and the terminal on the opposite side becomes the virtual ground terminal. For convenience, the potential of the virtual ground terminal is set to zero. At this time, when the capacitance value of the capacitor element 3331 is C1, the charge Q accumulated in the capacitor element 3331 is init It is expressed by the following formula (1). On the other hand, since both ends of the capacitor 3333 are short-circuited, the accumulated charge is zero.

[0134] Q init =C1×V init ...(1)

[0135] Next, consider that the switching element 3334 is turned off (opened) and the voltage of the terminal of the capacitor element 3331 on the buffer 332 side changes to V after In the case of after It is expressed by the following formula (2).

[0136] Qafter =C1×V after ...(2)

[0137] On the other hand, when the capacitance value of the capacitor element 3333 is C2 and the output voltage is V out When , the charge Q2 accumulated in the capacitor element 3333 is expressed by the following formula (3).

[0138] Q2=-C2×V out ...(3)

[0139] At this time, since the total amount of charge of capacitor 3331 and capacitor 3333 does not change, the following formula (4) holds.

[0140] Q init =Q after +Q2...(4)

[0141] When Formula (1) to Formula (3) are substituted into Expression (4) and deformed, the following Formula (5) is obtained.

[0142] V out =-(C1 / C2)×(V after -V init )...(5)

[0143] Formula (5) represents the subtraction operation of the voltage signal, and the gain of the subtraction result is C1 / C2. Because it is generally desired to maximize the gain, it is preferably designed to be large and C2 to be small. On the other hand, when C2 is too small, kTC noise increases and the noise characteristics may deteriorate. Therefore, the capacity reduction of C2 is limited to a range that can tolerate noise. In addition, since an address event detection unit 33 including a subtractor 333 is installed for each pixel 30, the capacitor element 3331 and the capacitor element 3333 have area restrictions. Taking this into account, the capacitance values C1 and C2 of the capacitor elements 3331 and 3333 are determined.

[0144] exist Figure 8 , the quantizer 334 includes a comparator 3341. The comparator 3341 takes the output signal of the inverter circuit 3332, that is, the voltage signal of the subtractor 430, as a non-inverting (+) input and converts a predetermined threshold voltage V th Then, the comparator 3341 compares the voltage signal from the subtractor 430 with the predetermined threshold voltage V th A comparison is performed, and a signal indicating the comparison result is output to the transmission unit 335 as an address event detection signal.

[0145] [Second Configuration Example of Address Event Detection Unit]

[0146] Figure 9 is a block diagram showing a second configuration example of the address event detection unit 33 in the light receiving area 21 b. Figure 10 1 is a block diagram showing a second configuration example of the address event detection unit 33 for light-shielded pixels. Figure 9 and Figure 10 As shown, the address event detection unit 33 according to the present configuration example includes a storage unit 336 and a control unit 337 in addition to a current-voltage conversion unit 331 , a buffer 332 , a subtractor 333 , a quantizer 334 , and a transmission unit 335 .

[0147] The storage unit 336 is provided between the quantizer 334 and the transmission unit 335, and accumulates the output of the quantizer 334 (i.e., the comparison result of the comparator 3341) based on the sample signal supplied from the control unit 337. The storage unit 336 may be a sampling circuit such as a switch, plastic, or capacitor, or may be a digital memory circuit such as a latch or a flip-flop.

[0148] The control unit 337 sets the predetermined threshold voltage V th Supplied to the inverting (-) input terminal of the comparator 3341. The threshold voltage V supplied from the control unit 337 to the comparator 3341 th Different voltage values can be provided in a time-division manner. For example, in a manner in which one comparator 3341 can detect multiple types of address events, the control unit 337 provides a threshold voltage V corresponding to a turn-on event indicating that the amount of change in the photocurrent exceeds the upper threshold at different timings. th1 and the threshold voltage V corresponding to the disconnection event indicating that the change amount is lower than the lower threshold th2 .

[0149] For example, the storage unit 336 may be at a threshold voltage V corresponding to a disconnection event. th2 In the period of time supplied from the control unit 337 to the inverting (-) input terminal of the comparator 3341, the threshold voltage V corresponding to the turn-on event is used. th1 To accumulate the comparison results of the comparator 3341. Note that the storage unit 336 may be inside the pixel 30 or may be outside the pixel 30. In addition, the storage unit 336 is not an essential component of the address event detection unit 33. That is, the storage unit 336 may be omitted.

[0150] [Image Forming Apparatus According to Second Configuration Example (Scanning Type)]

[0151] The imaging device 20 according to the first configuration example described above is an asynchronous imaging device that reads events using an asynchronous reading method. However, the event reading method is not limited to the asynchronous reading method and may be a synchronous reading method. The imaging device to which the synchronous reading method is applied is a scanning type imaging device similar to a general imaging device that performs imaging at a predetermined frame rate.

[0152] Figure 11 : is a block diagram showing an example of the configuration of an imaging apparatus according to a second configuration example (ie, a scanning-type imaging apparatus serving as the imaging apparatus 20 in the imaging system 10 to which the technology according to the present invention is applied).

[0153] like Figure 11 As shown, the imaging device 20 according to the second configuration example of the imaging device as the present disclosure includes a pixel array unit 21 , a driving unit 22 , a signal processing unit 25 , a setting unit 26 , a reading area selection unit 27 , and a signal generating unit 28 .

[0154] The pixel array unit 21 includes a plurality of pixels 30. The plurality of pixels 30 outputs output signals in response to the selection signal of the read region selection unit 27. For example, Figure 6 As shown in FIG, each of the plurality of pixels 30 may have a quantizer in the pixel. The plurality of pixels 30 output an output signal corresponding to the amount of change in light intensity. The plurality of pixels 30 may be arranged two-dimensionally as shown in FIG. Figure 11 The matrix shown.

[0155] In addition, similar to Figure 2 , the pixel array unit 21 is provided with a light-shielding area 21a that blocks light and a light-receiving area 21b other than the light-shielding area 21a. The light-shielding area 21a corresponds to at least one of an arrangement of a plurality of pixels 30 in rows or an arrangement of a plurality of pixels 30 in columns. In addition, the light-shielding area 21a is provided in at least one of the upper end, the lower end, the left end or the right end of the pixel array unit 21. As described above, the light-shielding area 21a is provided to cover the plurality of pixels 30 at the end of the pixel array unit 21. Except that light is not incident, the configuration of the pixels 30 in the light-shielding area 21a is the same as the configuration of the pixels 30 other than the light-receiving area 21b. Therefore, the plurality of pixels 30 in the light-shielding area 21a detect the presence or absence of an address event based on whether the amount of change in current caused by noise exceeds a predetermined threshold.

[0156] The driving unit 22 drives each of the plurality of pixels 30 to output a pixel signal generated in each pixel 30 to the signal processing unit 25. It should be noted that the driving unit 22 and the signal processing unit 25 are circuit units for acquiring grayscale information. Therefore, when only event information is acquired, the driving unit 22 and the signal processing unit 25 may not be provided.

[0157] The read region selection unit 27 selects some of the plurality of pixels 30 included in the pixel array unit 21. For example, the read region selection unit 27 selects any one or more rows from the rows included in the structure of the two-dimensional matrix corresponding to the pixel array unit 21. The read region selection unit 27 sequentially selects one or more rows according to a preset time period. In addition, the read region selection unit 27 may determine the selected region in response to a request from each pixel 30 of the pixel array unit 21.

[0158] The signal generating unit 28 generates an event signal corresponding to a valid pixel in which an event has been detected among the selected pixels based on the output signal of the pixel selected by the reading area selecting unit 27. An event is an event in which light intensity changes. A valid pixel is a pixel in which the amount of change in light intensity corresponding to the output signal exceeds or falls below a preset first threshold value or a second threshold value. For example, the signal generating unit 28 detects a first detection signal when the absolute value of the amount of change in the direction in which the signal level of the electric signal increases exceeds a first threshold value, and detects a second detection signal when the absolute value of the amount of change in the direction in which the signal level of the electric signal decreases exceeds a second threshold value. Then, the pixel in which the first detection signal or the second detection signal is detected is set as a valid pixel, and an event signal corresponding to the valid pixel is generated.

[0159] The signal generating unit 28 may include, for example, a column selection circuit that determines signals entering the signal generating unit 28. Furthermore, the signal generating unit 28 may be configured to output not only information of valid pixels where an event has been detected but also information of invalid pixels where no event has been detected.

[0160] The address information and time stamp information (e.g., (X, Y, Ton) and (X, Y, Toff)) of the valid pixel where the event has been detected are output from the signal generation unit 28 via the output line 15. However, the data output from the signal generation unit 28 may be not only address information and time stamp information but also information in a frame format (e.g., (0, 0, 1, 0, ...)). Ton represents the time when the on event is detected, and Toff represents the time when the off event is detected.

[0161] The counting unit 26a of the setting unit 26 counts the number of on-events and off-events within a predetermined time period using address information and time stamp information (e.g., (X, Y, Ton) and (X, Y, Toff)) of valid pixels and supplies the counted number to the threshold control unit 26b. The counting unit 26a may count one of the on-event and the off-event and supply the counted number to the threshold control unit 26b.

[0162] [Configuration example of chip structure]

[0163] As a chip (semiconductor integrated circuit) structure of the imaging device 20 according to the first configuration example or the second configuration example described above, for example, a laminated chip structure may be employed. Figure 12 is an exploded perspective view schematically showing a laminated chip structure of the imaging device 20 .

[0164] like Figure 12 As shown, the laminated chip structure, that is, the laminated structure has a structure in which at least two chips of the light receiving chip 201 as the first chip and the detection chip 202 as the second chip are laminated. Figure 4 In the circuit configuration of the pixel 30 shown, each light receiving element 311 is arranged on the light receiving chip 201, and all elements other than the light receiving element 311, elements of other circuit portions of the pixel 30, and the like are arranged on the detection chip 202. The light receiving chip 201 and the detection chip 202 are electrically connected via a connection portion such as a via (VIA), Cu-Cu bonding, or a bump.

[0165] It should be noted that here, a configuration example has been illustrated in which the light receiving element 311 is arranged on the light receiving chip 201 and elements other than the light receiving element 311, elements of other circuit parts of the pixel 30, etc. are arranged on the detection chip 202, but the present technology is not limited to this configuration example.

[0166] For example, in Figure 4 In the circuit configuration of the pixel 30 shown, each element of the light receiving unit 31 can be arranged on the light receiving chip 201, and elements other than the light receiving unit 31, elements of other circuit portions of the pixel 30, and the like can be arranged on the detection chip 202. In addition, each element of the light receiving unit 31, as well as the reset transistor 321 and floating diffusion layer 324 of the pixel signal generating unit 32, can be arranged on the light receiving chip 201, and other elements can be arranged on the detection chip 202. In addition, part of the elements constituting the address event detecting unit 33 can be arranged on the light receiving chip 201 together with each element of the light receiving unit 31 and the like.

[0167] [Configuration example pf column processing unit]

[0168] Figure 13 1 is a block diagram showing a configuration example of the column processing unit 24 of the imaging device 20 according to the first configuration example. Figure 13 As shown, the column processing unit 24 according to the present example includes a plurality of analog-to-digital converters (ADCs) 241 arranged for each pixel column of the pixel array unit 21 .

[0169] It should be noted that, here, an example of a configuration in which the analog-to-digital converters 241 are arranged in a one-to-one correspondence with the pixel columns of the pixel array unit 21 has been described, but the present technology is not limited to this configuration example. For example, the analog-to-digital converters 241 may be arranged in units of a plurality of pixel columns, and the analog-to-digital converters 241 may be used in a time-division manner between the plurality of pixel columns.

[0170] The analog-to-digital converter 241 converts the analog pixel signal SIG supplied via the vertical signal line VSL into a digital signal having a larger bit depth than the detection signal of the above-mentioned address event. For example, when the detection signal of the address event is 2 bits, the pixel signal is converted into a digital signal of 3 bits or more (16 bits, etc.). The analog-to-digital converter 241 supplies the digital signal generated by the analog-to-digital conversion to the signal processing unit 25.

[0171] [Noise Event]

[0172] Incidentally, the imaging device 20 is an imaging device including, for each pixel 30, a detection unit (ie, address event detection unit 33) that detects, in real time for each pixel address, that the light amount of the pixel exceeds a predetermined threshold as an address event.

[0173] In this imaging device, when a specific event (i.e., a real event) initially occurs in a scene, data that caused the real event to occur is acquired. However, even in a scene where no real event occurs, data may be wastefully acquired due to noise events (false events) such as sensor noise. Therefore, not only is a noise signal read, but the throughput of the signal output is also reduced. The imaging device 20 of the present disclosure described below performs sensitivity adjustment for real events and false events based on the number of events in the shielded pixels.

[0174] Figure 14 This is a diagram showing the relationship between noise events and temperature. The horizontal axis represents time, and the vertical axis represents the number of events and temperature. Line L12A shows the temporal change in temperature, and line L12B shows the temporal change in the number of dark-time noise events (which are dark-time noise events (when shielded from light)). Even in dark-time, pixel 30 outputs a dark-time current by performing photoelectric conversion. This is counted as a dark-time event.

[0175] like Figure 14 As shown, the time-series variation in the number of dark events correlates with, for example, the time-series variation in temperature. For example, reducing the sensitivity to noise events reduces noise events, but also reduces the detection sensitivity for real events. On the other hand, increasing the detection sensitivity for real events also increases the sensitivity for noise events.

[0176] To this end, the threshold control unit 26b adjusts the sensitivity to a true event and the sensitivity to a false event caused by dark-time noise by performing sensitivity adjustment according to the state of the imaging device 20. This will be described in more detail below.

[0177] Figure 15 2 is a flowchart illustrating an example of processing by the threshold control unit 26b of the setting unit 26. Here, an example of detecting an event by the pixels 30 in the light receiving area 21b after sensitivity adjustment using the light shielding pixels 30 in the light shielding area 21a will be described. As described above, the method of reading the detection signal of the event differs between the imaging device according to the asynchronous first configuration example and the imaging device according to the second configuration example (scanning type), but the method of setting the threshold of the imaging device according to the asynchronous first configuration example and the method of setting the threshold of the imaging device according to the second configuration example (scanning type) can perform equivalent processing.

[0178] like Figure 15 As shown, first, the threshold control unit 26b sets a threshold according to the noise level of the light-shielded pixels 30 in the light-shielded area 21a (step S1000).

[0179] Next, when the variation of the electric signal generated in the pixel 30 in the light receiving area 21b exceeds the threshold value set in the threshold control unit 26b, the address event detection unit 33 in the light receiving area 21b detects a detection signal (step S1100).

[0180] Next, the threshold control unit 26b determines whether the entire process is to be completed (step S1200). If not completed (No in step S1200), the process from step S102 is repeated. On the other hand, if the process is to be completed (Yes in step S1200), the entire process is completed.

[0181] As described above, since the threshold control unit 26 b sets the threshold according to the noise level of the light-shielded pixels 30 in the light-shielded area 21 a , sensitivity adjustment for real events and dark-time noise events can be performed.

[0182] Figure 16 : is a flowchart showing an example of processing of the threshold control unit 26b in parallel processing. Figure 16 As shown, setting of a threshold value corresponding to the noise level of the light-shielded pixel 30 (step S1000 ) and detection processing of the address event detection unit 33 in the light-receiving region 21 b (step S1100 ) are performed in parallel.

[0183] As described above, by performing the processing in parallel, sensitivity adjustment for real events and dark-time noise events can be performed without stopping event detection by the address event detection unit 33 in the light-receiving area 21 b .

[0184] Figure 17 It shows Figure 15 and Figure 16 Flowchart of a processing example of step S1000 in .

[0185] Figure 18 It is schematically shown in Figure 17 . The horizontal axis represents time and the vertical axis represents the threshold value. Figure 18 , schematically illustrates the amount of change in the first threshold value for a turn-on event and the second threshold value for a turn-off event relative to a reference level (AZ level). Here, the absolute values of the amount of change in the first threshold value for a turn-on event and the second threshold value for a turn-off event relative to the reference level (AZ level) are considered to be the same. For example, if the first threshold value rises by Vh1 volts from the initial setting value, the second threshold value falls by Vh1 volts from the initial setting value.

[0186] like Figure 17 As shown in FIG, the threshold control unit 26b sets the first threshold voltage V th1 and the second threshold voltage V th2 The light shielding pixel 30 and the comparator 3341 of the light receiving region 21b (see Figure 8 and Figure 10 ) inverting (-) input terminal (step S100). The comparator 3341 outputs the value corresponding to the value indicating that the change in current exceeds the first threshold voltage V to the counting unit 26a of the setting unit 26. th1 The event detection signal corresponding to the upper limit of the on event and the event detection signal indicating that the change in current exceeds the second threshold voltage V th2 The counting unit 26a counts the number of events that occur within a predetermined time period and supplies the counted number of events to the threshold control unit 26b (step S102).

[0187] Please note that Figure 16 In the case of parallel processing shown, Figure 17 The predetermined period in step S102 shown does not have to be the entire imaging period, and a predetermined period for counting the number of events of light-shielded pixels and a period not for counting the number of events of light-shielded pixels may be alternately set.

[0188] In the case of the imaging device (scanning type) according to the second configuration example, the read area selection unit 27 selects some of the plurality of pixels 30 included in the light shielding area 21a. For example, the read area selection unit 27 selects any one row or a plurality of rows among the rows in the structure corresponding to the two-dimensional matrix included in the light shielding area 21a. The read area selection unit 27 (see Figure 11) sequentially selects one or more rows according to a preset time period, sequentially reads address information and time stamp information (e.g., (X, Y, Ton) and (X, Y, Toff)) of the effective pixels of the light-shielded pixels 30, and supplies the read information to the counting unit 26a via the signal generating unit 28. The counting unit 26a of the setting unit 26 counts the number of on-events and off-events within the predetermined time period using the address information and time stamp information (e.g., (X, Y, Ton) and (X, Y, Toff)) of the effective pixels, and supplies the counted number to the threshold control unit 26b.

[0189] Next, the threshold control unit 26b determines whether the number of events (number of on events + number of off events) within the predetermined time period is within a predetermined range (step S104). If the number of events is outside the predetermined range ("yes" in step S104), the threshold control unit 26b further determines whether the event count is equal to or greater than a predetermined value (step S106). If the number of events is greater than the predetermined value ("yes" in step S106), the threshold control unit 26b reduces the sensitivity of the threshold (step S108). That is, the threshold control unit 26b reduces the sensitivity of the first threshold voltage V with respect to the pixels 30 of the entire pixel array unit 21. th1 Raise the predetermined value Th1a volts and set the second threshold voltage V th2 Lower the predetermined value Th1a volts.

[0190] On the other hand, when the number of events is less than the predetermined value ("No" in step S106), the sensitivity of the threshold is increased. The threshold control unit 26b sets the first threshold voltage V th1 Reduce the predetermined value Th2a volts and set the second threshold voltage V th2 The predetermined value Th2a volts is increased (step S110).

[0191] On the other hand, if the number of events is less than the predetermined value (No in step S104), the sensitivity of the threshold is maintained. That is, the threshold control unit 26b does not change the first threshold voltage V of the pixels 30 of the entire pixel array unit 21. th1 and the second threshold voltage V th2 (Step S112).

[0192] Figure 19 It shows Figure 17 The processing shown in FIG is performed for three time periods. The horizontal axis represents time and the vertical axis represents the threshold value. Figure 19 As shown, the first threshold value for the switch-on event and the second threshold value for the switch-off event may vary over time.

[0193] As described above, based on the absolute value of the change amount from the reference level of the electric signal generated by the photoelectric conversion element 311 in the light shielding area 21a within a predetermined period of time exceeding the threshold value (the first threshold voltage V th1 Or the second threshold voltage V th2 ) number, set the threshold (first threshold voltage V th1 Or the second threshold voltage V th2 ). Therefore, threshold control can be performed according to the number of occurrences of dark-time noise events in the light-shielded pixels 30 in the light-shielded area 21a, and sensitivity adjustment for real events and dark-time noise events can be performed.

[0194] Figure 20 FIG. 1 is a flowchart showing an example of a process of independently adjusting a first threshold value for a switch-on event and a second threshold value for a switch-off event. Figure 20 As shown, first, the threshold control unit 26b sets a first threshold according to the on event of the shading pixel 30 in the shading area 21a (step S1000a), and in parallel, sets a second threshold according to the level of the off event of the shading pixel 30 in the shading area 21a (step S1000b).

[0195] Next, when the change in the electrical signal generated in the pixel 30 in the light receiving area 21b exceeds the first threshold and the second threshold set independently in the threshold control unit 26b, the address event detection unit 33 in the light receiving area 21b detects the detection signal of the on event and the detection signal of the off event (step S1100).

[0196] Figure 21 1 is a diagram schematically showing an example of a process of independently adjusting a first threshold value for a power-on event and a second threshold value for a power-off event.

[0197] As described above, the threshold control unit 26b independently sets the first threshold and the second threshold based on the noise level of the light-shielded pixels 30 in the light-shielded area 21a. Therefore, sensitivity adjustment can be performed for a real event (where the absolute value of the amount of change in the direction of increase in the signal level of the electrical signal exceeds the first threshold) and a dark-time noise event corresponding to an on-event. In addition, at the same time, sensitivity adjustment can be performed for a real event where the absolute value of the amount of change in the direction of increase in the signal level of the electrical signal exceeds the second threshold and a dark-time noise event corresponding to an off-event. As described above, because the first threshold and the second threshold can be adjusted independently, sensitivity adjustment can be performed more appropriately even when there is a statistical deviation in the occurrence distribution of dark-time noise events corresponding to on-events and dark-time noise events corresponding to off-events.

[0198] Figure 22FIG. 1 is a flowchart showing an example of a process of independently adjusting a first threshold value and a second threshold value in parallel processing. Figure 21 As shown, the threshold control unit 26b performs in parallel the processing of setting the first threshold according to the level of the on-event of the shading pixel 30 in the shading area 21a (step S1000a), and performs in parallel the processing of setting the second threshold according to the level of the off-event of the shading pixel 30 in the shading area 21a (step S1000b), and detects in parallel the processing of the address event detection unit 33 in the light receiving area 21b (step S1100).

[0199] As described above, by performing the processes in parallel, the first threshold value and the second threshold value can be adjusted independently without stopping the event detection by the address event detection unit 33 in the light receiving area 21 b .

[0200] Figure 23 1000 is a flowchart showing an example of changing the threshold value in stages in the procedure of the threshold value setting process (step S1000 ).

[0201] Figure 24 It is schematically shown in Figure 23 The horizontal axis represents time and the vertical axis represents the threshold value. Figure 20 As shown, the amount of change in the first threshold value for a connection event and the second threshold value for a disconnection event relative to the reference level (AZ level) changes in a time-series, stage by stage. For example, the threshold value changes in several stages until it reaches a presumed convergence value. Here, assuming the convergence value is the AZ level + the predetermined value Vhf, the predetermined value Th1 is changed to the AZ level + Vhf × 0.8, the predetermined value Th2 is changed to the AZ level + Thf × 0.96, and so on, in such a way that the values gradually approach the presumed convergence value of the threshold value over time.

[0202] like Figure 23 As shown, the threshold control unit 26b determines whether the number of events (number of on events + number of off events) within a predetermined time period relative to the initially set threshold Th0 is greater than a predetermined range (step S104). If the number of events is greater than the predetermined value ("Yes" in step S104), the threshold control unit 26b gradually reduces the sensitivity of the threshold (step S108a). That is, within the first period, the threshold control unit 26b reduces the first threshold voltage V th1 The reference level is increased by a predetermined value Th1 volt, and the second threshold voltage V th2 The threshold voltage V is reduced by a predetermined value Th1 volt from the reference level. Next, in the second time period, the threshold control unit 26b reduces the first threshold voltage V to 1 with respect to the pixels 30 of the entire pixel array unit 21. th1The reference level is increased by a predetermined value Th2 volts, and the second threshold voltage V th2 The reference level is reduced by a predetermined value Th2 V. In this case, the absolute value of (Th2-Th1) is smaller than the absolute value of (Th1-Th0).

[0203] On the other hand, when the number of events is less than the predetermined value (No in step S104 ), the threshold control unit 26 b does not change the current threshold and ends the entire process.

[0204] As described above, since the threshold value is changed in stages, the sensitivity can be set close to the target number of dark noise events. In addition, since the threshold value is changed in stages, the threshold value can be changed while suppressing changes in the number of events occurring per unit time of the pixels 30 in the light receiving area 21b.

[0205] Figure 25 1000a) is a flowchart showing an example of a stage-by-stage change of the first threshold value in the process of setting the threshold value (step S1000a). Figure 25 As shown, the number of on-times of the light-shielded pixels is counted (step S102a), and if the counted number does not fall within a predetermined range, the first threshold is changed in stages (step S110a). As described above, even when the first threshold and the second threshold are adjusted independently, the process of changing the first threshold in stages can be performed.

[0206] Figure 26 Schematically shows an example of a first threshold value being changed in stages. The horizontal axis represents time, and the vertical axis represents the first threshold value. Figure 26 As shown in FIG. 1 , the first threshold value Th0a initially set is changed to the first threshold value Th1a in the first stage, and is changed to the first threshold value Th2a in the second stage. As described above, by making a rough adjustment and then gradually converging the first threshold value, it is possible to promote convergence while suppressing divergence. Note that, in Figure 26 The processing shown in is also applied to the processing of setting the threshold value (step S1000).

[0207] Figure 27 1000b is a flowchart showing an example of changing the second threshold value in stages during the threshold value setting process (step S1000b). Figure 27 As shown, the disconnection events of the light-shielded pixels are counted (step S102b), and when the counted number does not fall within the predetermined range, the second threshold is changed in stages (step S110b). As described above, even when the first threshold and the second threshold are adjusted independently, the process of changing the second threshold can be performed in stages.

[0208] Figure 28Schematically shows an example of changing the second threshold value in stages. The horizontal axis represents time, and the vertical axis represents the first threshold value. Figure 28 As shown in FIG. 1 , the first threshold value Th0b initially set is changed to the first threshold value Th1b in the first stage, and is changed to the first threshold value Th2b in the second stage. As described above, by making a rough adjustment and then gradually converging the second threshold value, it is possible to promote convergence while suppressing divergence. It should be noted that in Figure 28 The processing shown in is also applied to the processing of setting the threshold value (step S1000).

[0209] Figure 29 1000 is a flowchart showing an example in which a dead zone is set in the procedure of the threshold setting process (step S1000 b ). Figure 30 2 is a diagram schematically showing an example in which the second threshold value provided with a dead zone is changed in stages. The horizontal axis represents time, and the vertical axis represents the second threshold value.

[0210] Here, the dead zone refers to the thresholds set above and below the second threshold Th0b, Th1b, Th2b, etc. For example, 10% before and after the absolute value of the difference between the AZ level and the second threshold Th0b is set as the dead zone. More specifically, when the AZ level is, for example, 100 and the second threshold Th0b is 50, the absolute value of the difference between the AZ level and the second threshold Th0b is (100-50), and 10% before and after the absolute value of the difference is 50×0.1=±5. That is, the dead zone is 45 to 55. For example, in the case of changing the threshold of the second threshold Th0b, when the second threshold Th1b is within the dead zone, the threshold is not changed, and when the second threshold Th1b is outside the dead zone, the threshold is changed. In Figure 30 In the example, since the second threshold value Th1b exceeds the dead band range of the second threshold value Th0b, the value of the second threshold value Th1b is changed from the second threshold value Th0b. On the other hand, since the second threshold value Th2b is outside the dead band range of the second threshold value Th1b, the value of the second threshold value Th2b is the same as the second threshold value Th1b and is not changed.

[0211] like Figure 29 As shown, if the number of disconnection events is within the predetermined range ("No" in step S104b), threshold control unit 26b determines whether the value of the second threshold Th1b to be changed is within the dead band of the current second threshold Th0b (step S112b). If the value of the second threshold Th1b is outside the dead band, threshold control unit 26b changes the value of the second threshold Th1b from the second threshold Th0b ("No" in step S112b). On the other hand, if the value of the second threshold Th1b is within the dead band ("Yes" in step S112b), the value of the second threshold Th1b is maintained at the value of the second threshold Th0b, and the process ends.

[0212] When the amount by which the threshold should change is within the deadband, the threshold does not change. Figure 30 The processing shown in is also applied to the processing of setting the threshold value (step S1000 and step S1000a).

[0213] For example, when the threshold is changed, the viewer can perceive a change in the image. Therefore, increasing the dead zone does not change the threshold, preventing the viewer from perceiving a deviation in the image. On the other hand, increasing the dead zone suppresses sensitivity adjustment. Therefore, by adjusting the dead zone, a balance can be achieved between sensitivity adjustment and image change.

[0214] Figure 31 1000 is a flowchart showing an example in which the amount of change in the threshold value changes according to the temperature change in the procedure of the threshold value setting process (step S1000 b ). Figure 32 FIG. 1 is a flow chart schematically showing an example of how the amount of change in the threshold value changes according to temperature change. The horizontal axis represents time, and the vertical axis represents the second threshold value. Figure 32 As shown, the amount of change in the second threshold value Th1b increases in a region where the temporal change in the time-series value Tma of the temperature is large.

[0215] like Figure 31 As shown, if the number of disconnection events is within a predetermined range ("No" in step S104b), the threshold control unit 26b calculates the time change of the temperature using the temperature information corresponding to the pixels 30 in the light receiving area 21b recorded in time series in the recording unit 12. For example, the absolute value of the time difference of the temperature is calculated (step S116b). The temperature acquired from the temperature sensor 21c is recorded in time series in the recording unit 12.

[0216] Then, the threshold control unit 26b calculates the variation of the second threshold based on the variation of the temperature and changes the second threshold (step S118b). As described above, the variation of the second threshold is adjusted according to the variation of the temperature over time. Figure 14 As shown, because temperature variation is correlated with noise variation, the amount of variation of the second threshold value can be increased in a manner that can more effectively adjust the sensitivity when the temperature variation is large.

[0217] As described above, according to this embodiment, the threshold control unit 26b sets the threshold according to the noise level of the light-shielded pixels 30 in the light-shielded area 21a. Therefore, since the occurrence amount of dark noise events can be adjusted, sensitivity adjustment for real events and dark noise events can be performed.

[0218] (Modification of the First Embodiment)

[0219] The imaging system 10 according to the modification of the first embodiment is different from the imaging system 10 according to the first embodiment in that the arithmetic processing unit 15 for setting the threshold value is provided outside the imaging device 20. Differences from the imaging system 10 according to the first embodiment will be described below.

[0220] Figure 33 is a block diagram showing the configuration of an imaging device 20 according to a modification of the first embodiment. Figure 33 In FIG, the driving unit 22, the determining unit 23 and the column processing unit 24 are not shown. Figure 33 As shown, the operation processing unit 15 is provided outside the imaging device 20. The operation processing unit 15 is a device that allows the user to change the operation program and processing parameters. Therefore, the user can set the threshold value by any algorithm.

[0221] <Application Examples of Technology According to the Present Disclosure>

[0222] The technology disclosed herein can be applied to a variety of products. More specific application examples will be described below. For example, the technology disclosed herein can be implemented as a distance measuring device mounted on any type of mobile object, such as an automobile, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility device, airplane, drone, ship, robot, construction machinery, and agricultural machinery (tractor).

[0223] [Mobile Object]

[0224] Figure 34 7000 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. Figure 31 In the illustrated example, a vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, a vehicle exterior information detection unit 7400, a vehicle interior information detection unit 7500, and an integrated control unit 7600. A communication network 7010 connecting the plurality of control units may be, for example, an in-vehicle communication network conforming to any standard, such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), or FlexRay (registered trademark).

[0225] Each control unit includes a microcomputer that performs calculations according to various programs, a storage unit that stores the programs executed by the microcomputer, parameters for various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via the communication network 7010, and a communication I / F for communicating with devices, sensors, etc. inside and outside the vehicle through wired or wireless communication. Figure 34 76, as a functional configuration of integrated control unit 7600, a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, a vehicle interior device I / F 7660, an audio / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690 are shown. Other control units similarly include a microcomputer, a communication I / F, a storage unit, and the like.

[0226] The drive system control unit 7100 controls the operation of devices related to the vehicle's drive system according to various programs. For example, the drive system control unit 7100 functions as a control device for a drive force generating device such as an internal combustion engine or a drive motor for generating the vehicle's drive force, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the vehicle's steering angle, a braking device for generating the vehicle's braking force, and the like. The drive system control unit 7100 may also function as a control device for systems such as an anti-lock braking system (ABS) or an electronic stability control (ESC).

[0227] Drive system control unit 7100 is connected to vehicle state detection unit 7110. Vehicle state detection unit 7110 includes, for example, at least one of a gyroscopic sensor for detecting the angular velocity of the vehicle body's axial rotational motion, an acceleration sensor for detecting vehicle acceleration, or sensors for detecting the amount of accelerator pedal operation, amount of brake pedal operation, steering wheel angle, engine speed, wheel speed, and the like. Drive system control unit 7100 performs computational processing using signals input from vehicle state detection unit 7110 to control the internal combustion engine, drive motor, electric power steering, brake system, and the like.

[0228] The body system control unit 7200 controls the operation of various devices installed on the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, power windows, or various lights such as headlights, taillights, brake lights, blinkers, or fog lights. In this case, radio waves transmitted from a portable device that replaces a key or signals from various switches can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door locks, power windows, lights, and other functions.

[0229] The battery control unit 7300 controls the secondary battery 7310, which serves as a power supply source for driving the electric motor, according to various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from the battery device including the secondary battery 7310. The battery control unit 7300 uses these signals to perform arithmetic processing and control the temperature of the secondary battery 7310 or control a cooling device included in the battery device.

[0230] The vehicle exterior information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the imaging unit 7410 and the vehicle exterior information detection unit 7420 is connected to the vehicle exterior information detection unit 7400. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, or other cameras. The vehicle exterior information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or the weather, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0231] The environmental sensor may be, for example, at least one of a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunlight sensor for detecting sunlight levels, or a snow sensor for detecting snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, or a light detection and ranging, laser imaging detection and ranging (LIDAR) device. The imaging unit 7410 and the vehicle exterior information detection unit 7420 may be configured as independent sensors or devices, or may be configured as a device integrating multiple sensors or devices.

[0232] Here, Figure 35Examples of the installation locations of imaging unit 7410 and vehicle exterior information detection unit 7420 are shown. Imaging units 7910, 7912, 7914, 7916, and 7918 are installed in at least one of the front nose, side mirrors, rear bumper, rear door, or upper portion of the windshield of vehicle 7900. Imaging unit 7910, located in the front nose, and imaging unit 7918, located in the upper portion of the windshield, primarily capture images of the front of vehicle 7900. Imaging units 7912 and 7914, located in the side mirrors, primarily capture images of the sides of vehicle 7900. Imaging unit 7916, located in the rear bumper or rear door, primarily captures images of the rear of vehicle 7900. Imaging unit 7918, located in the upper portion of the windshield, primarily detects vehicles ahead, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

[0233] Please note that Figure 35 The figure shows examples of the imaging ranges of the respective imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of imaging unit 7910, which is located at the front nose; imaging ranges b and c indicate the imaging ranges of imaging units 7912 and 7914, which are located at the side mirrors, respectively; and imaging range d indicates the imaging range of imaging unit 7916, which is located at the rear bumper or rear door. For example, by overlapping the image data captured by imaging units 7910, 7912, 7914, and 7916, it is possible to obtain a bird's-eye view image of vehicle 7900 viewed from above.

[0234] Furthermore, the vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930 located above the front windshield, rear windshield, side windshields, and corner windshields of vehicle 7900 may also be, for example, ultrasonic sensors or radar devices. Furthermore, the vehicle exterior information detection units 7920, 7926, and 7930 located within the front nose, rear bumper, rear door, and upper portion of the windshield of vehicle 7900 may also be, for example, LIDAR devices. These vehicle exterior information detection units 7920 and 7930 are primarily used to detect preceding vehicles, pedestrians, obstacles, and the like.

[0235] return Figure 34, will continue to describe. The vehicle external information detection unit 7400 enables the imaging unit 7410 to capture images of the outside of the vehicle and receive captured image data. In addition, the vehicle external information detection unit 7400 receives detection information from the connected vehicle external information detection unit 7420. In the case where the vehicle external information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, ultrasonic waves, electromagnetic waves, etc. are transmitted, and information of the received reflected waves is received. The vehicle external information detection unit 7400 can perform object detection processing and distance detection processing of people, vehicles, obstacles, signs, text on the road surface, etc. based on the received information. The vehicle external information detection unit 7400 can perform environmental recognition processing to identify rainfall, fog, road conditions, etc. based on the received information. The vehicle external information detection unit 7400 can calculate the distance to objects outside the vehicle based on the received information.

[0236] In addition, the vehicle exterior information detection unit 7400 can also perform image recognition processing or distance detection processing for people, vehicles, obstacles, signs, text on the road, etc. based on the received image data. The vehicle exterior information detection unit 7400 can also perform distortion correction, alignment, and other processing on the received image data, and synthesize the image data captured by the different imaging units 7410 to generate bird's-eye views, panoramic images, etc. The vehicle exterior information detection unit 7400 can use the image data captured by the different imaging units 7410 to perform viewpoint transformation processing.

[0237] The vehicle interior information detection unit 7500 detects information about the interior of the vehicle. For example, a driver state detection unit 7510 that detects the driver's state is connected to the vehicle interior information detection unit 7500. The driver state detection unit 7510 may include a camera that images the driver, a biosensor that detects the driver's biometric information, a microphone that collects sounds inside the vehicle, and the like. For example, the biosensor is provided on a seat surface, a steering wheel, or the like, and detects biometric information of a passenger sitting on the seat or the driver holding the steering wheel. The vehicle interior information detection unit 7500 can calculate the driver's fatigue or concentration based on the detection information input from the driver state detection unit 7510, or can determine whether the driver is dozing off. The vehicle interior information detection unit 7500 can perform processing on the collected sound signals, such as noise cancellation processing.

[0238] Integrated control unit 7600 controls the overall operation of vehicle control system 7000 according to various programs. Input unit 7800 is connected to integrated control unit 7600. For example, input unit 7800 can be implemented using a device such as a touch panel, buttons, microphone, switch, or joystick that can be operated by a passenger to input information. Data obtained by performing voice recognition on voice input via the microphone can be input to integrated control unit 7600. Input unit 7800 can be, for example, a remote control device using infrared or other radio waves, or an externally connected device that corresponds to the operation of vehicle control system 7000, such as a mobile phone or personal digital assistant (PDA). Input unit 7800 can be, for example, a camera, in which case the passenger can input information through gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger can be input. Furthermore, input unit 7800 can include, for example, an input control circuit that generates an input signal based on information input by the passenger or the like using the input unit 7800 and outputs the input signal to integrated control unit 7600. By operating the input unit 7800 , a passenger or the like inputs various data to the vehicle control system 7000 or gives instructions regarding processing operations.

[0239] The storage unit 7690 may include a read-only memory (ROM) that stores various programs to be executed by the microcomputer and a random access memory (RAM) that stores various parameters, calculation results, sensor values, etc. In addition, the storage unit 7690 may be implemented by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0240] The general communication I / F 7620 is a general communication I / F that mediates communications with various devices present in the external environment 7750. The general communication I / F 7620 can implement a cellular communication protocol, such as Global System for Mobile Communications (GSM) (registered trademark), WiMAX, Long Term Evolution (LTE) or LTE-Advanced (LTE-A) or other wireless communication protocols, such as wireless LAN (also known as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general communication I / F 7620 can be connected to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or a company-specific network) via, for example, a base station or an access point. In addition, the general communication I / F 7620 can be connected to a terminal present near the vehicle (e.g., a terminal of a driver, a pedestrian, or a store, or a machine type communication (MTC) terminal) using, for example, peer-to-peer (P2P) technology.

[0241] The dedicated communication I / F 7630 is a communication I / F that supports communication protocols developed for use in vehicles. For example, the dedicated communication I / F 7630 can implement standard protocols such as Wireless Access in Vehicular Environments (WAVE) (which is a combination of the lower layer IEEE 802.11p and the upper layer IEEE 1609), Dedicated Short Range Communication (DSRC), or cellular communication protocols. The dedicated communication I / F 7630 typically implements V2X communication, a concept that includes one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0242] Positioning unit 7640 receives a global navigation satellite system (GNSS) signal (e.g., a global positioning system (GPS) signal from a GPS satellite) from a GNSS satellite, performs positioning, and generates position information including the latitude, longitude, and altitude of the vehicle. Note that positioning unit 7640 can specify the current position by exchanging signals with a wireless access point, or can obtain position information from a terminal such as a mobile phone, PHS, or a smartphone with a positioning function.

[0243] The beacon receiving unit 7650 receives radio waves or electromagnetic waves transmitted from, for example, a wireless station installed on a road, and obtains information such as the current position, traffic congestion, closed roads, required time, etc. Note that the function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

[0244] The onboard device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and the various onboard devices 7760 present in the vehicle. The onboard device I / F 7660 can establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless USB (WUSB). In addition, the onboard device I / F 7660 can establish a wired connection such as a universal serial bus (USB), a high-definition multimedia interface (HDMI) (registered trademark), or a mobile high-definition link (MHL) via a connection terminal (and, if necessary, a cable) not shown. The onboard device 7760 may include, for example, a mobile device or wearable device owned by a passenger, or at least one of an information device carried in or attached to the vehicle. In addition, the onboard device 7760 may include a navigation device that searches for a route to an arbitrary destination. The onboard device I / F 7660 exchanges control signals or data signals with these onboard devices 7760.

[0245] The in-vehicle network I / F 7680 is an interface that coordinates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like according to a predetermined protocol supported by the communication network 7010.

[0246] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the vehicle-mounted device I / F 7660, or the vehicle-mounted network I / F 7680. For example, the microcomputer 7610 can calculate control target values for the driving force generation device, the steering mechanism, or the braking device based on the acquired information about the interior and exterior of the vehicle, and output control commands to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control to implement functions of an advanced driver assistance system (ADAS), including collision avoidance or collision mitigation of the vehicle, follow-up driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 can perform cooperative control for autonomous driving, etc. by controlling the driving force generating device, steering mechanism, braking device, etc. based on the information obtained around the vehicle, in which the vehicle travels autonomously without relying on the driver's operation.

[0247] Microcomputer 7610 can generate three-dimensional distance information between the vehicle and objects such as surrounding structures or people based on information acquired via at least one of general communication I / F 7620, dedicated communication I / F 7630, positioning unit 7640, beacon receiving unit 7650, in-vehicle device I / F 7660, or in-vehicle network I / F 7680, and create local map information including surrounding information of the vehicle's current location. Furthermore, microcomputer 7610 can predict dangers (such as a vehicle collision, the approach of a pedestrian, or entering a closed road) based on the acquired information and generate an alarm signal. The alarm signal can be, for example, a signal for generating an alarm sound or turning on an alarm light.

[0248] The audio image output unit 7670 transmits an output signal of at least one of sound or image to an output device capable of visually or auditorily notifying a passenger of the vehicle or the outside of the vehicle of information. Figure 34In the example of , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are shown as output devices. The display unit 7720 may include, for example, at least one of an onboard display or a head-up display. The display unit 7720 may have an augmented reality (AR) display function. The output device may be another device different from these devices, such as a wearable device, such as a headset or glasses-type display worn by a passenger, a projector, or a lamp. In the case where the output device is a display device, the display device visually displays the results obtained by the various processes performed by the microcomputer 7610 or the information received from another control unit in various formats (such as text, images, tables, and graphics). In addition, in the case where the output device is a sound output device, the sound output device converts an audio signal including reproduced sound data, sound data, etc. into an analog signal and audibly outputs the analog signal.

[0249] Note that in Figure 34 In the example shown, at least two control units connected via the communication network 7010 can be integrated into one control unit. Optionally, each control unit may include multiple control units. In addition, the vehicle control system 7000 may include another control unit (not shown). In addition, in the above description, some or all of the functions performed by any control unit can be supplied to another control unit. In other words, as long as information is sent and received via the communication network 7010, predetermined operation processing can be performed by any control unit. Similarly, a sensor or device connected to any control unit can be connected to another control unit, and multiple control units can send and receive detection information to each other via the communication network 7010.

[0250] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the imaging units 7910, 7912, 7914, 7916, and 7918, the vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930, the driver state detection unit 7510, and the like in the above configuration. Specifically, the imaging device including the present disclosure Figure 1 The imaging system 10 in FIG. 1 can be applied to these imaging units and detection units. Then, by applying the technology according to the present disclosure, the effects of noise events such as sensor noise can be mitigated in a manner that enables safe vehicle driving, and the occurrence of real events can be reliably and quickly sensed.

[0251] It should be noted that the present technology can have the following configurations.

[0252] (1) An imaging device comprising:

[0253] a photoelectric conversion unit including a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal;

[0254] a setting unit that sets a threshold value according to a noise level of a predetermined area of the plurality of photoelectric conversion elements; and

[0255] The first detection unit detects a detection signal when the amount of change in the electrical signals generated by the plurality of photoelectric conversion elements exceeds a threshold value.

[0256] (2) The imaging device according to (1), wherein

[0257] The photoelectric conversion element in a predetermined area is shielded from light, and

[0258] The photoelectric conversion elements other than the predetermined area each photoelectrically convert incident light to generate an electric signal.

[0259] (3) The imaging device according to (2), wherein the plurality of photoelectric conversion elements are arranged in a two-dimensional matrix, and the light shielding area corresponds to at least one of the arrangement of the photoelectric conversion elements in row units or the arrangement of the photoelectric conversion elements in column units.

[0260] (4) The imaging device according to (3), wherein the arrangement of the photoelectric conversion elements in row units and the arrangement of the photoelectric conversion elements in column units include end portions of the plurality of photoelectric conversion elements arranged in a two-dimensional matrix.

[0261] (5) The imaging device according to any one of (1) to (4), wherein the setting unit sets the threshold value based on the number of times that the absolute value of the amount of change in the electric signal generated by the photoelectric conversion element in the predetermined area exceeds the threshold value in the predetermined time period.

[0262] (6) The imaging device according to any one of (1) to (5), wherein

[0263] a first detection unit that detects a first detection signal when the absolute value of the amount of change in the direction in which the signal level of the electric signal increases exceeds a first threshold value, and detects a second detection signal when the absolute value of the amount of change in the direction in which the signal level of the electric signal decreases exceeds a second threshold value, and

[0264] Setting up the unit

[0265] setting a first threshold value based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the electric conversion element in the predetermined area increases exceeds the first threshold value in a predetermined period of time, and

[0266] The second threshold is set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the electric conversion element in the predetermined area decreases exceeds the second threshold in the predetermined period.

[0267] (7) The imaging device according to any one of (1) to (5), wherein the setting unit sets the threshold value in stages based on the number of times the threshold value is exceeded in a predetermined period of time.

[0268] (8) The imaging device according to (7), wherein the setting unit reduces a rate of change of the threshold value as time passes.

[0269] (9) The imaging device according to (8), wherein the setting unit reduces the rate of change of the threshold value in such a manner that the value asymptotically approaches the initial setting value.

[0270] (10) The imaging device according to (8), wherein the setting unit reduces the change rate in a manner gradually approaching a predetermined setting value after setting the threshold value to the threshold value of the first stage.

[0271] (11) The imaging device according to any one of (1) to (10), wherein the setting unit does not change the threshold value if the number of times the threshold value is exceeded in the predetermined period of time is less than a predetermined value.

[0272] (12) The imaging device according to any one of (1) to (10), wherein the setting unit sets the threshold value according to temperatures corresponding to the plurality of photoelectric conversion elements.

[0273] (13) The imaging device according to (12), wherein the setting unit increases the rate of change of the threshold value as the change in temperature increases.

[0274] (14) The imaging device according to any one of (1) to (13), wherein

[0275] The first detection unit sequentially reads the electrical signals of the photoelectric conversion elements in a predetermined area, and

[0276] The setting unit counts the number of detection signals exceeding a predetermined period of time.

[0277] (15) An imaging method comprising:

[0278] Setting a threshold value according to a noise level of the photoelectric conversion element that is shielded from light; and

[0279] A detection signal is detected when an absolute value of a change amount of an electric signal generated by a plurality of photoelectric conversion elements, each of which performs photoelectric conversion on incident light to generate an electric signal, exceeds a threshold value.

[0280] The various aspects of the present disclosure are not limited to the individual embodiments described above, and include various modifications that can be thought of by those skilled in the art, and the effects of the present disclosure are not limited to the above. That is, various additions, modifications, and partial deletions may be made without departing from the conceptual concept and scope of the present disclosure as defined in the claims and their equivalents.

[0281] Reference Signs List

[0282] 10 Imaging system 20 Imaging device 21 Pixel array unit 21a Light shielding area 21b Light receiving area 22 Driving unit 23 Determination unit 24 Column processing unit 25 Signal processing unit 26 Setting unit 26a Counting unit 26b Threshold value control unit 30 Pixel 31 Light receiving unit 33 Address event detection unit

Claims

1. An imaging device comprising: a photoelectric conversion unit including a plurality of photoelectric conversion elements that perform photoelectric conversion to generate an electrical signal; a setting unit that sets a threshold value according to a noise level of a predetermined area among the plurality of photoelectric conversion elements; as well as a first detection unit configured to detect a detection signal when the amount of change in the electrical signal generated by the plurality of photoelectric conversion elements exceeds the threshold value; The setting unit sets the threshold value in stages based on the number of times that the absolute value of the amount of change of the electric signal generated by the photoelectric conversion element in the predetermined area exceeds the threshold value in a predetermined time period.

2. The imaging device according to claim 1, wherein The photoelectric conversion element in the predetermined area is shielded from light, and The photoelectric conversion elements other than the predetermined area each photoelectrically convert incident light to generate the electric signal.

3. The imaging device according to claim 2, wherein The plurality of photoelectric conversion elements are arranged in a two-dimensional matrix, and the light shielding region corresponds to at least one of an arrangement of the photoelectric conversion elements in a row unit and an arrangement of the photoelectric conversion elements in a column unit.

4. The imaging device according to claim 3, wherein The arrangement of the photoelectric conversion elements in rows and the arrangement of the photoelectric conversion elements in columns include end portions of the plurality of photoelectric conversion elements arranged in a two-dimensional matrix.

5. The imaging device according to claim 1, wherein The first detection unit detects a first detection signal when the absolute value of the amount of change in the direction in which the signal level of the electrical signal increases exceeds a first threshold, and detects a second detection signal when the absolute value of the amount of change in the direction in which the signal level of the electrical signal decreases exceeds a second threshold, and Setting unit, setting the first threshold value based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the photoelectric conversion element in the predetermined area increases exceeds the first threshold value in a predetermined period of time, and The second threshold is set based on the number of times that the absolute value of the amount of change in the direction in which the signal level of the electric signal generated by the photoelectric conversion element in the predetermined area decreases exceeds the second threshold in a predetermined period of time.

6. The imaging device according to claim 1, wherein The setting unit reduces the rate of change of the threshold value as time passes.

7. The imaging device according to claim 6, wherein The setting unit reduces the change rate of the threshold value to asymptotically approach an initial setting value.

8. The imaging device according to claim 6, wherein The setting unit reduces the change rate to gradually approach a predetermined setting value after setting the threshold value to the threshold value of the first stage.

9. The imaging device according to claim 1, wherein The setting unit does not change the threshold value when the number of times the threshold value is exceeded within a predetermined period of time is less than a predetermined value.

10. The imaging device according to claim 1, wherein The setting unit sets the threshold value according to temperatures corresponding to the plurality of photoelectric conversion elements.

11. The imaging device according to claim 10, wherein The setting unit increases the rate of change of the threshold value as the change of the temperature increases.

12. The imaging device according to claim 1, wherein The first detection unit sequentially reads the electrical signals of the photoelectric conversion elements in the predetermined area, and The setting unit counts the number of the detection signals exceeding the threshold value within a predetermined period of time.

13. An imaging method comprising: Setting a threshold value according to the noise level of the photoelectric conversion element that is shielded from light; as well as detecting a detection signal when an absolute value of a change in an electric signal generated by a plurality of photoelectric conversion elements, each of which performs photoelectric conversion on incident light to generate the electric signal, exceeds the threshold value, The threshold value is set in stages based on the number of times that the absolute value of the amount of change of the electric signal generated by the photoelectric conversion element in a predetermined area exceeds the threshold value in a predetermined time period.

Images