Imaging system, imaging method, and computer program

Abstract

To provide an imaging system that can prevent reverse bias voltage of an avalanche photodiode to obtain a high-quality image.SOLUTION: In an imaging system, an imaging apparatus 300 has a control unit that controls a voltage parameter related to reverse bias voltage of an avalanche photodiode of a photoelectric conversion element 100, and controls an exposure parameter of the photoelectric conversion element to increase or decrease the sensitivity of the photoelectric conversion element according to the voltage parameter and increase or decrease the amount of exposure or the digital gain of the photoelectric conversion element according to the exposure parameter in conjunction with each other, and thereby controls the quality of an image obtained from the photoelectric conversion element.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an imaging system, an imaging method, a computer program, and the like. [Background technology]

[0002] 2. Description of the Related Art In recent years, imaging devices have been proposed that digitally count the number of photons arriving at a single-photon avalanche diode (SPAD) and output the counted value from a pixel as a photoelectrically converted digital signal.

[0003] When a photon is incident on a SPAD with a reverse bias voltage lower than the breakdown voltage applied, the photoelectrically converted electrons are amplified, causing avalanche amplification, in which a large current flows. The amount of light can be detected by counting the number of avalanche amplifications that occur.

[0004] It is known that the sensitivity of a SPAD changes depending on the reverse bias voltage. In a SPAD, the sensitivity increases as the bias applied in excess of the breakdown voltage increases. For example, Patent Document 1 proposes a method of photographing dark and bright areas by switching the reverse bias voltage for exposure. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6573186 Summary of the Invention [Problem to be solved by the invention]

[0006] In Prior Art Document 1, it is possible to obtain images with different brightness by exposing the image by switching the reverse bias voltage. However, depending on the scene to be photographed, it may be desirable to improve aspects other than brightness.

[0007] For example, in scenes involving specific controls such as camera (focus and pan-tilt) control and application (people counting, face recognition) control, it is desirable to obtain high-quality images (with less noise, less subject blur, less blur, etc.).

[0008] Therefore, an object of the present invention is to provide an imaging system capable of controlling the reverse bias voltage of an avalanche photodiode and obtaining a high-quality image. [Means for solving the problem]

[0009] According to one aspect of the present invention, there is provided an imaging system comprising: a voltage parameter control means for controlling a voltage parameter relating to a reverse bias voltage of an avalanche photodiode of the photoelectric conversion element; an exposure parameter control means for controlling an exposure parameter of the photoelectric conversion element; and a control means for controlling the image quality of the image obtained from the photoelectric conversion element by increasing or decreasing the sensitivity of the photoelectric conversion element using the voltage parameter and decreasing or increasing the exposure amount or digital gain of the photoelectric conversion element using the exposure parameter in conjunction with each other. Effect of the Invention

[0010] According to the present invention, it is possible to realize an imaging system capable of controlling the reverse bias voltage of an avalanche photodiode and obtaining high-quality images. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing a configuration example of a photoelectric conversion element according to a first embodiment. [Diagram 2] 3 is a diagram showing a configuration example of a sensor substrate according to the first embodiment. FIG. [Diagram 3] 1 is a diagram showing an example of the configuration of a circuit board according to a first embodiment. [Figure 4] 3 is a diagram showing a configuration example of an equivalent circuit of a signal processing circuit corresponding to a pixel of a photoelectric conversion element according to the first embodiment. FIG. [Diagram 5] 3 is a diagram illustrating a relationship between the operation of the APD 201 according to the first embodiment and an output signal. FIG. [Figure 6] 1 is a functional block diagram of an imaging device according to a first embodiment. [Figure 7] 5 is a flowchart showing an example of an imaging method performed by a control unit 303 according to the first embodiment. [Figure 8] 6(A) to 6(D) are diagrams showing examples of timing for changing voltage parameters and exposure parameters according to the first embodiment. [Figure 9] 13(A) to 13(C) are diagrams showing examples of timing for changing voltage parameters and exposure parameters according to the second embodiment. [Figure 10] 13 is a flowchart showing an example of an imaging method performed by a control unit 303 according to the third embodiment. [Figure 11] 13(A) to 13(D) are diagrams showing examples of timing for changing voltage parameters and exposure parameters according to the third embodiment. [Figure 12] FIG. 1 is a diagram showing an example of the configuration of an imaging system including an imaging device 300 according to first to third embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified.

[0013] <Embodiment 1> Fig. 1 is a diagram showing a configuration example of a photoelectric conversion element according to embodiment 1. As shown in Fig. 1, the photoelectric conversion element 100 of embodiment 1 is a photoelectric conversion element having a so-called stacked structure, in which two semiconductor substrates, a sensor substrate 11 and a circuit substrate 21, are stacked and electrically connected.

[0014] However, a so-called non-laminated structure in which the components included in the sensor substrate and the components included in the circuit substrate are disposed on a common semiconductor layer may be used. The sensor substrate 11 includes a pixel region 12. The circuit substrate 21 includes a circuit region 22 that processes signals detected in the pixel region 12.

[0015] 2 is a diagram showing a configuration example of a sensor substrate according to the first embodiment, in which a pixel region 12 of a sensor substrate 11 includes pixels 101 arranged two-dimensionally to form a plurality of rows and a plurality of columns. The pixels 101 include a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter, APD). Note that the number of rows and columns of the pixel array forming the pixel region 12 is not limited to the example shown in FIG. 2.

[0016] Fig. 3 is a diagram showing an example of the configuration of a circuit board according to embodiment 1. The circuit board 21 has a signal processing circuit 103 that processes the electric charges photoelectrically converted by the photoelectric conversion unit 102 in Fig. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110.

[0017] The vertical scanning circuit unit 110 receives a control pulse supplied from a control pulse generating unit 115, and supplies the control pulse to a plurality of pixels for each row. The vertical scanning circuit unit 110 uses logic circuits such as a shift register and an address decoder.

[0018] The signals output from the photoelectric conversion units 102 of the pixels are processed by the corresponding signal processing circuits 103. The signal processing circuits 103 are provided with a counter, a memory, etc., and digital values ​​are held in the memory.

[0019] The horizontal scanning circuit unit 111 supplies a control pulse for sequentially selecting each column to the signal processing circuit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held. The signal of the selected column is output from the signal processing circuit 103 of the pixel of the row selected by the vertical scanning circuit unit 110 to a signal line 113. The signal output to the signal line 113 is output to the outside of the photoelectric conversion element 100 via an output circuit 114.

[0020] 2 and 3, a plurality of signal processing circuits 103 are disposed below a plurality of pixel regions 12. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are disposed below a region between an end of the sensor substrate 11 and the pixel regions 12.

[0021] That is, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are arranged below the non-pixel region.

[0022] The arrangement of the signal lines 113, the readout circuits 112, and the output circuits 114 is not limited to that shown in Fig. 3. For example, the signal lines 113 may be arranged to extend in the row direction, and the readout circuits 112 may be arranged at the ends of the signal lines 113.

[0023] Furthermore, the function of the signal processing circuit does not necessarily need to be provided for each photoelectric conversion unit, but one signal processing circuit may be shared by multiple photoelectric conversion units and configured to perform signal processing sequentially.

[0024] FIG. 4 is a diagram showing an example of the configuration of an equivalent circuit of a signal processing circuit corresponding to a pixel of a photoelectric conversion element in embodiment 1, and shows an equivalent circuit of a pixel 101 and a signal processing circuit 103 corresponding to the pixel 101.

[0025] The photoelectric conversion element 100 has an avalanche photodiode (APD) 201, which generates a pair of charges according to incident light by photoelectric conversion. The anode of the APD 201 is connected to a power supply line through which a driving voltage VL is supplied.

[0026] The cathode of the APD 201 is connected to a power supply line to which a drive voltage VH higher than the drive voltage VL is supplied via a quench element 202. A reverse bias voltage VR that causes the APD 201 to perform an avalanche multiplication operation is supplied to the anode of the APD 201. Here, the reverse bias voltage VR is calculated by Equation 1. VR = VL - VH (Equation 1)

[0027] By applying such a reverse bias voltage VR, the charge generated by the incident light undergoes avalanche multiplication, generating an avalanche current.

[0028] When a reverse bias voltage VR is supplied, there are two modes: Geiger mode, in which the voltage difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the voltage difference between the anode and cathode is close to or less than the breakdown voltage. An APD that operates in Geiger mode is called a SPAD. In the case of a SPAD, the drive voltage VL is, for example, -30V, and the drive voltage VH is, for example, 1V.

[0029] The quench element 202 is connected to a power supply line to which a driving voltage VH is supplied and to the cathode of the APD 201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and performs a quenching operation to suppress avalanche multiplication by suppressing the voltage supplied to the APD 201.

[0030] Moreover, the quench element 202 performs a recharge operation to return the voltage supplied to the APD 201 to the drive voltage VH by passing a current corresponding to the voltage drop caused by the quench operation.

[0031] The signal processing circuit 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212. In Fig. 4, an example in which the signal processing circuit 103 has the waveform shaping section 210, the counter circuit 211, and the selection circuit 212 is shown, but it is sufficient for the signal processing circuit 103 to have at least one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212.

[0032] The waveform shaping unit 210 shapes the voltage change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 4, a circuit in which a plurality of inverters are connected in series may be used, or another circuit having a waveform shaping effect may be used.

[0033] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse RES is supplied via a drive line 213, the count value held in the counter circuit 211 is reset.

[0034] A control pulse SEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via a drive line 214 (not shown in Fig. 3) in Fig. 4. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal, and selectively outputs an output signal from the counter circuit 211 of the pixel to a signal line 113 in response to the control pulse SEL.

[0035] 4 shows an example in which the output signal is switched by the selection circuit 212, but electrical connections may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion unit 102 and the signal processing circuit 103. Similarly, the supply of the driving voltage VH or the driving voltage VL supplied to the photoelectric conversion unit 102 may be electrically switched using a switch such as a transistor.

[0036] 5 is a diagram illustrating a relationship between the operation of the APD 201 according to the first embodiment and an output signal, in which the input side of the waveform shaping unit 210 is node A and the output side is node B. Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201.

[0037] When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage at nodeA drops.

[0038] When the amount of voltage drop further increases and the potential difference applied to the APD 201 decreases, the avalanche multiplication of the APD 201 stops as at time t2, and the voltage level of nodeA does not drop beyond a certain value.

[0039] After that, between time t2 and time t3, a current flows through nodeA to compensate for the voltage drop from the drive voltage VL, and at time t3, nodeA is stabilized to its original potential level. At this time, the portion of the output waveform at nodeA that has become equal to or lower than a predetermined determination threshold is shaped by the waveform shaping section 210 and is output as a pulse signal at nodeB.

[0040] In addition, since the breakdown voltage of the APD 201 has temperature dependency, it is desirable to set the reverse bias voltage VR in conjunction with the temperature. Therefore, in this embodiment, the temperature of the photoelectric conversion element 100 is detected, and the reverse bias voltage VR is corrected to match the temperature dependency of the breakdown voltage.

[0041] Fig. 6 is a functional block diagram of the imaging device according to embodiment 1. Note that some of the functional blocks shown in Fig. 6 are realized by causing a CPU 311 serving as a computer included in a control unit 303 to execute a computer program stored in a memory (storage unit 304) serving as a storage medium.

[0042] However, some or all of these may be realized by hardware. For the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) may be used. Furthermore, each functional block shown in Fig. 6 does not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths.

[0043] The imaging device 300 has a photoelectric conversion element 100, an imaging optical system 301, an image processing unit 302, a control unit 303, a storage unit 304, an exposure parameter determination unit 305, and a voltage parameter determination unit 306. The imaging device 300 also has an imaging power supply unit 307, an aperture control unit 308, an aperture mechanism 309, a communication unit 310, etc. Some of the configuration and functions of the imaging device 300 may be provided on the side of an external device (client device) (not shown) instead of the imaging device 300.

[0044] The imaging optical system 301 is composed of multiple lenses and a barrel member that holds them, and makes an image incident on the photoelectric conversion element 100. In addition, zoom and focus can be controlled by a drive motor or the like. Furthermore, it may have a configuration in which an infrared cut filter, which is a filter that transmits or attenuates a specific wavelength, can be inserted or removed.

[0045] The aperture mechanism 309 is a mechanism for adjusting the amount of light incident on the photoelectric conversion element 100. The aperture amount is controlled by an aperture control unit 308. That is, the aperture mechanism 309 controls the aperture value (F-number) of the optical system that makes an image incident on the photoelectric conversion element. The photoelectric conversion element 100 has an on-chip color filter in, for example, a Bayer array of RGB.

[0046] An image formed by the imaging optical system 301 is captured by the photoelectric conversion element 100, and is AD converted and output as digital pixel data. In addition to image generation and correction processing, the pixel data is subjected to processing such as black level correction, gamma curve adjustment, noise reduction, white balance correction, color conversion, data compression, etc. in the image processing unit 302 to generate final image data.

[0047] In this embodiment, data input to the image processing unit 302 is called pixel data, and data output from the image processing unit 302 is called image data.

[0048] Also, an AE evaluation value for AE (Auto Exposure) is calculated from the image data generated in the image processing unit 302. The aperture, exposure time (image capture time) and digital gain are adjusted based on the AE evaluation value so that the brightness level of the image data becomes an appropriate level. Also, in the image processing unit 302, the brightness of the pixel data output from the photoelectric conversion element 100 can be changed by the digital gain.

[0049] The storage unit 304 is a storage medium capable of temporarily storing and reading out image data output by the image processing unit 302. Furthermore, the storage unit 304 is also used as a storage area for computer programs executed by a CPU 311 of the control unit 303 (described later), a storage area for various parameters, and a work area during program execution.

[0050] The communication unit 310 converts the image data into a format that complies with a communication protocol, and distributes the image data to an external device (client device) (not shown). More specifically, the communication unit 310 performs compression encoding processing such as H.264 or H.265.

[0051] The communication unit 310 also receives setting commands for various parameters from the external device, outputs them to the control unit 303, and transmits a response to the external device. The external device can instruct the imaging device 300 to zoom or focus, and can receive a distributed image and display the image enlarged / reduced on the display unit of the external device.

[0052] Furthermore, the imaging device 300 is capable of acquiring operation information and control information of an external device. In this embodiment, a system including the external device and the imaging device 300 is called an imaging system.

[0053] The exposure parameter determination unit 305 determines the exposure parameters of the imaging device 300. The exposure parameters in this embodiment include at least one of the exposure time (photoelectric conversion time) of the photoelectric conversion element, the digital gain, and the aperture value, and at least one of the transition amount (amount of change) or transition time (number of transitions, transition interval, and transition timing).

[0054] It should be noted that the digital gain here refers to the digital gain in the image processor 302 that processes the image signal output from the photoelectric conversion element, and the aperture value refers to the aperture value of the optical system that makes an image incident on the photoelectric conversion element.

[0055] Here, the exposure parameter determination unit 305 functions as an exposure parameter control unit that controls exposure parameters such as the exposure period of the photoelectric conversion element according to the type of event. In the following embodiments, exposure and exposure are used interchangeably. In the following embodiments, transition and change are used interchangeably.

[0056] The voltage parameter determination unit 306 determines the reverse bias voltage VR of the photoelectric conversion element 100. The voltage parameters include either the transition amount or transition time (the number of transitions, the transition interval, and the transition timing) of the reverse bias voltage VR.

[0057] The voltage parameter determination unit 306 changes the determined voltage parameters to the imaging power supply unit 307. Here, the voltage parameter determination unit 306 functions as an exposure parameter control means for controlling the reverse bias voltage applied to the avalanche photodiode. Also, the voltage parameter determination unit 306 determines the transition amount or transition time of the voltage parameters depending on the type of event.

[0058] The imaging power supply unit 307 includes a circuit for variably controlling the output voltage supplied to the photoelectric conversion element 100. In this embodiment, the output voltage is adjustable by rewriting a register of a control IC included in the imaging power supply unit 307, but this is not limiting. For example, the imaging power supply unit 307 may include multiple power sources, and one of the multiple power sources to supply power to the photoelectric conversion element 100 may be selected based on a command from the control unit 303.

[0059] An aperture control unit 308 controls an aperture mechanism 309 so as to achieve the aperture amount determined by the exposure parameter determination unit 305. The control unit 303 has a built-in CPU 311 as a computer, and executes a computer program stored in a storage unit 304 as a storage medium by the CPU 311, thereby comprehensively controlling each component of the imaging device 300. The control unit 303 also sets various parameters and issues instructions for transmitting and receiving data.

[0060] Fig. 7 is a flowchart showing an example of an imaging method by the control unit 303 according to embodiment 1. Note that the operation of each step in the flowchart in Fig. 7 is performed sequentially by a CPU or the like as a computer in the control unit 303 executing a computer program stored in a memory. Also, the flow in Fig. 7 is periodically executed at a predetermined cycle.

[0061] First, in step S1000, the control unit 303 determines whether or not an event has been detected. In this embodiment, a case will be described in which the event is an instruction to autofocus the imaging device 300. Here, step S1000 functions as an event detection step (event detection means) that detects an event.

[0062] When the imaging device 300 receives an autofocus instruction from a user's external device via the communication unit 310, it determines that an event has been detected (yes) and proceeds to step S1001. If an event has not been detected, the imaging device 300 returns to step S1000 and waits until an event is detected.

[0063] In step S1001, the control unit 303 determines the amount of sensitivity transition. The amount of sensitivity transition is determined based on the type of event or the value of captured pixel data. In this embodiment, an example will be described in which the event is an autofocus command, and autofocus is performed smoothly by acquiring a high-quality image with an improved S / N ratio.

[0064] That is, in order to improve the S / N ratio, the reverse bias voltage VR is changed to increase the sensitivity by, for example, 12%.

[0065] In this embodiment, in an environment where the subject is bright and prone to whiteout, the sensitivity is not increased by changing the reverse bias voltage. Therefore, the sensitivity is increased by transitioning the reverse bias voltage only when the exposure time before the transition of the reverse bias voltage is not saturated on the whiteout side (shorter exposure time side) and there is a margin of one or more steps until saturation.

[0066] If there is not enough room for one step or more until saturation, that is, if the brightness of the subject is above a predetermined level, or the exposure time is below a predetermined time, or the aperture value is above a predetermined value, the reverse bias voltage as a voltage parameter is not transitioned and the transition amount is set to 0. If the transition amount is 0, this flow may be ended at this point. Also, if the current transition amount (before the transition of the reverse bias voltage) is 0, the reverse bias voltage VR is set to -31.0V.

[0067] In this embodiment, it is assumed that AGC (Auto Gain Control) is performed using digital gain in the image processing unit 302. In this case, when the digital gain is small, it is assumed that the subject is sufficiently bright and the S / N is sufficiently high.

[0068] Therefore, when the digital gain is equal to or less than a predetermined value, the transition of the reverse bias voltage in this embodiment may not be performed. Note that, in this embodiment, the digital gain can be changed by control from the control unit 303.

[0069] In step S1002, the control unit 303 determines the voltage parameters. Since an image is generally blurred before autofocus, even if the image quality (brightness or noise) changes slightly by changing the reverse bias voltage VR, this is often not noticeable.

[0070] Therefore, the time until autofocus starts is shortened by making the transition of the reverse bias voltage VR steep. To achieve this, the transition amount of the reverse bias voltage (12% of the sensitivity) is made in, for example, one transition.

[0071] There is a linear characteristic between the transition amount ΔX [V] of the reverse bias voltage and the transition amount ΔY [%] of the sensitivity of the photoelectric conversion element 100. If the coefficient of the linear characteristic is A, it can be expressed by the following formula 1. -ΔX=A×ΔY (Equation 1)

[0072] For example, when ΔY[%] is about 6%, ΔX[V] is -100mV, and when ΔY[%] is about +12%, ΔX[V] is -200mV. Any deviations from the linear characteristics may be corrected appropriately.

[0073] Here, to increase the sensitivity by 12%, the transition amount ΔX [V] of the reverse bias voltage VR is set to -200 mV. If the reverse bias voltage before the transition is VRa and the reverse bias voltage after the transition is VRb, it can be expressed by the following equation 2. VRb=VRa+ΔX...(Formula 2)

[0074] Therefore, when the reverse bias voltage VRa before the transition is −31.0 [V], the reverse bias voltage VRb after the transition can be expressed by the following equation 3. VRb=-31.0-0.2=-31.2[V]...(Formula 3)

[0075] In step S1003, the control unit 303 determines the exposure parameters based on the voltage parameters. In this embodiment, in order to improve the focus evaluation value (precision), the digital gain is reduced by the amount of increase in sensitivity due to the transition of the reverse bias voltage to improve the S / N ratio. Therefore, the digital gain is reduced by 12% from the current value. The number of transitions is also set to one, the same as the voltage parameters.

[0076] In step S1004, the control unit 303 changes the voltage parameters and exposure parameters. The voltage parameters and exposure parameters are changed in synchronization with each other by the control unit 303, and are changed for the image of the same frame. That is, the voltage parameters and exposure parameters are both controlled for the same frame. The transition (change) of the voltage parameters is controlled so as to be completed during a blanking period before the start of the exposure period.

[0077] Here, step S1004 functions as a voltage parameter control step (voltage parameter control means) for controlling a voltage parameter related to the reverse bias voltage of the avalanche diode of the photoelectric conversion element, and also functions as an exposure parameter control step (exposure parameter control means) for controlling the exposure parameter of the photoelectric conversion element.

[0078] Furthermore, step S1004 functions as a control step (control means) that increases and decreases the sensitivity of the photoelectric conversion element by the voltage parameter and decreases and increases the exposure amount or digital gain of the photoelectric conversion element by the exposure parameter in conjunction with each other. Also, by changing the voltage parameter and the exposure parameter according to the type of event by this step S1004, the quality of the image obtained from the photoelectric conversion element is controlled according to the type of event.

[0079] This allows shooting at the same sensitivity throughout the entire exposure period. In addition, changes to the digital gain as an exposure parameter are made to the pixel data output after the reverse bias voltage is changed. Here, the reverse bias voltage is changed to VRb=-31.2V, and the digital gain is reduced by 12%. This makes it possible to prevent abrupt changes in image brightness caused by transitions in the reverse bias voltage.

[0080] In step S1005, the control unit 303 determines whether the transition of the voltage parameters and the exposure parameters is complete. If the transition is to be performed once, the transition is completed in one frame, and the process proceeds to step S1006. If the transition is to be performed multiple times, step S1004 is repeated until the transition is completed.

[0081] In step S1006, the control unit 303 starts camera / application control. Here, autofocus, which is camera control, is performed. For controls such as autofocus that can wait for the start of control, it is desirable to control them after the transition of the voltage parameters and exposure parameters is completed.

[0082] In step S1007, the control unit 303 detects the end of the camera / application control. Here, the control unit 303 determines the end of autofocus.

[0083] In step S1008, after the autofocus control ends, the control unit 303 returns the voltage parameters and the exposure parameters by the same amount as the transition amount performed in step S1004. Since the event has already ended, there is little need to return the transition amount abruptly. Therefore, the transition amount may be divided and returned in multiple transitions.

[0084] As a result, in step S1008, the brightness and noise of the image do not suddenly change, and the sense of discomfort felt by the user can be suppressed. Also, at this time, by simultaneously controlling the voltage parameters and exposure parameters in the same way as in step S1004, the change in brightness of the image data can be suppressed.

[0085] Next, the timing of changing the voltage parameters and the exposure parameters will be described with reference to Fig. 8. Figs. 8(A) to (D) are diagrams showing examples of the timing of changing the voltage parameters and the exposure parameters according to the first embodiment.

[0086] Fig. 8(A) is a diagram showing a waveform of a reverse bias voltage, Fig. 8(B) is a diagram showing a change in brightness of pixel data read out from photoelectric conversion element 100, Fig. 8(C) is a diagram showing a waveform of digital gain, and Fig. 8(D) is a diagram showing a change in brightness of an image read out from an imaging device. In Figs. 8(A) to (D), the horizontal axis represents time.

[0087] The imaging interval for one frame is indicated by a dotted line. The frame 400 is a frame in which the voltage parameters and exposure parameters are changed after the start of an event is detected.

[0088] The transition of the reverse bias voltage VR will be described with reference to Fig. 8(A). The voltage parameter is changed from VRa to VRb immediately before the next frame 400 after the start of the event is detected. At this time, it is desirable to complete the change (transition) of the voltage parameter during the blank period immediately before the exposure period of the frame 400 to be captured.

[0089] By changing the voltage parameters, the brightness of the pixel data read out from the photoelectric conversion element 100 captured in frame 400 becomes brighter due to improved sensitivity. Meanwhile, as shown in FIG. 8C, the digital gain is reduced by the amount of the increased sensitivity. The digital gain is calculated for the captured pixel data. Therefore, the digital gain is reduced at the timing when the pixel data of the photoelectric conversion element 100 in frame 400 becomes brighter.

[0090] As shown in Fig. 8(D), in the image data read out from the imaging device 300, the digital gain is transitioned in a direction that cancels out the change in brightness due to the transition of the reverse bias voltage, so the brightness of the image data is approximately constant. Also, while the image during the period 401 in which the voltage parameters are being changed has high sensitivity, the digital gain is suppressed, resulting in a high-quality image with a good S / N ratio. This allows the scene from the start to the end of the event to be captured in high quality.

[0091] In this embodiment, as described above, an example is described in which the event is an instruction for autofocusing of the imaging device 300, and contrast AF (autofocus), for example, is described as focus control.

[0092] The imaging device 300 calculates a focus evaluation value by calculating a contrast ratio from pixel data captured by the photoelectric conversion element 100. The imaging device 300 determines an optimal focus position by searching for a focus position at which the focus evaluation value peaks while changing the focus position.

[0093] Although the focus control has been described using contrast AF (autofocus) as an example, the present invention is not limited to this. For example, the focus control may be performed using a phase difference AF method. Autofocus is also fine.

[0094] 7 may be performed by a CPU on the imaging device 300 side, or may be performed by a CPU on the side of an external device (client device) for controlling the imaging device 300.

[0095] In the above description, the brightness of the subject does not change after the voltage and exposure parameters are changed, but if the brightness of the subject changes, the digital gain and exposure time may be changed in response to the change. However, the transition amount when the voltage and exposure parameters are returned does not change.

[0096] In addition, although it is desirable to complete the change of the voltage parameters before exposing the frame to be captured, the voltage parameters may change during the exposure time. In that case, it is desirable to correct the sensitivity fluctuation caused by the reverse bias voltage change during the exposure period by using the exposure parameters or digital gain to cancel it out.

[0097] That is, if the rising and falling edges of the waveform in FIG. 8(A) have a slope, then the rising and falling edges of the waveform in FIG. 8(C) should also have a slope to match.

[0098] <Embodiment 2> In the first embodiment, an example was shown in which the change in the reverse bias voltage was cancelled by using digital gain as an exposure parameter, but in the second embodiment, an example is shown in which the change is cancelled by using exposure time and aperture as exposure parameters. Since the control method is similar for exposure time and aperture, the exposure time will be taken as an example for explanation.

[0099] While digital gain is a parameter for the output of pixel data, exposure time is a parameter for the input of pixel data. In either case, the point that the brightness is corrected by changing the exposure parameter is the same for image data whose sensitivity has been increased or decreased by changing the voltage parameter.

[0100] The difference between the first and second embodiments is that the exposure time is changed as an exposure parameter, and changes in the brightness of the image are detected and treated as a motion detection event, and image recognition such as facial recognition is performed.

[0101] The flow of the second embodiment is similar to that of Fig. 7, and the flow of the second embodiment will be described with reference to Fig. 7. In step S1000 of Fig. 7, when the brightness of continuously captured images changes by a predetermined value or more, the control unit 303 detects this as a moving object detection event and determines yes.

[0102] In step S1002, the control unit 303 determines voltage parameters in the same manner as in the first embodiment, and in step S1003 determines exposure parameters. The exposure parameters are the amount of transition of the exposure time and the transition time (the number of transitions, the transition interval).

[0103] In step S1004, the control unit 303 changes the voltage parameters and the exposure parameters. In step S1005, the control unit 303 determines whether the transition is complete as in the first embodiment, and if yes, in step S1006, performs image recognition such as face recognition on the detected moving object area.

[0104] In step S1007, the control unit 303 detects the end of face authentication (image recognition), and in step S1008, restores the voltage parameters and exposure parameters to their original values ​​in the same manner as in the first embodiment.

[0105] Next, the timing of changing the voltage parameters and the exposure parameters will be described with reference to Fig. 9. Figs. 9(A) to (C) are diagrams showing examples of the timing of changing the voltage parameters and the exposure parameters according to the second embodiment.

[0106] Figure 9(A) shows the waveform of a reverse bias voltage, Figure 9(B) shows the change in exposure time or aperture opening due to a change in exposure parameters, and Figure 9(C) shows the change in brightness of an image read out from an imaging device, where the horizontal axis in Figures 9(A) to (C) represents time.

[0107] 9(A) and (B), the voltage parameters and exposure parameters are changed before the start of exposure in the frame after the start of the event. This is because the exposure time is a parameter that should be changed during exposure, whereas the digital gain in the first embodiment is a process for pixel data after exposure.

[0108] Figure 9 shows an example in which the reverse bias voltage VR and the exposure time are transitioned (changed) at the same time. Also, unlike Figure 8, the brightness of the pixel data read from the photoelectric conversion element does not change in Figure 9. Note that the exposure time of the image during the period in which the voltage parameters are being changed is shortened by the amount of the higher sensitivity.

[0109] Specifically, the exposure time (photoelectric conversion time) is shortened by shortening the counting period from when the counter circuit 211 is reset to when the counting ends within a frame period. This results in a high-quality image with less subject blur, and scenes where a moving object is detected can be captured with high quality, enabling highly accurate image recognition (subject recognition).

[0110] In this way, in the second embodiment, it is possible to obtain a high-quality image with suppressed subject blur by shortening the exposure time by an amount corresponding to the sensitivity associated with the transition of the reverse bias voltage. This makes it possible to obtain a high-quality image of a moving object detected by a moving object detection event, thereby improving the accuracy of image recognition of the moving object.

[0111] In the second embodiment, an example of face recognition (image recognition) is shown in step S1006 of FIG. 7, but it is also possible to simply photograph or record a moving object without subject blur, and control processing such as face recognition (image recognition) may be omitted.

[0112] In this case, in step S1007, if a moving object cannot be detected, or after a predetermined period of time has elapsed, event detection is terminated. In this way, even if a high-quality image is captured immediately after an event is detected without involving camera control such as face recognition (image recognition) or application control, the user can check the image in high quality. Therefore, even a relatively small subject or small movement can be easily checked.

[0113] In addition, although an example of changing the exposure time as an exposure parameter has been described in Fig. 9, it is possible to similarly apply any parameter related to exposure. For example, even when changing the aperture instead of the exposure time, the transition is made in the same way as the exposure time shown in Fig. 9.

[0114] In addition, when controlling the aperture, the depth of field can be changed by changing the sensitivity by transition of the reverse bias voltage. Note that both the exposure time and the aperture may be changed as exposure parameters according to a preset exposure program diagram.

[0115] In the first and second embodiments, after an event is detected, the sensitivity is increased by the transition of the reverse bias voltage, but the sensitivity may be decreased. That is, in the first and second embodiments, the reverse bias voltage VR is changed from, for example, -31.0 V to, for example, -31.2 V to increase the sensitivity.

[0116] However, the sensitivity may be reduced by changing the reverse bias voltage VR from, for example, −31.0 V to, for example, −30.8 V. Reducing the sensitivity in this manner is effective, for example, when capturing a light trail image using long exposure photography, when taking portraits with the aperture set to the widest aperture, when switching to power saving mode due to event detection, etc.

[0117] Also, if the absolute value of the reverse bias voltage is increased (-31.0V to -31.2V) to improve sensitivity, the power consumption increases accordingly as the absolute value of the voltage increases. Therefore, in step S1007 of Fig. 7, a predetermined time limit (e.g., 60 seconds) may be set for detecting the end of the event. In other words, the transition of the voltage parameter may be returned after the predetermined time has elapsed.

[0118] By doing so, even if the event does not end (for example, a moving object continues to exist), the change in the reverse bias voltage may be terminated after a predetermined time has elapsed. Also, if the frequency of detecting events is high, the event detection may be performed at a predetermined interval (for example, 60 seconds). Also, although 60 seconds is used as an example here, the event detection interval may be set to a different time interval depending on the type of event.

[0119] Furthermore, if the time required for controlling the camera / application from step S1006 to step S1007 in FIG. 7 can be predicted, the transition amount of the reverse bias voltage may be determined based on the predicted control time.

[0120] For example, the control time and the transition amount of the reverse bias voltage are inversely proportional to each other, and the transition amount of the reverse bias voltage is reduced as the control time in step S1001 in Fig. 7 becomes longer. This makes it possible to prevent an increase in power consumption due to a longer control time. Also, the transition amount of the reverse bias voltage may be determined from the frequency of events instead of the control time.

[0121] <Embodiment 3> In the first and second embodiments, the exposure parameters and the voltage parameters are synchronously controlled by the control unit 303. In the first and second embodiments, the exposure parameters are determined in accordance with the voltage parameters.

[0122] However, in the third embodiment, the exposure parameters and the voltage parameters are controlled asynchronously. That is, the transition of the voltage parameters and the transition of the exposure parameters are performed asynchronously. Also, in the third embodiment, the transition amount and transition time (number of transitions, transition interval) of the voltage parameters are determined based on the exposure parameters.

[0123] For example, in video shooting, if the brightness of the image is changed suddenly even though there is no change in the subject, the visibility will deteriorate. Therefore, the control unit 303 changes the brightness slowly over multiple frames due to the digital gain, exposure time, or aperture.

[0124] For example, when changing the brightness by one step using the exposure parameter, an example of the transition taking 1 second will be described. If the transition speed of the exposure parameter at this time is expressed as the transition speed Vex, which is the ratio of the number of sensitivity steps and the transition time, it can be expressed as the following formula 4. Transition speed Vex [steps / s] = number of sensitivity steps [steps] / time [s] = 1 (Equation 4)

[0125] In this case, if the minimum changeable transition amount of the exposure of the image capture device 300 is 0.1 stops, the minimum transition time for changing the exposure is 0.1 seconds. Also, if the image capture device captures moving images at 30 fps (frames per second), the minimum transition time (number of frames) for changing the exposure is 3 frames.

[0126] In addition, the number of steps can be expressed by the following formulas 5 and 6. For example, if the sensitivity increases by 12%, the image becomes 0.16 steps brighter. Number of steps [steps] = Log2(magnification) (Equation 5) Number of stages [stages] = Log2(1.12) = 0.16 (Equation 6)

[0127] Next, a procedure when the voltage parameters and the exposure parameters are asynchronous will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of an imaging method by the control unit 303 according to the third embodiment. Note that the operation of each step in the flowchart in Fig. 10 is performed sequentially by a CPU or the like as a computer in the control unit 303 executing a computer program stored in a memory. Also, the flow in Fig. 10 is periodically executed at a predetermined cycle.

[0128] 10, steps with the same numbers as those in FIG. 7 indicate the same processes, and the description thereof will be omitted. In step S2002, the transition amount of the reverse bias voltage VR is determined as in the first embodiment. Here, an example in which the transition amount of the sensitivity is set to, for example, 12% as in the first embodiment will be described. However, in the third embodiment, the total transition amount of the voltage parameters is determined based on the type of event.

[0129] In step S2003, the voltage parameter is determined based on the exposure parameter, by referring to the settable minimum transition amount and minimum transition time when changing the exposure, which is the exposure parameter.

[0130] First, it is determined whether the transition amount of the reverse bias voltage is less than the minimum transition amount that can be set by using the following formula 7. However, in formula 7, the transition amount is expressed by converting it into sensitivity. Transition amount < minimum configurable transition amount (Equation 7)

[0131] In this case, if the transition amount is less than the minimum transition amount that can be set, the change in brightness due to the exposure change will be greater, and the change in brightness due to the change in reverse bias voltage will be less noticeable to the user. Therefore, when Equation 7 is true, the number of transitions is set to one, and the transition amount is changed all at once.

[0132] In this embodiment, since the transition amount of 0.16 steps is greater than the minimum settable transition amount of 0.1 steps, the reverse bias voltage is transitioned in multiple steps.

[0133] Here, the transition amount is divided in half, and the number of divisions when the divided transition amount becomes smaller than the smallest settable transition amount is set as the number of transitions. Here, dividing 0.16 steps in half results in 0.08 steps, which is less than 0.1 steps. Therefore, the transition amount per time is set to 0.08 steps, and the number of transitions is set to 2. In other words, when transitioning a voltage parameter, the transition may be divided and made to occur over multiple frames.

[0134] Next, let's look at the minimum transition time that can be set when changing exposure. It is desirable to leave an interval of at least three frames between voltage parameter changes, which is the minimum transition time (number of frames) that can be set when changing exposure. If the reverse bias voltage were to transition at intervals of one frame, the exposure parameter transition speed would not be able to keep up, and the change in image brightness would be noticeable.

[0135] Incidentally, in the first and second embodiments as well, when the number of transitions is divided into multiple times, it is desirable to change the voltage parameters at intervals of three or more frames, which is the minimum transition time (number of frames) that can be set when changing exposure.

[0136] In step S2004, the control unit 303 changes the voltage parameters in accordance with the transition amount and transition time of the voltage parameters determined in step S2003.

[0137] In step S2005, the control unit 303 determines whether the change of the voltage parameter is complete. At this time, it may also determine whether the exposure is stable from the image or exposure setting value. Alternatively, it may provide a sufficient settling time until the exposure is stable, and determine whether the settling time has elapsed.

[0138] If the answer is yes in step S2005, the process proceeds to step S1006, and if the answer is no, the process returns to step S2004. Steps S1006 to S1008 are the same as those in FIG.

[0139] Next, the timing of changing the voltage parameters and the exposure parameters will be described with reference to Figures 11(A) to (D). Figures 11(A) to (D) are diagrams showing examples of the timing of changing the voltage parameters and the exposure parameters according to the third embodiment.

[0140] Fig. 11(A) is a diagram showing the waveform of a reverse bias voltage, Fig. 11(B) is a diagram showing the change in exposure time or aperture opening amount due to a change in exposure parameters, and Fig. 11(C) is a diagram showing the change in brightness of pixel data read out from photoelectric conversion element 100. Also, Fig. 11(D) is a diagram showing the change in brightness of an image read out from an imaging device. In Figs. 11(A) to (D), the horizontal axis represents time.

[0141] Figure 11 shows an example where the reverse bias voltage transition count is set to two. The first voltage parameter change is made in the blank period immediately before the first frame after an event is detected. The second voltage parameter change is made in the blank period immediately before a specified frame after an interval of five frames.

[0142] 11 shows an example in which the frame interval is 5 frames, but the frame interval is not limited to 5. While the voltage parameter is actively changed, the exposure parameter is changed when the brightness of the imaging pixel of the photoelectric conversion element 100 is calculated and the difference from the appropriate value (target brightness) is equal to or greater than a predetermined value.

[0143] In order to suppress hunting in exposure changes, the brightness to be calculated is a moving average of multiple frames, and the exposure is changed if the difference between the moving average and the appropriate value is equal to or greater than a predetermined value. Hysteresis is also provided by making the appropriate value different when increasing and decreasing the exposure.

[0144] Therefore, the timing of the change in the voltage parameter and the change in the exposure are not necessarily the same. Also, in Fig. 11, an example is shown in which the moving average of the captured pixels for five frames enclosed by the dotted line 402 exceeds a predetermined optimum value by a predetermined value or more, and the exposure time is changed to be shorter.

[0145] As described above, in this embodiment, by determining the voltage parameters based on the exposure parameters, even if the voltage parameters and the exposure parameters are asynchronous, it is possible to obtain a high-quality image in which subject blur is suppressed by shortening the exposure time. Note that, although an example of shortening the exposure time has been shown, control to reduce the digital gain or increase the aperture value may also be used.

[0146] Although the above embodiment shows an example in which the transition amount of the reverse bias voltage is divided in half, the transition amount does not have to be divided in half.

[0147] Furthermore, the minimum number of steps by which the exposure time can be changed depends on the configuration and characteristics of the photoelectric conversion element 100, and in some cases only a rough setting such as 1 / 3 steps can be possible. In contrast, the digital gain only changes the numerical value used in the image processing calculation, and can be finely set such as 1 / 10 steps.

[0148] Therefore, when the minimum number of steps in which the exposure time can be changed as shown in the second embodiment is too rough and the same transition amount as the transition amount of the reverse bias voltage cannot be set, causing a sensitivity step difference, the sensitivity step difference may be corrected by digital gain. That is, the exposure time or aperture value may be controlled as the exposure parameter, and the sensitivity step difference that occurs when the voltage parameter is controlled may be corrected by digital gain.

[0149] This makes it possible to keep the brightness constant with high precision even when the reverse bias voltage is changed. Note that, although the above description has been given of the case where the minimum number of steps at which the exposure time can be changed is coarse, the same applies to the case where the initial number of steps of aperture control is coarse.

[0150] Note that, as examples of events, an autofocus command has been described in the first embodiment, and detection of a moving object has been described in the second embodiment, but the present invention is not limited to these. For example, a change in brightness of a captured image, detection / change of an edge, or detection / change of a feature point may be detected as an event.

[0151] Alternatively, an event may be detected when an image recognition score such as person authentication exceeds a threshold value. Alternatively, an event may be detected when an image change is detected, such as when an object is left behind or taken away.

[0152] In addition, the camera's automatic controls, such as pan / tilt / zoom driving by preset rotation, pan / tilt / zoom driving including tracking operation, insertion / removal of an infrared cut filter, insertion / removal of an ND (Neutral Density) filter, etc. may be detected as events. Furthermore, the flash on / off mode switching may be detected as an event.

[0153] In addition to the photoelectric conversion element 100, input information from a sensing device mounted on the imaging device 300 may also be regarded as an event. For example, sound information (screams, loud sounds, sounds of specific frequencies, etc.) acquired from a microphone as a sensing device, or distance information input from a ToF (Time of Flight) sensor may also be regarded as an event.

[0154] Alternatively, the event may be angular velocity information input from a gyro sensor, velocity information input from an acceleration sensor, or light quantity information from a visible light sensor or infrared light sensor other than the photoelectric conversion element 100, which are sensing devices.

[0155] Furthermore, the process for detecting an event based on a captured image and an application executed based on an event may be executed on the client device connected to the image capture device 300, rather than on the image capture device 300. The client device may acquire images in real time, perform image processing such as face recognition and people counting, and detect an event based on the results of this image processing.

[0156] Also, when a user is operating in real time while viewing a camera image, the user's input operation may be regarded as an event, for example, an instruction to operate pan-tilt-zoom, an instruction to enlarge the display using electronic zoom, an instruction to specify an area of ​​interest on the screen, etc.

[0157] Furthermore, when the user is paying attention to something, such as when enlarging with electronic zoom or specifying an area of ​​interest, for example when the reverse bias voltage is being changed to increase sensitivity, a message such as "Sensitivity UP" may be displayed on the screen of the display device of the client device.

[0158] Furthermore, when the user selects various events, the user may be able to check the difference in image quality by comparing or switching on the display screen how the image quality of the images displayed changes depending on the type of event.

[0159] Also, although an example has been shown in which the reverse bias voltage is increased to improve sensitivity when an event is detected, the reverse bias voltage may be decreased (e.g., from -31.0 V to -30.8 V) in order to reduce power consumption when an event is not detected.

[0160] In the first and second embodiments, the means for changing the exposure parameters may be determined depending on the type of event or control to be performed after the event. For example, when attention is paid to noise, the digital gain may be changed, when attention is paid to movement, the exposure time may be changed, and when attention is paid to blur, the aperture may be changed.

[0161] In other words, by changing the digital gain, it is possible to change the amount of noise when shooting at the same brightness, by changing the exposure time, it is possible to change the degree to which the subject is blurred, and by changing the aperture, it is possible to change the amount of blur of the subject.

[0162] Moreover, although the example in which the exposure parameter is any one of the digital gain, the exposure time, and the aperture has been shown, the exposure parameter may be changed by combining two or more of the digital gain, the exposure time, and the aperture.

[0163] The amount of transition of the reverse bias voltage may also be determined depending on the type of event. For example, a moving object detection event can be detected by knowing the change in brightness, so the amount of transition of the reverse bias voltage (sensitivity increase) is set small. In contrast, a face recognition event requires a higher resolution image, such as for detecting feature points, so the amount of transition of the reverse bias voltage (sensitivity increase) is set large.

[0164] In addition, since increasing the digital gain deteriorates the S / N ratio, by increasing the sensitivity by changing the reverse bias voltage and decreasing the digital gain, it is possible to obtain a high-quality image with an improved S / N ratio. This is particularly effective when calculating evaluation values ​​based on user visibility and image processing.

[0165] In addition, since the exposure time contributes to subject blur, by shortening the exposure time by changing the reverse bias voltage (increasing the sensitivity), it is possible to obtain a high-quality image in which subject blur caused by movement is suppressed. On the other hand, by lengthening the exposure time by changing the reverse bias voltage (decreasing the sensitivity), it is possible to capture an afterimage (light trail image, etc.) caused by the movement of the subject, and to obtain a high-quality image.

[0166] In addition, since the aperture contributes to the depth of field, by controlling the aperture to close by changing the reverse bias voltage (increasing sensitivity), the depth of field can be expanded, and a high-quality image with a wide range of resolution can be obtained. Conversely, by controlling the aperture to open by changing the reverse bias voltage (decreasing sensitivity), the depth of field can be narrowed, and a high-quality image with a blurred background suitable for portrait photography can be obtained.

[0167] 12 is a diagram showing an example of the configuration of an imaging system including an imaging device 300 according to embodiments 1 to 3. The imaging system includes the imaging device 300, a network 500, a client device 501, a display device 502 (display unit), and an input device (input unit) 503. In this manner, the imaging system includes the imaging device 300 having a photoelectric conversion element, and a client device as an external device connected to the imaging device 300 via the network 500 as a communication path.

[0168] The imaging device 300 is capable of communicating with a client device 501 via a network 500. The imaging device 300 captures an image of a subject to generate an image, and transmits the captured image to the client device 501 via the network 500. A display device 502 and an input device 503 are connected to the client device 501, and an image sent from the network 500 is displayed on the display device 502.

[0169] The input device 503 includes a keyboard, a mouse, and the like, and is used to operate the UI of a display device connected to the client device 501. Also, instructions and operations can be given to the imaging device 300 via the client device 501 and the network 500 in response to operations on the UI.

[0170] In the example shown in FIG. 12, the client device 501, the display device 104, and the input device 105 are separate entities, but the client device 501, the display device 502, and the input device 503 may be integrated into one device, such as a notebook PC having a touch panel display.

[0171] Moreover, the imaging device 300 and the client device 501 do not necessarily need to be connected via the network 500, and may be directly connected to each other. Furthermore, the camera (imaging device 300), the client device 501, the display device 502, and the input device 503 may be integrated into one device, such as a consumer camera having a touch panel display.

[0172] Although the present invention has been described in detail based on the preferred embodiment, the present invention is not limited to the above embodiment, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention includes the following combinations.

[0173] (Configuration 1) An imaging system comprising: a voltage parameter control means for controlling a voltage parameter related to the reverse bias voltage of an avalanche photodiode of a photoelectric conversion element; an exposure parameter control means for controlling an exposure parameter of the photoelectric conversion element; and a control means for controlling the image quality of an image obtained from the photoelectric conversion element by increasing or decreasing the sensitivity of the photoelectric conversion element using the voltage parameter and decreasing or increasing the exposure amount or digital gain of the photoelectric conversion element using the exposure parameter in conjunction with each other.

[0174] (Configuration 2) The imaging system according to configuration 1, wherein the control means changes the voltage parameters and the exposure parameters in accordance with a type of event.

[0175] (Configuration 3) The imaging system according to configuration 2, wherein the exposure parameter control means changes the exposure parameters depending on the type of the event.

[0176] (Configuration 4) The imaging system according to configuration 2 or 3, wherein the voltage parameter control means determines a transition amount or transition time of the voltage parameter depending on the type of the event.

[0177] (Configuration 5) The imaging system according to any one of configurations 1 to 4, wherein the voltage parameter includes either a transition amount or a transition time of the reverse bias voltage.

[0178] (Configuration 6) An imaging system described in any one of configurations 1 to 5, characterized in that the exposure parameters include either a transition amount or a transition time of at least one of the exposure time of the photoelectric conversion element, the digital gain of the image signal output from the photoelectric conversion element, and the aperture value of an optical system through which an image is incident on the photoelectric conversion element.

[0179] (Configuration 7) The imaging system described in Configuration 6, wherein the control means controls the exposure time or the aperture value as the exposure parameter, and corrects sensitivity differences that occur when controlling the voltage parameter with the digital gain.

[0180] (Configuration 8) The imaging system described in any one of configurations 1 to 7, characterized in that the control means does not transition the voltage parameter when the brightness of the subject is at or above a predetermined level, or the exposure time is less than a predetermined time, or the aperture value is at or above a predetermined value.

[0181] (Configuration 9) The imaging system according to any one of configurations 1 to 8, wherein the control means controls both the voltage parameter and the exposure parameter for the same frame.

[0182] (Configuration 10) The imaging system according to any one of configurations 1 to 9, wherein the control means performs the transition of the voltage parameter and the transition of the exposure parameter asynchronously.

[0183] (Configuration 11) The imaging system according to any one of configurations 1 to 10, wherein the control means, when transitioning the voltage parameter, transitions the voltage parameter in a divided manner over a plurality of frames.

[0184] (Configuration 12) The imaging system according to any one of configurations 1 to 11, wherein the control means returns the transition of the voltage parameter to its original state after a predetermined time has elapsed.

[0185] (Configuration 13) The imaging system according to any one of configurations 1 to 12, wherein the control means performs the transition of the voltage parameter in a blank period before the start of an exposure period.

[0186] (Configuration 14) The imaging system according to any one of configurations 1 to 13, further comprising an imaging device having the photoelectric conversion element, and an external device connected to the imaging device via a communication path.

[0187] (Method) An imaging method comprising: a voltage parameter control step for controlling a voltage parameter related to the reverse bias voltage of an avalanche diode of a photoelectric conversion element; an exposure parameter control step for controlling an exposure parameter of the photoelectric conversion element; and a control step for controlling the image quality of an image obtained from the photoelectric conversion element by increasing or decreasing the sensitivity of the photoelectric conversion element using the voltage parameter and decreasing or increasing the exposure amount or digital gain of the photoelectric conversion element using the exposure parameter in conjunction with each other.

[0188] (Program) A computer program for controlling each of the means of the imaging system according to any one of configurations 1 to 14 by a computer.

[0189] In order to realize a part or all of the control in the above-mentioned embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to an imaging device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the imaging device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0190] 100: Photoelectric conversion element 11: Sensor board 12: Pixel area 21: Circuit board 22: Circuit area 101: Pixel 102: Photoelectric conversion unit 103: Signal processing circuit 110: Vertical scanning circuit 111: Horizontal scanning circuit 112: Readout circuit 113: Signal line 114: Output circuit 115: Control pulse generation unit 201: Avalanche photodiode 202: Quench element 210: Waveform shaping section 211: Counter circuit 212: Selection circuit 213: Drive wire 300: Imaging device 301: Imaging optical system 302: Image processing unit 303: Control unit 304: Storage section 305: Exposure parameter determination unit 306: Voltage parameter determination unit 307: Imaging power supply unit 308: Aperture control unit 309: Aperture mechanism 310: Communications Department

Claims

1. a voltage parameter control means for controlling a voltage parameter relating to a reverse bias voltage of the avalanche photodiode of the photoelectric conversion element; an exposure parameter control means for controlling exposure parameters including an exposure amount and a gain of the photoelectric conversion element in accordance with the voltage parameter, and determining a transition amount of the exposure amount or a transition amount of the gain based on the transition amount of the exposure parameter; and a control means for controlling the quality of an image obtained from the photoelectric conversion element by increasing or decreasing the sensitivity of the photoelectric conversion element using the voltage parameter and decreasing or increasing the exposure amount or the gain of the photoelectric conversion element using the exposure parameter in conjunction with each other.

2. 2. The imaging system according to claim 1, wherein the control means changes the voltage parameter and the exposure parameter depending on the type of event.

3. 3. The imaging system according to claim 2, wherein the exposure parameter control means changes the exposure parameters depending on the type of the event.

4. 3. The imaging system according to claim 2, wherein the voltage parameter control means determines the transition amount or transition time of the voltage parameter depending on the type of the event.

5. 2. The imaging system according to claim 1, wherein the voltage parameter includes either a transition amount or a transition time of the reverse bias voltage.

6. 2. The imaging system according to claim 1, wherein the exposure parameters include either a transition amount or a transition time of at least one of an exposure time of the photoelectric conversion element, the gain of the image signal output from the photoelectric conversion element, and an aperture value of an optical system through which an image is incident on the photoelectric conversion element.

7. 7. The imaging system according to claim 6, wherein the control means controls the exposure time or the aperture value as the exposure parameter, and corrects sensitivity differences that occur when the voltage parameter is controlled using the gain.

8. 2. The imaging system according to claim 1, wherein the control means does not change the voltage parameter when the brightness of the subject is above a predetermined level, the exposure time is less than a predetermined time, or the aperture value is greater than a predetermined value.

9. 2. The imaging system according to claim 1, wherein said control means controls both said voltage parameter and said exposure parameter for the same frame.

10. 2. The imaging system according to claim 1, wherein the control means performs the transition of the voltage parameter and the transition of the exposure parameter asynchronously.

11. 2. The imaging system according to claim 1, wherein the control means divides the voltage parameter into transitions over a plurality of frames.

12. 2. The imaging system according to claim 1, wherein the control means returns the transition of the voltage parameter to its original state after a predetermined time has elapsed.

13. 2. The imaging system according to claim 1, wherein the control means changes the voltage parameter during a blank period before the start of an exposure period.

14. an imaging device having the photoelectric conversion element; 2. The imaging system according to claim 1, further comprising an external device connected to said imaging device via a communication path.

15. a voltage parameter control step of controlling a voltage parameter related to a reverse bias voltage of an avalanche diode of the photoelectric conversion element; an exposure parameter control step of controlling exposure parameters including an exposure amount and a gain of the photoelectric conversion element in accordance with the voltage parameter, and determining a transition amount of the exposure amount or a transition amount of the gain based on the transition amount of the exposure parameter; and a control step of controlling the image quality of the image obtained from the photoelectric conversion element by increasing or decreasing the sensitivity of the photoelectric conversion element using the voltage parameter and decreasing or increasing the exposure amount or the gain of the photoelectric conversion element using the exposure parameter in conjunction with each other.

16. A computer program for causing a computer to function as each of the means of the imaging system described in any one of claims 1 to 14.

Images