Imaging apparatus, control method for the same, and program

Abstract

To provide an imaging apparatus that can prevent a decrease in image quality even in low-light photography.SOLUTION: The imaging apparatus has a photoelectric conversion element having an avalanche photodiode, a signal processing unit that corrects a signal acquired by the photoelectric conversion element on the basis of position information of a pixel to be corrected, and a control means for controlling the number of pulses of the pulse signal during the period for acquiring signals for one frame. A threshold value for determining whether a signal is one for which the signal processing unit performs correction differs between a first mode and a second mode in which the number of pulses in a pulse signal is less than that in the first mode.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] In recent years, a photoelectric conversion device has been proposed that digitally counts the number of photons arriving at an avalanche photodiode (APD) and outputs the counted value as a photoelectrically converted digital signal from a pixel. In the configuration of Patent Document 1, a pixel having an APD includes the APD, a quench circuit connected to the APD, a signal control circuit to which a signal output from the APD is input, and a pulse generation circuit connected to the quench circuit and the signal control circuit. A method for controlling a pulse signal input to the quench circuit when there are multiple driving modes is also disclosed. For example, it is disclosed how to determine the number and period of pulse signals in a first mode and a second mode for shooting in an environment darker than the first mode (low-illumination shooting).

[0003] The signal of a pixel having an APD includes a signal based on photoelectric conversion in response to incident light, as well as a signal caused by an abnormality in the trap level of the pixel. There are pixels (hereinafter referred to as "defective pixels") that generate a larger amount of signal caused by an abnormality in the trap level of the pixel than the surrounding pixels.

[0004] This defective pixel also generates charges in adjacent pixels, causing cluster-shaped defects (hereinafter referred to as "cluster defects"). Also, shooting in low illuminance is more likely to cause cluster defects than shooting in an environment brighter than the second mode (high illuminance shooting). In particular, when switching from high illuminance shooting to low illuminance shooting, if the number of drive pulse signals is the same, the possibility of cluster defects occurring increases. If an image is formed using the signal output from the cluster defect as is, image quality degradation (decrease in image quality) occurs. Therefore, for example, it is known that information such as the address of the defective pixel in the image sensor (hereinafter referred to as "defect information") is stored in a non-volatile memory at the time of factory shipment, and the defective pixel is corrected by referring to the defect information during actual shooting. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2022-106649 A Summary of the Invention [Problem to be solved by the invention]

[0006] Patent Document 1 does not disclose a method for detecting and correcting defective pixels in acquired signals when there are multiple shooting modes, nor does it disclose changing the number of drive pulse signals when switching from high-illumination shooting to low-illumination shooting.

[0007] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide an imaging device capable of suppressing degradation in image quality even during low-illumination imaging. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an imaging device according to one aspect of the present invention comprises a photoelectric conversion element having an avalanche photodiode, a signal processing unit that performs correction on a signal acquired by the photoelectric conversion element based on positional information of a pixel to be corrected, and a control means that controls the number of pulses of a pulse signal during a period in which one frame's worth of signals is acquired, and is characterized in that a threshold value used by the signal processing unit to determine whether or not a signal requires the correction differs between a first mode and a second mode in which the number of pulses of the pulse signal is fewer than that in the first mode. Effect of the Invention

[0009] According to the present invention, it is possible to provide an imaging device capable of suppressing degradation in image quality even during low-illumination imaging. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an imaging device according to a first embodiment. [Diagram 2] 2 is a diagram showing an internal configuration of a defective pixel detection unit according to the first embodiment. FIG. [Diagram 3] 5 is a diagram showing an example of a format of defective pixel threshold data of the defective pixel detection unit according to the first embodiment; FIG. [Figure 4] FIG. 2 is a diagram illustrating an internal configuration of a defective pixel correction unit according to the first embodiment. [Diagram 5] FIG. 1 is a diagram of a photoelectric conversion element according to a first embodiment. [Figure 6] FIG. 2 is a diagram of a sensor substrate according to the first embodiment. [Figure 7] FIG. 2 is a diagram of a circuit board according to the first embodiment. [Figure 8] 2 is a diagram of an equivalent circuit corresponding to a pixel of the photoelectric conversion element according to the first embodiment. FIG. [Figure 9] FIG. 2 is a timing chart of the photoelectric conversion element according to the first embodiment. [Figure 10] FIG. 2 is a diagram for explaining inter-pixel crosstalk according to the first embodiment. [Figure 11] 10A and 10B are timing charts of a photoelectric conversion element of a conventional photoelectric conversion device under high illuminance and low illuminance conditions. [Figure 12] 1 is a diagram showing the output count numbers of a photoelectric conversion element under high illuminance and low illuminance conditions in a conventional photoelectric conversion device. [Figure 13] 5A to 5C are diagrams illustrating the influence of inter-pixel crosstalk of a photoelectric conversion element under high illuminance and low illuminance conditions according to Example 1. [Figure 14] 4 is a timing chart of the photoelectric conversion element under high illuminance and low illuminance conditions according to the first embodiment. FIG. [Figure 15] 4 is a diagram showing the output count numbers of a photoelectric conversion element under high illuminance conditions and low illuminance conditions according to Example 1. FIG. [Figure 16] 5A to 5C are diagrams illustrating the influence of inter-pixel crosstalk of a photoelectric conversion element under high illuminance and low illuminance conditions according to Example 1. [Figure 17] 11 is a diagram showing the relationship between the number of pulse signals of the control signal CLK and the pixel level of a cluster-shaped defect in the first embodiment. FIG. [Figure 18] 11 is a diagram showing the relationship between the number of CLK pulse signals and the levels of pixels adjacent to a central pixel of a cluster-shaped defect in the first embodiment. FIG. [Figure 19] 4A to 4C are diagrams showing images of defects in an image and data of defective pixels in the image according to the first embodiment. [Figure 20] 6 is a diagram showing a flow of defective pixel detection according to the number of pulse signals of a control signal CLK in the first embodiment. [Figure 21] 6 is a diagram showing a flow of correction of a defective pixel according to the number of pulse signals of a control signal CLK according to the first embodiment. [Figure 22] FIG. 11 is a diagram showing data of a defective pixel according to the second embodiment. [Diagram 23] FIG. 11 is a diagram showing an internal configuration of a defective pixel detection unit according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following describes in detail the embodiments for carrying out the present invention. Note that the embodiments described below are examples for realizing the present invention, and should be appropriately modified or adjusted depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments. Also, in each drawing, parts having the same functions are given the same numbers, and repeated explanations are omitted.

[0012] <Example 1> First, each block in the imaging device 1 of the first embodiment will be described below with reference to Fig. 1. Fig. 1 is a block diagram of the imaging device 1 in this embodiment. The imaging device 1 is configured to include a lens 101, an imaging element 102, a signal processing unit (video signal processing unit) 103, a control unit 104, a RAM 105, a ROM 106, and an output unit 107.

[0013] The lens 101 is an imaging optical system (imaging optical system) that forms an optical image of a subject, and includes a variable magnification lens (zoom lens) that moves in the optical axis direction when changing magnification, and a focus lens that moves in the optical axis direction when adjusting focus (focusing). The lens 101 also includes an aperture and ND that control the amount of incident light, and an IRCF that controls the amount of incident infrared light.

[0014] The image sensor 102 is a photoelectric conversion device that photoelectrically converts the optical image formed by the lens 101 into an electrical signal. The image sensor 102 in this embodiment is an image sensor using an avalanche photodiode (APD). Specifically, it is a photoelectric conversion device configured to include a photoelectric conversion element 400 shown in FIG. 5, which will be described later. An optical image obtained by exposure is output to the outside as a pixel signal from each pixel. In this embodiment, the signal is output to a signal processing unit 103. Details of the image sensor 102 and its operation will be described later.

[0015] The signal processing unit 103 is configured to include, as functional units, a defective pixel detection unit 108, a defective pixel correction unit 109, a video signal generation unit 110, and an image processing unit 111. Each of these functional units is realized by the control unit 104, which will be described later, loading a program stored in the ROM 106 into the RAM 105 and executing it. The control unit 104 then stores the execution results of each process, which will be described later, in the RAM 105 or a predetermined storage medium.

[0016] The signal processing unit 103 performs a correction process on a signal acquired by a photoelectric conversion element 400 (described later) based on position information of a pixel (defective pixel) to be corrected using each of the above functional units. Then, a video signal of a predetermined format is generated based on the corrected pixel signal in frame units. The signal processing unit 103 can multiply the signal acquired by the photoelectric conversion element 400 (described later) by a gain (digital gain) for correction.

[0017] The defective pixel detection unit 108 detects defective pixels that require correction from the output of the image sensor 102. Specifically, it detects defects contained in each signal by using the pixel signals output from the image sensor 102. The defective pixel detection unit 108 stores (records) defect information generated based on defective pixel data (pixel information) that is the result of detection for each pixel in the RAM 105, ROM 106, etc.

[0018] The defective pixel correction unit 109 performs correction processing on the pixel signal output by the image sensor 102 on a frame-by-frame basis based on the result of the defective pixel detection unit 108. That is, the defective pixel correction unit 109 corrects defective pixels by using defect information generated based on defective pixel data detected by the defective pixel detection unit 108 for a signal (pixel signal) acquired by the photoelectric conversion element 400 described below. More specifically, the defective pixel correction unit 109 performs correction using defect correction data selected from defect information stored in the RAM 105, ROM 106, etc. described below to be used for defect correction by the defective pixel correction unit 109. This selection is made by the control unit 104 based on the imaging conditions (ISO sensitivity and exposure time), the temperature of the image sensor 102, and a plurality of driving modes with different pulse numbers of pulse signals during an exposure period in which one frame's worth of signals is acquired from the image sensor 102, etc.

[0019] The video signal generation unit 110 generates a video signal in a predetermined format based on the pixel signals in units of frames corrected by the defective pixel correction unit 109. The image processing unit 111 performs arbitrary image processing and analysis processing on the image signal and the video signal, and outputs the result to each block. Note that the image processing unit 111 may apply a gain to the above signals for correction.

[0020] The signal processing unit 103 further includes an aberration correction process for correcting optical aberration of the lens 101, and a noise reduction processing unit (not shown) for reducing fixed pattern noise and random noise in the pixel signal output from the image sensor 102. The signal processing unit 103 may also include a digital signal processing unit that performs various corrections on the pixel signal and then compresses and encodes the pixel signal into a video signal.

[0021] The control unit (control means) 104 is a central processing unit that controls the entire imaging device 1. The control unit 104 includes a CPU that performs various arithmetic processing and controls the entire imaging device 1. The CPU generally controls each component and sets various setting parameters in order to control the entire imaging device 1. The CPU executes processing in the signal processing unit 103 and each functional unit included in the signal processing unit 103. The CPU also includes a cache memory in which data can be electrically written and erased, and executes programs recorded therein. The memory is used as a storage area for programs executed by the CPU, a work area during program execution, a storage area for data, etc.

[0022] The control unit 104 calculates setting values ​​for the image sensor 102, the lens 101, exposure, the number of pulses of the control signal CLK for the image sensor, etc., based on the analysis results of each signal output from the signal processing unit 103. Specifically, the stored program diagram is used based on the brightness of the subject included in the analysis results by the signal processing unit 103. Then, parameters related to exposure, such as the exposure time to be set for the image sensor 102 and the F-number of the aperture included in the lens 101, are determined. Note that, although various types of processing such as various calculations, detection processing, and correction processing are performed by the signal processing unit 103, a part or all of the various types of processing such as various calculations, detection processing, and correction processing may be performed by the control unit 104. The exposure time here refers to the execution timing of the electronic shutter.

[0023] The RAM 105 temporarily stores the results of calculations by the control unit 104 and the output signals of the signal processing unit 103. This may be provided separately from the cache memory included in the CPU, or may be the same.

[0024] The ROM 106 stores defect information and various adjustment values. The various data stored in the ROM 106 is expanded in the RAM 105 under the control of the control unit 104 at a predetermined timing such as at the time of start-up.

[0025] The output unit 107 can output the captured video or image to an external monitor or recorder, or record the captured video or image on a recording medium such as an SD card, a hard disk (HDD), or a non-volatile memory. Alternatively, the output unit 107 can output the video signal via the Internet. The Internet connection may be wired, or a wireless communication method such as wireless LAN may be adopted.

[0026] The imaging device 1 may be configured to have a recording medium such as the hard disk described above. In this case, the recording medium may be disposed inside the imaging device 1, or may be disposed outside the imaging device 1 and electrically connected to the imaging device 1 via the output unit 107. The imaging device 1 may also be configured to have a display device such as a monitor or a display. In this case, the display device is electrically connected to the imaging device 1 via the output unit 107. The recording medium and the display device may be connected to the imaging device 1 via other lines, not via the output unit 107.

[0027] A method for detecting defective pixels will be described below with reference to Figures 2 and 3. Figure 2 is a diagram illustrating an example of the internal configuration of defective pixel detection unit 108. Figure 3 is a diagram illustrating an example of the format of defective pixel threshold data of defective pixel detection unit 108. Defective pixel detection unit 108 is configured to include, as functional units, a detection unit 201, an address calculation unit 202, and a threshold selection unit 203.

[0028] The detection unit (determination unit) 201 executes a determination as to whether or not the acquired pixel signal is a defective pixel. Specifically, the detection unit 201 inputs pixel signals output from the image sensor 102 to the detection unit 201, and determines whether or not each input pixel signal is a defective pixel. That is, the detection unit 201 determines whether or not the signal acquired by the image sensor 102 is a pixel to be corrected. Thereafter, the detection unit 201 outputs defective pixel data, which is the determination result, and stores it in the RAM 105, the ROM 106, etc. The defective pixel data includes the amount of defect, the address of the defective pixel, the type of defect, etc.

[0029] Here, the defect amount is the output value of the pixel of interest. The defect amount may be an absolute value of the pixel level or a difference between adjacent pixels. The address of the defective pixel indicates the position in the pixel array of the image sensor 102, and may be an absolute position format combining horizontal and vertical coordinates, or a relative position format storing only the relative position (distance) between defective pixels in the signal scanning direction. The type of defect is determined by the cause of the defect. For example, there is a white defect whose level is higher than that of the surrounding pixels, and a black defect whose level is lower than that of the surrounding pixels. The defect amount of each defective pixel depends on different conditions (imaging conditions, environmental conditions, etc.) depending on the cause of the defect. For this reason, pixel signals are acquired under multiple imaging conditions at the time of shipping from the factory, and the results are stored as the type of defect for each condition.

[0030] The address calculation unit 202 inputs the pixel signal output from the image sensor 102 to the detection unit 201 at the same time as inputting the pixel signal output from the image sensor 102 to the detection unit 201, or in conjunction with inputting the pixel signal output from the image sensor 102 to the detection unit 201. The address calculation unit 202 counts the number of pixel signals input in the scanning order of the image sensor 102, and generates address information corresponding to a pixel signal of a target pixel that is a defective pixel.

[0031] The threshold selection unit 203 selects a threshold used to determine whether or not a pixel is defective. The threshold selection unit 203 selects an appropriate threshold from the threshold data for defective pixels depending on the control settings, which are setting information of the shooting conditions in the control unit 104, and the type of defect. Note that the control setting values ​​and the number of corresponding thresholds shown in Fig. 3 are merely examples and can be changed as appropriate.

[0032] FIG. 3A is a diagram showing an example of threshold data for defective pixel type K1. FIG. 3B is a diagram showing an example of threshold data for defective pixel type K2. The data shown in FIG. 3A and FIG. 3B each have two elements: a control setting and a threshold. For example, the defective pixel threshold for control setting M1 is TH1, the defective pixel threshold for control setting M2 is TH2, and the defective pixel threshold for control setting M3 is TH3. Therefore, the threshold selection unit 203 selects a threshold to be used for determining defective pixels from among a plurality of thresholds thus set in advance. When selecting a threshold, the threshold selection unit 203 selects a threshold according to the number of pulses of the pulse signal of the control signal CLK, which will be described in detail later. In addition, the threshold values ​​are set to be smaller as the pulse signal of the control signal CLK decreases.

[0033] For example, in the case of control setting M1, the detection target (pixel to be corrected) in defective pixel detection unit 108 is a pixel whose pixel level exceeds threshold TH1, i.e., a pixel equal to or higher than threshold TH1. Therefore, when defective pixel detection unit 108 determines whether the pixel signal output from image sensor 102 is a defective pixel, if there is a pixel whose pixel level exceeds threshold TH1, it outputs the pixel that exceeds threshold TH1 as a pixel to be corrected, i.e., a defective pixel. In this way, defective pixel detection unit 108 of this embodiment determines whether the pixel signal output from image sensor 102 is a defective pixel based on the threshold selected by threshold selection unit 203.

[0034] The control settings are the shutter speed (exposure time), gain, temperature, etc., and an appropriate threshold is calculated according to the control settings and used to detect defective pixels. In addition, since the amount of a defect differs depending on the type of defect, threshold data is provided for each type of defect.

[0035] A method of correcting defective pixels will be described below with reference to Fig. 4. Fig. 4 is a diagram showing an example of the internal configuration of defective pixel correction unit 109. Defective pixel correction unit 109 is configured to include, as functional units, a correction unit 301, an address calculation unit 202, and a defect information selection unit (data selection unit) 302. Address calculation unit 202 is similar to address calculation unit 202 described above, and therefore description thereof will be omitted.

[0036] The correction unit 301 corrects the defective pixel of the pixel signal input from the image sensor 102 based on information included in the defective pixel data, using the pixel level of the surrounding pixels, and then outputs the correction result.

[0037] The defect information selection unit 302 reads out the defective pixel data selected based on the control settings from the control unit 104 from the RAM 105, the ROM 106, etc. Then, the defect information selection unit 302 transmits the read defective pixel data to the correction unit 301.

[0038] Then, the correction unit 301 identifies the position of the defective pixel from the address input from the address calculation unit 202 and the address (address data) of the defective pixel data received from the defect information selection unit 302, and extracts a correction target for the pixel signal. The correction unit 301 performs correction on the extracted correction target by using surrounding pixels.

[0039] A configuration common to the photoelectric conversion devices in the following embodiments including Example 1 will be described below with reference to Fig. 5 to Fig. 8. Fig. 5 is a diagram showing a configuration example of a photoelectric conversion element 400 in an RGB Bayer array according to Example 1.

[0040] In the following, a photoelectric conversion device having a so-called stacked structure in which the photoelectric conversion element 400 is configured by stacking and electrically connecting two substrates, the sensor substrate 11 and the circuit substrate 21, will be described as an example. Note that instead of a photoelectric conversion device having a so-called stacked structure, a photoelectric conversion device having a so-called non-stacked structure in which the components included in the sensor substrate 11 and the components included in the circuit substrate 21 are arranged on a common semiconductor layer may be used. Also, as described above, the photoelectric conversion element 400 is included in the imaging element 102 that also functions as a photoelectric conversion device in this embodiment.

[0041] The sensor substrate 11 includes a pixel region 12 (where the pixel region 12 is disposed). The circuit substrate 21 includes a circuit region 22 that processes signals detected in the pixel region 12 (where the circuit region 22 is disposed).

[0042] Fig. 6 is a diagram showing a configuration example of the sensor substrate 11. The pixel region 12 of the sensor substrate 11 included in the photoelectric conversion element 400 includes a plurality of pixels 401 arranged two-dimensionally across a plurality of rows and columns. The pixels 401 include a photoelectric conversion unit 402 including an avalanche photodiode (hereinafter, APD). Note that the number of rows and columns of the pixel array constituting the pixel region 12 shown in Fig. 6 is an example, and is not limited to the numbers shown in Fig. 6.

[0043] 7 is a diagram showing an example of the configuration of the circuit board 21. The circuit board 21 included in the photoelectric conversion element 400 is configured to have a signal processing circuit 403, a vertical scanning circuit 410, a horizontal scanning circuit 411, a readout circuit 412, a signal line 413, an output circuit 414, and a control pulse generating unit 415. The photoelectric conversion element 400 and the signal processing circuit 403 are electrically connected via connection wiring provided for each pixel.

[0044] The signal processing circuit 403 processes the charges photoelectrically converted by the photoelectric conversion unit 402 shown in FIG.

[0045] The vertical scanning circuit 410 receives a control pulse supplied from the control pulse generating unit 415 and supplies a control pulse to each pixel. The vertical scanning circuit 410 is composed of a shift register and an address decoder in which multiple rows are connected as one unit, and realizes high-speed readout by reading multiple rows at once. In particular, in the case of a photoelectric conversion device that digitally counts the number of photons arriving at the APD 501 and outputs the counted value from the pixel as a photoelectrically converted digital signal, it takes time for the counter circuit that digitally counts the number of photons to operate. Therefore, it is preferable to read multiple rows simultaneously for high-speed readout. That is, the vertical scanning circuit 410 that functions as a readout circuit that reads pixel signals from the pixels 401 simultaneously reads pixel signals from the pixels 401 included in multiple rows in the pixel area 12.

[0046] The control pulse generating unit 415 has a signal generating unit 515 that generates a control signal CLK that controls the on / off of the switch 502 described below. That is, the signal generating unit 515 generates a pulse signal as a control signal for controlling the switching of the switch 502. The signal generating unit 515 is configured to be able to change at least one of the period, the number of pulses, and the pulse width of the pulse signal that controls the switch 502. The control pulse generating unit 415 preferably has, for example, a frequency dividing circuit. This enables simple control and makes it possible to suppress an increase in the number of elements.

[0047] The signal output from the photoelectric conversion unit 402 of the pixel is processed by the signal processing circuit 403. The signal processing circuit 403 is provided with a counter, a memory, etc., and the memory holds digital values.

[0048] In order to read out the digital signals from the memories of the pixels, the horizontal scanning circuit 411 inputs control pulses for sequentially selecting each column to the signal processing circuit 403 via the readout circuit 412. For the selected column, a signal is output from the signal processing circuit 403 of the pixel selected by the vertical scanning circuit 410 to a signal line 413. The signal output to the signal line 413 is output to the outside of the photoelectric conversion device via an output circuit 414.

[0049] 6 and 7, a plurality of signal processing circuits 403 are arranged in a region overlapping with the pixel region 12 in a planar view. Note that a planar view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer in which the photoelectric conversion element 400 is arranged. Note that, when the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the planar view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0050] The vertical scanning circuit 410, the horizontal scanning circuit 411, the readout circuit 412, the output circuit 414, and the control pulse generating unit 415 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. The vertical scanning circuit 410, the horizontal scanning circuit 411, the readout circuit 412, the output circuit 414, and the control pulse generating unit 415 are arranged in a region overlapping the non-pixel region in a planar view.

[0051] The arrangement of the signal lines 413, the readout circuits 412, and the output circuits 414 are not limited to the arrangement (arrangement) shown in Fig. 7. For example, the signal lines 413 may be arranged extending in the row direction, and the readout circuits 412 may be arranged at the ends of the signal lines 413. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion unit, and one signal processing unit may be shared by multiple photoelectric conversion units to perform signal processing sequentially.

[0052] FIG. 8 is a diagram illustrating an equivalent circuit of the pixel 401 and the signal processing circuit 403 corresponding to the pixel 401 in FIGS.

[0053] The APD501 generates a pair of electric charges according to the incident light by photoelectric conversion. One of the two nodes of the APD501 is connected to a power supply line to which a driving voltage VL (first voltage) is supplied. The other of the two nodes of the APD501 is connected to a power supply line to which a driving voltage VH (second voltage) higher than the voltage VL is supplied. In FIG. 8, one node of the APD501 is an anode, and the other node of the APD501 is a cathode. A reverse bias voltage is supplied to the anode and cathode of the APD501 such that the APD501 performs an avalanche multiplication operation. By supplying such a voltage, the electric charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0054] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the voltage difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the voltage difference between the anode and cathode is close to or less than the breakdown voltage. An APD that operates in Geiger mode is called a SPAD. In the case of a SPAD, for example, the voltage VL (first voltage) is -30V, and the voltage VH (second voltage) is 1V.

[0055] The switch 502 is connected to a power supply line to which the driving voltage VH is supplied (applied) and one of the anode and cathode nodes of the APD 501. The switch 502 switches the resistance between the APD 501 and the power supply line to which the driving voltage VH is supplied. That is, the switch 502 switches the resistance between one of the anode and cathode nodes of the APD 501 and the power supply line to which the driving voltage VH is supplied (applied). Here, switching the resistance value means preferably changing the resistance value by 10 times or more, and more preferably changing the resistance value by 100 times or more. Hereinafter, a decrease in the resistance value is also referred to as the switch 502 being on, and an increase in the resistance value is also referred to as the switch 502 being off. The switch 502 functions as a quenching element.

[0056] The switch 502 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, suppressing the voltage supplied to the APD 501 and suppressing avalanche multiplication (quench operation).The switch 502 also functions to return the voltage supplied to the APD 501 to the drive voltage VH by passing a current equivalent to the voltage drop caused by the quench operation (recharge operation).

[0057] The switch 502 can be composed of a MOS transistor. In the example shown in Fig. 8, the switch 502 is a PMOS transistor. A control signal CLK for the switch 502, which is supplied from a signal generating unit 515, is applied to the gate electrode of the MOS transistor constituting the switch 502. In this embodiment, the on / off state of the switch 502 is controlled by controlling the voltage applied to the gate electrode of the switch 502.

[0058] The signal processing circuit 403 is configured to include a waveform shaping section 510, a counter circuit 511, and a selection circuit 512. In the example shown in Fig. 8, the signal processing circuit 403 includes the waveform shaping section 510, the counter circuit 511, and the selection circuit 512. However, it is sufficient for the signal processing circuit 403 to include at least one of the waveform shaping section 510, the counter circuit 511, and the selection circuit 512.

[0059] The waveform shaping unit 510 shapes the voltage change of the cathode of the APD 501 obtained when a photon is detected, and outputs a pulse signal. The node on the input side of the waveform shaping unit 510 is nodeA, and the node on the output side is nodeB. The waveform shaping unit 510 changes the output voltage from the node nodeB depending on whether the input voltage to the node nodeA is equal to or higher than a predetermined value or lower than a predetermined value. For example, in the example shown in FIG. 9, when the input voltage to the node nodeA is equal to or higher than the judgment threshold, the output voltage from the node nodeB becomes low level. Then, when the input voltage to the node nodeA is lower than the judgment threshold, the output voltage from the node nodeB becomes high level. For example, an inverter circuit is used as the waveform shaping unit 510. In the example shown in FIG. 8, an example in which one inverter is used as the waveform shaping unit 510 is shown, but a circuit in which multiple inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.

[0060] The counter circuit 511 counts the pulse signal output from the waveform shaping unit 510 and holds the count value. When a control pulse RES is supplied via a drive line 513, the signal held in the counter circuit 511 is reset.

[0061] A control pulse SEL is supplied to the selection circuit 512 from the vertical scanning circuit 410 shown in FIG. 7 via a drive line 514 (not shown in FIG. 7) shown in FIG. 8, and switches between electrical connection and non-connection between the counter circuit 511 and the signal line 413. The selection circuit 512 may include, for example, a buffer circuit for outputting a signal. An output signal OUT shown in FIG. 8 is an output signal from a pixel. Note that a switch such as a transistor may be disposed between the switch 502 and the APD 501, or between the photoelectric conversion unit 402 and the signal processing circuit 403 to switch the electrical connection. Similarly, the supply of the voltage VH or the voltage VL supplied to the photoelectric conversion unit 402 may be electrically switched using a switch such as a transistor.

[0062] It is possible to perform a quench operation and a recharge operation using the switch 502 in response to avalanche multiplication in the APD 501, but depending on the timing of photon detection, this may not be determined as an output signal. For example, assume that avalanche multiplication occurs in the APD 501, the input voltage to the node A becomes low level, and a recharge operation is being performed.

[0063] In general, the decision threshold of the waveform shaping unit 510 is set to a voltage higher than the voltage difference at which avalanche multiplication occurs in the APD 501. When a photon is incident when the voltage of the node A is lower than the decision threshold due to the recharge operation and the voltage is such that avalanche multiplication is possible in the APD 501, avalanche multiplication occurs in the APD 501 and the voltage of the node A drops. In other words, since the voltage of the node A drops at a voltage lower than the decision threshold, the output voltage from the node B does not change even though a photon is detected. Therefore, even though avalanche multiplication occurs, it is not judged as a signal. In particular, under high illuminance, photons enter continuously in a short period of time, making it difficult to judge as a signal. As a result, even though the illuminance is high, the actual number of incident photons and the output signal tend to diverge.

[0064] On the other hand, by applying a control signal CLK to the switch 502 to switch the switch 502 on and off, it is possible to determine a signal even when photons continuously enter the APD 501 in a short period of time. In Fig. 9, an example will be described in which the control signal CLK is a pulse signal with a repetitive cycle.

[0065] FIG. 9 is a diagram showing a schematic diagram of the relationship between the control signal CLK of the switch, the voltage of the node nodeA, the voltage of the node nodeB, and the output signal. In addition, t in FIG. 9 indicates time. In the photoelectric conversion device in this embodiment, when the control signal CLK is at a high level, the driving voltage VH is difficult to be supplied to the APD 501, and when the control signal CLK is at a low level, the driving voltage VH is supplied to the APD 501. The high level of the control signal CLK is, for example, 1V, and the low level of the control signal CLK is, for example, 0V. When the control signal CLK is at a high level, the switch is turned off, and when the control signal CLK is at a low level, the switch is turned on. In addition, the resistance value of the switch when the control signal CLK is at a high level is higher than the resistance value of the switch when the control signal CLK is at a low level.

[0066] When the control signal CLK is at a high level, a recharge operation is unlikely to occur even if avalanche multiplication occurs in the APD 501, and therefore the voltage supplied to the APD 501 becomes a voltage equal to or lower than the breakdown voltage of the APD 501. Therefore, the avalanche multiplication operation in the APD 501 stops.

[0067] At time t1, the control signal CLK changes from high level to low level, the switch is turned on, and the recharge operation of the APD 501 is started. This causes the voltage of the cathode of the APD 501 to transition to high level. Then, the voltage difference between the voltages applied to the anode and cathode of the APD 501 becomes a state in which avalanche multiplication is possible. The voltage of the cathode is the same as that of the node nodeA. Therefore, when the voltage of the cathode transitions from low level to high level, the voltage of the node nodeA becomes equal to or higher than the judgment threshold value at time t2. At this time, the pulse signal output from the node nodeB is inverted and goes from high level to low level. When the recharge is completed, the voltage difference of the drive voltage VH-drive voltage VL is applied to the APD 501. After that, the control signal CLK becomes high level, and the switch is turned off.

[0068] Next, at time t3, when a photon is incident on the APD 501, avalanche multiplication occurs in the APD 501, an avalanche multiplication current flows through the switch 502, and the voltage at the cathode drops. That is, the voltage at the node nodeA drops. When the voltage at the node A drops below the decision threshold, the voltage at the node nodeB goes from low to high. That is, the part of the output waveform at the node A that exceeds the decision threshold is shaped by the waveform shaping unit 510 and output as a signal at the node B. Then, the counter circuit 511 counts, and the count value of the counter signal output from the counter circuit 511 increases by 1 LSB.

[0069] 9, photons are incident on the APD 501 between time t3 and time t4, but the switch is off at this time, and the voltage applied to the APD 501 does not have a voltage difference that allows avalanche multiplication. Therefore, in this state, the voltage level of the node A does not exceed the decision threshold.

[0070] At time t4, the control signal CLK changes from high to low, turning the switch on. As a result, a current flows through node A to compensate for the voltage drop from the drive voltage VL, and the voltage of node A transitions to its original voltage level. At this time, the voltage of node A becomes equal to or higher than the decision threshold at time t5, so the pulse signal of node B is inverted and goes from high to low.

[0071] At time t6, the node A is stabilized to the original voltage level, and the control signal CLK goes from low to high. After this, the voltages of the nodes and signal lines change in response to the control signal CLK and the incidence of photons, as described from time t1 to time t6.

[0072] In this way, it is possible to control the recharge frequency of the APD 501 by applying the control signal CLK to the switch 502 to switch the switch 502 on and off. As described below, by controlling the timing of the control signal CLK to the switch, image quality in a dark environment (when shooting in low illuminance) is improved.

[0073] First, crosstalk between pixels will be described with reference to FIG. 10. FIG. 10 is a diagram for explaining crosstalk between pixels according to the first embodiment. FIG. 10(A) is a diagram showing an output distribution of 5×5 pixels in a pixel region. FIG. 10(B) is a diagram showing an example of relative output values ​​on the X-axis of FIG. 10(A). Pixel 401a indicates a defective pixel (a damaged pixel). Pixels (adjacent pixels) 401b and 401c are pixels arranged around pixel 401a. Specifically, pixel 401b is a pixel adjacent to pixel 401a and pixel 401c. Pixel 401c is a pixel adjacent to pixel 401b.

[0074] As shown in FIG. 10(A) and FIG. 10(B), if there is a defective pixel in the pixel region, the output levels of the pixels 401b and 401c arranged around the pixel 401a rise regardless of the illuminance of the incident light. This is thought to be because avalanche emission in the pixel 401a generates electric charges in the adjacent pixels, and the electric charges cause avalanche multiplication in the adjacent pixels. In other words, a defective pixel 401a makes it easier for avalanche emission to occur, which causes clustered defects, increases the relative output of the pixels around the pixel 401a, and reduces image quality. Clustered defects stand out more in an image than single defects, leading to a significant reduction in image quality.

[0075] A conventional photoelectric conversion device in which the timing of the control signal CLK to the switch 502 is always constant will be described below as a comparative example with reference to Figures 11, 12, and 13. Then, a photoelectric conversion device in this embodiment in which the timing of the control signal CLK is controlled will be described with reference to Figures 14, 15, and 16.

[0076] Fig. 11 is a diagram showing a timing chart of the photoelectric conversion element 400 in the shooting mode under high illuminance and low illuminance conditions of the conventional photoelectric conversion device. Fig. 11(A) is a diagram showing a timing chart of the control signal and the avalanche light emission in the pixel 401a during one frame period during high illuminance shooting (in a bright environment) of the conventional photoelectric conversion device. Fig. 11(B) is a diagram showing a timing chart of the control signal and the avalanche light emission in the pixel 401a during one frame period during low illuminance shooting (in a dark environment) of the conventional photoelectric conversion device.

[0077] In a conventional photoelectric conversion device, a constant control signal CLK is supplied to the switch regardless of whether shooting is performed under high illuminance or low illuminance.

[0078] The one frame period refers to, for example, a period from the rising edge of the pulse signal VD to the rising edge of the next pulse signal VD. In this embodiment, the one frame period is, for example, a period during which the vertical scanning circuit 410 scans the first to last row pixels arranged in the pixel region. In other words, the period from when the pulse signal VD, which is a vertical synchronization signal, becomes high level once until it becomes high level again is the one frame period. Here, it is not necessary to scan all the pixels in the first to last rows in the one frame period. For example, when scanning some of all the rows while thinning out, the period from a certain row to scanning in one direction until the last row is finished is set as the one frame period. Also, when scanning a certain row while thinning out and then scanning the thinned out rows, the one frame period may be set as the period until the scanning of the thinned out rows is finished.

[0079] During each frame period, it is preferable to reset the count value of the counter circuit 511 of the signal processing circuit 403. The count value may be reset in common for all pixels, or may be reset sequentially for each pixel row.

[0080] In this embodiment, the exposure period refers to a period during which light can be incident on the APD 501 and the APD 501 and the signal processing circuit 403 can read out signals. The non-exposure period refers to a period during which the APD 501 in the pixel region 12 is shielded from light and no light is incident on it.

[0081] Here, the state in which light can be incident refers to a state in which light is not blocked by a mechanical or electrical shutter or the like. Moreover, the period in which the APD 501 and the signal processing circuit 403 are in a state in which a signal can be read out refers to a period in which the APD 501 and the signal processing circuit 403 are not intentionally turned off. Note that in this embodiment and each of the following embodiments, the period of the quench operation in the APD 501 does not fall under this, and the quench operation period is a period in which a signal can be read out. Note that the exposure period and the non-exposure period are not limited to the opening and closing of the shutter, and may be defined by adjusting the bias applied to the APD 501 to change whether or not a photon signal can be obtained.

[0082] The control signal CLK is a signal that controls the on / off of the switch 502 described with reference to FIG. 8 and FIG.

[0083] Dark events shown in FIG. 11(A) and FIG. 11(B) are diagrams showing the timing of photon generation due to avalanche emission in pixel 401a, which is the defective pixel in FIG. 10. In dark events, the rising timing is the timing of photon generation. Note that both the solid line and the dashed line shown in dark events are the timing of photon generation, but the solid line is the timing of photon generation counted as a signal, and the dashed line is the timing of photon generation not counted as a signal. This is because, as described above, the voltage of node nodeA drops before the voltage of node nodeA becomes equal to or higher than the determination threshold, and is not determined as a signal. Note that FIG. 11 does not show avalanche emission caused by light from a subject being incident on the APD 501, but only shows avalanche emission caused by dark events, which is generated by the trap level of pixel 401a, etc.

[0084] Dark counts indicate the counting operation of the counter circuit 511 included in the pixel 401a. When the voltage of the control signal CLK is at a low level, the APD 501 recharges. Therefore, after the recharge operation, when a photon is avalanche-multiplied in the APD 501, it is determined as a signal, and the count number of the counter circuit 511 is increased by one.

[0085] Crosstalk events shown in Figures 11(A) and 11(B) are diagrams showing timings at which crosstalk occurs in pixels surrounding pixel 401a. Since the timing at which crosstalk occurs is random, Figures 11(A) and 11(B) show only one example.

[0086] Fig. 12 is a diagram showing the output count number of the photoelectric conversion element 400 in the shooting mode under high illuminance condition and low illuminance condition of the conventional photoelectric conversion device. Fig. 12(A) shows the relationship between the number of incident photons and the output count number per pixel in the shooting mode under high illuminance condition of the conventional photoelectric conversion device shown in Fig. 11(A). Fig. 12(B) shows the relationship between the number of incident photons and the output count number per pixel in the shooting mode under low illuminance condition of the conventional photoelectric conversion device shown in Fig. 11(B).

[0087] As shown in Fig. 12(A), when shooting in high illuminance, the output level of the defect (flaw output level) is low relative to the signal level. On the other hand, as shown in Fig. 12(B), when shooting in low illuminance, the output level of the defect may be high relative to the signal level.

[0088] Fig. 13 is a diagram showing the influence of inter-pixel crosstalk of the photoelectric conversion element 400 in the photographing mode under high and low illumination conditions of the photoelectric conversion device according to the first embodiment. Fig. 13(A) is a diagram showing the output distribution of the pixel 401a and its surrounding 5 x 5 pixels during high illumination photographing. Fig. 13(B) is a diagram showing an example of the relative output value on the X axis of Fig. 13(A). Fig. 13(C) is a diagram showing the output distribution of the pixel 401a and its surrounding 5 x 5 pixels during low illumination photographing, and Fig. 13(D) is a diagram showing an example of the relative output value on the X axis of Fig. 13(C).

[0089] 13B and 13D show the signal levels at each shooting. During shooting at high illuminance, the relative output value of the signal level is higher than the relative output value of pixel 401a, so that the abnormal output of pixel 401a due to defects or the like is not easily noticeable. On the other hand, during shooting at low illuminance, the relative output value of the signal level is lower than the relative output value of pixel 401a, so that the abnormal output of pixel 401a is noticeable. Therefore, during shooting at low illuminance, degradation of image quality is likely to occur due to signals generated due to crosstalk.

[0090] Fig. 14 is a timing chart of the photoelectric conversion element 400 in the imaging mode under high illumination and low illumination conditions according to the embodiment 1. Fig. 14(A) is a timing chart of the control signal and the avalanche light emission in the pixel 401a during one frame period during high illumination imaging of the photoelectric conversion device of the embodiment 1. Fig. 14(B) is a timing chart of the control signal and the avalanche light emission in the pixel 401a during one frame period during low illumination imaging of the photoelectric conversion device of the embodiment 1.

[0091] As shown in FIG. 14B, in the first embodiment, the number of pulse signals of the control signal CLK in one frame period during low illuminance shooting is set to be smaller than the number of pulse signals of the control signal CLK in one frame period during high illuminance shooting. In this way, in the first embodiment, the control unit 104 controls the signal generating unit 515 to make the number of pulses of the pulse signal generated by the signal generating unit 515 different during high illuminance shooting and during low illuminance shooting during the period during which one frame's worth of signals is acquired. That is, the number of pulses of the pulse signal is changed during the period during which one frame's worth of signals is acquired according to the illuminance of the signal acquired by the photoelectric conversion element 400. With this configuration, it becomes difficult to detect an abnormal output of the pixel 401a, which is a defective pixel, during low illuminance shooting, and deterioration of image quality can be suppressed compared to the conventional photoelectric conversion device.

[0092] For example, the lower the illuminance of the environment where the subject or the like exists, the smaller the number of pulse signals of the control signal CLK in one frame period is preferable. This is because, when the number of pulse signals (the number of pulses of the pulse signal) is equivalent to that of an environment with high illuminance, the lower the illuminance of the environment where the subject or the like exists, the more likely it is that degradation of image quality due to crosstalk will occur. Details of changing the number of pulse signals of the control signal CLK according to the illuminance of the environment will be described later.

[0093] Fig. 15 is a diagram showing the output count number of the photoelectric conversion element in the shooting mode under high illuminance conditions and low illuminance conditions according to Example 1. Fig. 15(A) shows the relationship between the number of incident photons and the output count number per pixel during high illuminance shooting of the photoelectric conversion device of Example 1 shown in Fig. 14(A). Fig. 15(B) shows the relationship between the number of incident photons and the output count number per pixel during low illuminance shooting of the photoelectric conversion device of Example 1 shown in Fig. 14(B).

[0094] 15(B), in the imaging device 1 including the photoelectric conversion device of this embodiment, the control unit 104 controls the output level of the defect to be lower than the signal level. Therefore, it is possible to reduce the signal readout caused by crosstalk from the pixel 401a, and it is possible to suppress the deterioration of image quality during low-illumination shooting compared to the conventional photoelectric conversion device.

[0095] FIG. 16 is a diagram showing the influence of inter-pixel crosstalk of the photoelectric conversion element 400 in the shooting modes under high and low illumination conditions according to the first embodiment. FIG. 16(A) is a diagram showing the output distribution of the pixel 401a and its surrounding 5×5 pixels in high illumination shooting. FIG. 16(B) is a diagram showing an example of the relative output value on the X axis of FIG. 16(A). FIG. 16(C) is a diagram showing the output distribution of the pixel 401a and its surrounding 5×5 pixels in low illumination shooting. FIG. 16(D) is a diagram showing an example of the relative output value on the X axis of FIG. 16(C).

[0096] Figures 16(B) and 16(D) show the signal levels during each shooting. As shown in Figures 16(C) and 16(D), according to the imaging device 1 including the photoelectric conversion device of this embodiment, the relative output value of the defective pixel pixel 401a can be made lower than that of the conventional photoelectric conversion device during low-illumination shooting. Therefore, during low-illumination shooting, the occurrence of cluster-shaped defects caused by crosstalk can be reduced, and degradation of image quality during low-illumination shooting can be suppressed.

[0097] In the following, in the first embodiment, the threshold for detecting defective pixels is changed according to the number of pulse signals of the control signal CLK in one frame period (exposure period for acquiring one frame's worth of signal), and defective pixel data corresponding to the number of pulse signals of the control signal CLK is created. An example of a method for correcting a signal acquired by the image sensor 102 based on the information of the created defective pixel data is shown below.

[0098] Hereinafter, the relationship between the number of pulse signals of the control signal CLK and the pixel level of a cluster defect will be described with reference to FIG. 17 and FIG. 18. FIG. 17 is a diagram showing the relationship between the number of pulse signals of the control signal CLK and the pixel level of a cluster defect according to the first embodiment. FIG. 17(A) is a diagram showing an example of the pixel level of a cluster defect when the number of pulse signals is N1. FIG. 17(B) shows the pixel level of a cluster defect when the number of pulse signals is N2. However, the condition is that the number of pulse signals N1 is greater than the number of pulse signals N2. Also, it is assumed that a gain is applied so that the average pixel output value of the image is equivalent for the number of pulse signals N1 and the number of pulse signals N2. Therefore, the pixel level of pixel 401a arranged at the center of the center of the cluster defect is equivalent for the number of pulse signals N1 and the number of pulse signals N2.

[0099] 18 is a diagram showing the relationship between the number of pulse signals of the control signal CLK and the level of the adjacent pixel 401b relative to the pixel 401a, which is the central pixel of the cluster-shaped defect. The relationship between the level of the adjacent pixel 401b relative to the pixel 401a, which is the central pixel of the cluster-shaped defect, is calculated by dividing the level of the adjacent pixel 401b of the cluster-shaped defect by the proportion of the pixel 401a.

[0100] 17(A) and 17(B), if the levels of adjacent pixels 401b of a cluster-shaped defect at pulse signal counts N1 and N2 are L1 and L2, respectively, L2 is smaller than L1. This indicates that the level of adjacent pixels 401b of a cluster-shaped defect is smaller when the number of pulse signals is smaller. In order to detect cluster-shaped defects at the same time as setting the number of pulse signals, it is necessary to change the threshold value.

[0101] 18, the level of the adjacent pixel 401b relative to pixel 401a, which is the central pixel in the cluster defect, gradually decreases as the number of pulse signals is reduced, and the amount of decrease is particularly large when the number of pulse signals is small. On the other hand, if only the gain is increased without changing the number of pulse signals of the control signal CLK, the value of the defective pixel increases with the gain, and in this case in this embodiment, the threshold value for detecting defective pixels is increased.

[0102] Fig. 19 is a diagram showing an example of data of a defective pixel in this embodiment. Fig. 19(A) is a diagram showing an image of a defective pixel in an image. Fig. 19(B) is a diagram showing an example of data of the defective pixel shown in Fig. 19(A).

[0103] 19A is an image of defective pixels in an image, and the pixels surrounded by black lines in the figure are examples of defective pixels. In the figure, (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5), (X6, Y6), (X7, Y7), and (X8, Y8) indicate the addresses of the defective pixels.

[0104] Defective pixel data consists of four elements: the amount of defect (the level of the defective pixel), the address of the defective pixel in the image, the type of defective pixel, and the control setting. The amount of defect may be the pixel level, or it may be the level difference between the pixel of interest and the median or average value of the surrounding pixels.

[0105] The address of a defective pixel within an image is coordinate information (position information) that indicates the position of the defective pixel in the image. It may be a two-dimensional address consisting of a horizontal address and a vertical address, or it may be a one-dimensional address that is the relative value of the number of pixels counted from the top left of the image.

[0106] The defective pixel type is the type of defective pixel, and stores information on the characteristics of the defect. For example, it may be a cluster defect that spreads to surrounding pixels or a single defect. In this embodiment, the cluster defect to be corrected can be confirmed under conditions where no light is incident, so it is preferable to obtain the pixel level in a light-shielded state using a mechanical or electrical shutter, but it is also possible to obtain the pixel level using a light source with uniform brightness. The control setting indicates the exposure conditions, the number of pulse signals of the control signal CLK, the magnitude of the gain, etc., and is set by the control unit 104.

[0107] Fig. 19(B) is a diagram showing an example of defective pixel data for a defective pixel in the image shown in Fig. 19(A). Fig. 19(B) shows control setting data such as the defect amount, address, defect type, and number of pulses. For example, the defect amount of an address (X1, Y1), which is the pixel's position information (coordinate information), is Z1, the defect type is a cluster defect K1, and the number of pulse signals N1 of the control signal CLK at that time, the gain magnitude, and other control settings M1 (not shown in Fig. 19) are stored.

[0108] However, the types of defects are not limited to those mentioned above. For example, there may be a cluster-shaped defect, but the amount of defect in adjacent pixels is small, and the defective pixel detection unit 108 cannot accurately detect it. In that case, it is detected as a single defect, and processing is performed by the defective pixel correction unit 109 as a single defect. Furthermore, the defective pixel data is stored in the RAM 105 and the ROM 106 in a predetermined format. That is, the RAM 105 and the ROM 106 function as a storage unit that stores position information of pixels (defective pixels) to be corrected that has been acquired in advance. Furthermore, the defective pixel data may be updated or added each time the defective pixel data is detected. Furthermore, if the defective pixel is known in advance from the sensor specifications, the defective pixel data may be added to it.

[0109] Hereinafter, a flow of detecting defective pixels, which are pixels to be corrected by the defective pixel detection unit 108, according to the gain and the number of pulse signals of the control signal CLK set by the control unit 104 will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the flow of detecting defective pixels according to the number of pulse signals of the control signal CLK according to the first embodiment. Note that, hereinafter, it is assumed that a table having defective pixel threshold data, which is information on the multiple control settings and thresholds shown in Fig. 3, is already stored before starting this process.

[0110] As described above, in low-illuminance shooting, the output level of the defective pixel can be made lower than the signal level by reducing the number of pulse signals of the control signal CLK in one frame period. Since shooting is performed at low illuminance, the output from the image sensor 102 is naturally small. It is desirable that the average value of the output on a monitor or the like is always constant, whether the device is used as an image capture device or whether shooting is performed at high illuminance or low illuminance.

[0111] For this reason, it is desirable to apply a gain in the signal processing unit 103 during low-illuminance shooting to keep the average value of the output on the monitor or the like approximately constant. In this embodiment, when applying a gain during low-illuminance shooting, the number of pulse signals of the control signal CLK is changed simultaneously or in conjunction with the process of multiplying the gain. The signal processing unit 103 changes the value by which the gain is multiplied depending on the illuminance of the subject.

[0112] For example, when the illuminance of the subject becomes 1 / 4 (from high illuminance to low illuminance), signal processing unit 103 multiplies the gain by 4 to maintain brightness. Then, simultaneously or in conjunction with this, the number of pulse signals of the control signal CLK in one frame period is multiplied by 1 / 4. That is, control unit 104 controls signal generation unit 515 to multiply the number of pulses generated by signal generation unit 515 in one frame period by 1 / 4. For example, when photographing in high illuminance, if the number of pulse signals of the control signal CLK in one frame period is 2048, control unit 104 controls the number of pulse signals of the control signal CLK to be reduced to 512 in one frame period.

[0113] In this manner, in this embodiment, when the signal processing unit 103 applies a gain in accordance with the illuminance based on at least the signal acquired by the photoelectric conversion element 400, the signal generating unit 515 changes the number of pulse signals to be generated in one frame period simultaneously or in conjunction with this. That is, when shooting in low illuminance is performed after shooting in high illuminance, the number of pulse signals to be generated in one frame period is changed to a smaller number than that during shooting in high illuminance.

[0114] The imaging device 1 of this embodiment is configured to have a mode according to the number of pulse signals described above. That is, it has a first mode, which is a mode in which the number of pulse signals generated in one frame period is not changed during high-illuminance shooting, and a second mode, which is a mode in which the number of pulse signals generated in one frame period is changed to be less than that in the first mode during low-illuminance shooting. The first mode and the second mode are controlled by the control unit 104 so as to be switched according to at least the illuminance based on the signal acquired by the photoelectric conversion element 400 as described above. Also, as described above, even when a gain is applied, the number of pulse signals generated in one frame period is changed, so that mode switching can be performed. Note that, when switching between the first mode and the second mode, the signal generation unit 515 functions as a switching unit. Note that, the control unit 104 may function as a switching unit in order to control the signal generation unit 515.

[0115] When changing the number of pulse signals in one frame period, the control unit 104 may select the number of pulse signals from among the settable numbers of pulse signals according to at least the illuminance based on the signal acquired by the photoelectric conversion element 400. Then, the control unit 104 controls the number of pulse signals to be changed to the selected number of pulse signals. Also, as described above, in the first embodiment, the threshold for detecting defective pixels is changed according to the number of pulse signals of the control signal CLK in one frame period. That is, the threshold selection unit 203 selects a threshold according to the number of pulse signals of the control signal CLK in one frame period from a table having defective pixel threshold data as shown in FIG. 3, and sets the threshold.

[0116] In this embodiment, when the number of pulse signals of the control signal CLK is reduced, it is desirable that the gain is automatically increased simultaneously or in conjunction with the reduction, but the gain may be manually set and increased. As described above, when only the gain is increased without changing the number of pulse signals of the control signal CLK, the value of a defective pixel increases with the gain, so the threshold for detecting defective pixels must be increased. Therefore, when the number of pulse signals of the control signal CLK during one frame period is not changed, that is, when the gain is applied in the same mode, the threshold for detecting defective pixels is increased. For example, the threshold selection unit 203 selects a threshold value larger than that of the current mode from the defective pixel threshold data simultaneously or in conjunction with the timing of applying the gain in the same mode, and sets the threshold value.

[0117] 20, a flow of detecting defective pixels by the defective pixel detection unit 108 will be described below. Each process shown below is realized by the control unit 104 of the imaging device 1 executing a program stored in the ROM 106 or RAM 105. Each process (step) is represented by adding an S to the beginning to omit the representation of the process (step).

[0118] First, in S11, the control unit 104 sets the number of pulse signals of the control signal CLK in one frame period. For example, it is set to the number N1 of pulse signals of the control signal CLK that can be set. At this time, the control unit 104 changes the gain according to the number of pulse signals of the control signal CLK. Note that the gain does not have to be applied at this time.

[0119] Next, in S12, the threshold selection unit 203 of the defective pixel detection unit 108 selects a threshold for detecting defective pixels with respect to the number of pulse signals of the control signal CLK set in S11. When selecting the threshold, a threshold corresponding to the number of pulse signals N1 of the control signal CLK is selected from among a plurality of thresholds in the defective pixel threshold data. The plurality of thresholds in the defective pixel threshold data have different threshold values ​​according to the pulse signals of the control signal CLK. Specifically, the smaller the number of pulse signals of the control signal CLK generated in one frame period, the smaller the threshold value is set. In this way, the threshold value for determining whether or not the signal processing unit 103 performs correction differs between the first mode and the second mode in which the number of pulses of the pulse signal in the exposure period in which one frame's worth of signals is acquired is smaller than that in the first mode.

[0120] The relationship of decreasing the threshold value in accordance with the decrease in the number of pulse signals of the control signal CLK is as shown in Figure 18. On the other hand, if the level of a defective pixel falls below the threshold value when the number of pulse signals of the control signal CLK in one frame period is decreased, it is determined that correction is not necessary. In other words, it is determined that there is no defect in the image, and pixel correction is not performed.

[0121] Next, in S13, the defective pixel detection unit 108 detects defective pixels based on the threshold selected in S12. When detecting defective pixels, an image is captured under specific conditions, and the amount of defects is calculated from the difference between the pixel level predicted from the conditions and the output from the image sensor 102. More specifically, when a video signal is acquired under conditions where no light enters the imaging device 1, the pixel level output from the image sensor 102 is considered to be zero. On the other hand, defective pixels have a high pixel level. This pixel level is compared with the threshold selected in S12 to detect defective pixels.

[0122] Next, in S14, the defective pixel detection unit 108 stores the defect amount, defect position, defect type, and control settings such as the pulse signal setting and gain setting of the control signal CLK at that time of the defective pixel detected in S13 as pixel defect data in the RAM 105 and ROM 106.

[0123] It is desirable to detect defective pixels by performing the process described above in Fig. 20 for all possible numbers of pulses of the control signal CLK. However, it is also possible to perform defective pixel detection for multiple numbers of pulses, as described below, and predict the value of defective pixel data for other numbers of pulses from the results.

[0124] The defect information may be generated every time the imaging device 1 is started, or may be generated only once during adjustment during the manufacturing process of the imaging device 1. Alternatively, the defect information may be generated by a user operation during shooting. If the defect information is generated during adjustment, the data generated in RAM 105 is stored in ROM 106, and from the next time onwards, the generated defect information can be used by defective pixel correction unit 109 simply by loading the data in ROM 106 into RAM 105 at startup.

[0125] Hereinafter, a flow of defective pixel correction according to the number of pulse signals of the control signal CLK in one frame period will be described with reference to Fig. 21. Fig. 21 is a diagram showing a flow of defective pixel correction according to the number of pulse signals of the control signal CLK according to the first embodiment. During the process in Fig. 21, the photoelectric conversion element 400 performs correction processing on a signal acquired based on position information of a pixel to be corrected, using the defective pixel correction unit 109 described in Fig. 4.

[0126] First, when the imaging device 1 starts shooting, in S21, the control unit 104 acquires information on the pulse setting of the control signal CLK in one frame period. As described above, the number of pulse signals of the control signal CLK in one frame period in this embodiment is determined based on each driving mode (first mode, second mode). The number of pulse signals of the control signal CLK in one frame period may be set manually based on an external instruction received through an external I / F or the like. Alternatively, the control unit 104 may automatically set the number of pulse signals based on information analyzed by the signal processing unit 103 based on the output of the imaging element 102. In the following description, for example, it is assumed that the number of pulse signals (number of clocks) of the control signal CLK in one frame period is N1.

[0127] Next, in S22, the defect information selection unit 302 of the defective pixel correction unit 109 selects the defective pixel data for the set clock number N1 from the defective pixel data detected and held in S14, and inputs it to the correction unit 301.

[0128] Next, in S23, correction unit 301 of defective pixel correction unit 109 corrects the input defective pixel. That is, defective pixel correction unit 109 corrects the signal acquired by photoelectric conversion element 400 based on the position information of the defective pixel.

[0129] When correcting a defective pixel input by the defect information selection unit 302, the position of the defective pixel to be corrected is identified based on the address input by the address calculation unit 202 and the address data included in the defective pixel data as described above, and the pixel is extracted. The correction unit 301 corrects the extracted pixel to be corrected using pixels arranged around the pixel to be corrected. For example, it is possible to calculate the median of the levels of a 5 x 5 pixel area around the pixel to be corrected and replace the target pixel with the result. The range for which the median is calculated is not limited to this, and an average value or the like may be used instead of the median.

[0130] Next, in S24, the video signal generator 110 generates a video signal of a predetermined format based on the pixel signal corrected in S23, i.e., the pixel signal in frame units, and outputs the video signal to each block. Note that any image processing may be performed on the video signal at this time.

[0131] As described above, according to the imaging device of the first embodiment, during low-illumination imaging, the occurrence of cluster-like defects caused by crosstalk can be reduced, and degradation of image quality can be suppressed.

[0132] <Example 2> In the second embodiment, defective pixels are corrected in a mode for all pulse signal numbers from defective pixel data acquired in a mode for a specific number of pulse signals. The method in the second embodiment will be described below. Note that the configuration and each functional unit of the imaging device 1 including the photoelectric conversion device in the second embodiment are the same as those in the first embodiment, and therefore will not be described below as appropriate.

[0133] As described in the first embodiment, in high-illuminance shooting, the amount of incident light is large and the signal level is also large, so that cluster-shaped defects are small compared to the signal level and are not noticeable. On the other hand, in low-illuminance shooting (dark-illuminance shooting), the amount of incident light is small, so that cluster-shaped defects are large compared to the signal level and are easily noticeable. Therefore, by reducing the number of pulse signals of the control signal CLK in one frame period, the level of the cluster-shaped defects is lowered and control is performed so that the cluster-shaped defects are not noticeable. At this time, control is also performed to switch from the first mode to the second mode.

[0134] When an image is taken at low illuminance, the signal level is small, so it is desirable to apply a gain to make the output level consistent with an image taken at high illuminance. However, applying a gain also raises the level of a single defective pixel, making the defect more noticeable. On the other hand, in a cluster defect, the number of pulse signals of the control signal CLK in one frame period is small. Therefore, as shown in Figures 17 and 18, the pixel level of pixel 401a, which is the central pixel in the cluster defect, increases with the gain, but the level of adjacent pixel 401b, which is an adjacent defective pixel, decreases.

[0135] That is, when the number of pulse signals of the control signal CLK in one frame period is changed from a high number in a high-illuminance environment (when shooting in high-illuminance) to a low number in a low-illuminance environment (when shooting in low-illuminance), the signal level of the defect changes. That is, when the first mode is changed to a second mode in which the number of pulse signals of the control signal CLK in one frame period is smaller than that of the first mode, the signal level of the single defect and the center of the cluster defect becomes high, and the signal level of the adjacent part of the cluster defect becomes low. It is necessary to detect the pixel to be corrected in response to such a change in the signal level of the defect and correct the pixel.

[0136] In the second embodiment, defective pixels are detected using a plurality of numbers of pulse signals of the control signal CLK in order to correct the varying defect levels. For example, defective pixels are detected using N1, which is the smallest number of pulse signals among the number of pulse signals of the control signal CLK in one frame period, and Nn, which is the largest number of pulse signals of the control signal CLK in one frame period, and defective pixel data is created. The detection method is the same as that shown in the first embodiment. In the second embodiment, the case where the number of pulse signals is N1 is the first mode, and the case where the number of pulse signals is Nm is the second mode.

[0137] FIG. 22 is a diagram illustrating defective pixel data that is the result of defective pixel detection. In the second embodiment, there are two pieces of defective pixel data, as shown in FIG. 22(A) and FIG. 22(B). FIG. 22(A) shows defective pixel data for pulse signal number N1. FIG. 22(B) shows defective pixel data for pulse signal number Nn. Note that the tables of defective pixel data shown in FIG. 22(A) and FIG. 22(B) are arranged in descending order of the amount of defect from the top of the table.

[0138] When the number of pulse signals is N1, the pixels to be corrected are those exceeding the threshold value TH1 in the defective pixel data in Fig. 22(A). Similarly, when the number of pulse signals is Nn, the pixels to be corrected are those exceeding the threshold value TH1 in the defective pixel data in Fig. 22(B).

[0139] For pixels to be corrected in a driving mode with a pulse signal number Nm (m is any natural number from 1 to n) between N1 and Nn, a table for the pulse signal number N1 and a table for the pulse signal number Nn are determined. In the second embodiment, a mode with a pulse signal number Nm where the pulse signal number N1 and the pulse signal number Nm are different signal numbers is set as a third mode. Thus, in the second embodiment, there are three driving modes with different numbers of pulse signals of the control signal CLK in one frame period. Note that these modes can be switched by the signal generating unit 515 in the same manner as in the first embodiment. In this switching process, the signal generating unit 515 functions as a switching unit. Note that the control unit 104 may function as a switching unit.

[0140] When the pulse signal number Nm is close to the pulse signal number N1, the defective pixel data at the pulse signal number N1 is used to determine pixels whose pixel level is equal to or greater than a threshold value TH1 as defective pixels, and when the pulse signal number Nm is close to the pulse signal number Nn, the defective pixel data at the pulse signal number Nn is used to determine pixels whose pixel level is equal to or greater than a threshold value TH2 as defective pixels.

[0141] Alternatively, thresholds TH1_m and THn_m are determined according to the difference in the number of pulses between pulse signal numbers N1, Nn, and Nm. For example, as the difference between pulse signal numbers N1 and Nm increases, threshold TH1_m for defective pixel data at pulse signal number N1 is set to gradually increase. Also, for example, as the difference between pulse signal numbers Nn and Nm increases, threshold THn_m for defective pixel data at pulse signal number Nn is set to gradually increase.

[0142] Then, using the defective pixel data for the pulse signal count N1 and the pulse signal count Nn, pixels that are equal to or greater than the threshold value TH1_m and the threshold value THn_m are treated as defective pixels and are subject to correction. For example, if the threshold value TH1 for the pulse signal count N1 is between Z5 and Z6, Z1 to Z5 are pixels that exceed the threshold value, and the pixels (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), and (X5, Y5) are subject to correction.

[0143] Similarly, for the number of pulse signals Nn, correction is performed for defects having defect amounts equal to or greater than the threshold value THn. If the threshold value THn is between W5 and W6, pixels W1 to W5 exceed the threshold value, and the pixels (X9, Y9), (X10, Y10), (X11, Y11), (X12, Y12), and (X13, Y13) become the correction targets. For the number of pulse signals Nm, if the correction target threshold value TH1_m is between Z2 and Z3, and THn_m is between W2 and W3, the pixels (X1, Y1), (X2, Y2), (X9, Y9), and (X10, Y10) corresponding to the defect amounts Z1, Z2, W1, and W2 become the correction targets.

[0144] When correcting a defective pixel, correction is performed using the median level of surrounding pixels, as in the first embodiment, based on defective pixel data at the number Nm of pulse signals of the control signal CLK during one frame period set by the control unit 104.

[0145] In the imaging device 1 of the second embodiment, the modes that can be switched based on the number of pulse signals of the control signal CLK for detecting defective pixel data are not limited to those described above, and more modes may be added. The more the number of modes increases, the more the detection accuracy can be improved. As described above, in the imaging device 1 of the second embodiment, in addition to the same effects as those of the first embodiment, image degradation due to insufficient or excessive correction can be prevented.

[0146] <Example 3> A method for detecting and correcting cluster-shaped defects will be described in Example 3. Note that the configuration and each functional unit of the image pickup device 1 including the photoelectric conversion device in Example 3 are the same as those in Example 1 except for the defective pixel calculation unit 601, so the following describes the points that are different from Example 1, and omits a description of the points that are the same as those in Example 1.

[0147] 17 and 18, the fewer the number of pulse signals of the control signal CLK in one frame period, the lower the level of the adjacent pixel 401b of the cluster defect. Therefore, when the number of pulse signals of the control signal CLK is changed, the level of the adjacent pixel 401b changes relative to the level of pixel 401a, which is the central pixel of the cluster defect. By applying this, adjacent defects can be detected.

[0148] For example, as described in the first embodiment, a method is used in which the number of pulse signals of the control signal CLK in one frame period is reduced and a gain is applied when capturing an image using the imaging device 1. By doing so, it is possible to keep the output of pixel 401a, which is the central pixel in the cluster defect, constant.

[0149] By comparing the pixel level of the adjacent pixel 401b at this time, it is possible to determine that the pixel whose pixel level has changed with a decrease in the number of pulse signals of the control signal CLK in one frame period is an adjacent pixel of a cluster-shaped defect, and therefore the pixel can be the target of correction. That is, when determining whether a pixel is the target of correction, the detection unit 201 of the defective pixel detection unit 108 can make the determination based on the change in pixel level of the pixels arranged around the pixel to be corrected when the mode is switched from the first mode to the second mode. Note that this may not be limited to mode switching, but may be accompanied by a decrease in the number of pulse signals as described above, or at the timing when the number of pulse signals is decreased.

[0150] Furthermore, since there is a high possibility that adjacent defects will occur around a pixel with a significantly large defect amount, the area around a pixel with a significantly large defect amount compared to the surrounding pixels may be stored in the defective pixel data as an adjacent defect.

[0151] If cluster-shaped defective pixels can be detected, it is possible to prevent insufficient correction of cluster-shaped defects. The adjacent pixels (pixels arranged around pixel 401a) adjacent to pixel 401a, which is the central pixel of the cluster-shaped defect, change their defect amount depending on the number of pulse signals of the control signal CLK. Therefore, if the defective pixel of adjacent pixel 401b is identified, the defect detection process performed when the number of pulse signals of the control signal CLK in one frame period is changed only needs to detect adjacent pixel 401b, which reduces the amount of detection processing. At the same time, the amount of information of defective pixel data can also be reduced.

[0152] Fig. 23 is a diagram showing the internal configuration of the defective pixel detection unit 108 in Example 3. The defective pixel detection unit 108 shown in Fig. 23 has a configuration in which a defective pixel calculation unit 601 is added to the configuration of the defective pixel detection unit 108 described in Fig. 2. Therefore, a description of the configuration other than the defective pixel calculation unit 601 will be omitted.

[0153] 17 and 18, the fewer the number of pulse signals of the control signal CLK, the lower the level of the adjacent pixel 401b of the cluster defect. The relationship between the number of pulse signals of the control signal CLK and the pixel level of the adjacent pixel 401b to the pixel 401a, which is the central pixel in the cluster defect, is as shown in FIG.

[0154] The defective pixel calculation unit (estimation unit) 601 estimates (predicts) defective pixel data of control settings in other modes using defective pixel data acquired by driving with a plurality of pulse signal numbers of the control signal CLK. That is, using defective pixel data acquired in a plurality of modes with different clock numbers shown in the first and second embodiments, defective pixel data of control settings in other modes with different clock numbers from the plurality of modes is estimated. Specifically, defective pixel data of other control settings is predicted based on the relationship between the pulse signal number of the control signal CLK shown in FIG. 18 and the pixel level of the adjacent pixel 401b with respect to pixel 401a, which is the central pixel in the cluster defect. For example, the pixel level of the adjacent pixel 401b is acquired from the pixel value when the pulse signal number of the control signal CLK is N1 and the defective pixel data when the pulse signal number of the control signal CLK is Nn.

[0155] 18, the pixel level of the neighboring pixel 401b relative to the pixel 401a, which is the central pixel in the cluster defect, gradually decreases as the number of pulse signals is reduced, and the decrease becomes particularly large when the number of pulse signals is small. Specifically, it is expressed by the following formula (1).

number

[0156] The defective pixel data including the size of the adjacent defect of adjacent pixel 401b calculated by the defective pixel calculation unit 601 is input to the detection unit 201. The detection unit 201 then compares the defective pixel data with threshold data driven by the number of pulse signals of the control signal CLK, and creates defective pixel data. The detection unit 201 stores the defective pixel data in the RAM 105 and ROM 106 in a predetermined format, as in the first embodiment. Then, the defective pixel correction unit 109 corrects the defective pixel in the same manner as in the first embodiment.

[0157] As described above, in the imaging device 1 of the third embodiment, by utilizing defective pixel data in multiple driving modes, it is possible to predict defective pixel data in other control settings, thereby reducing the amount of processing required for defect detection and the amount of information required for defective pixel data.

[0158] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0159] The disclosure of this embodiment includes the following configuration, method, and program.

[0160] (Configuration 1) a photoelectric conversion element having an avalanche photodiode; a signal processing unit that performs correction on the signal acquired by the photoelectric conversion element based on position information of a pixel to be corrected; a control means for controlling the number of pulses of the pulse signal during a period for acquiring one frame of the signal, an imaging device, characterized in that a threshold value used by the signal processing unit to determine whether or not a signal requires the correction is different between a first mode and a second mode in which the number of pulses of the pulse signal is smaller than that of the first mode.

[0161] (Configuration 2) 2. The imaging device according to claim 1, wherein the signal processing unit selects a threshold value corresponding to the number of pulses of the pulse signal from among a plurality of preset threshold values.

[0162] (Configuration 3) 3. The imaging apparatus according to claim 2, wherein the signal processing unit determines whether or not the signal is a correction target based on the selected threshold value.

[0163] (Configuration 4) 4. The imaging device according to configuration 3, wherein when there is a pixel that exceeds the selected threshold, the signal processing unit outputs the pixel that exceeds the threshold as a pixel to be corrected.

[0164] (Configuration 5) Further comprising a switching unit that switches between the first mode and the second mode, 5. The imaging device according to any one of configurations 1 to 4, wherein the switching unit switches between the first mode and the second mode depending on illuminance based on at least a signal acquired by the photoelectric conversion element.

[0165] (Configuration 6) 6. The imaging device according to any one of configurations 1 to 5, further comprising a storage unit that stores position information of the pixel to be corrected.

[0166] (Configuration 7) The imaging device described in any one of configurations 1 to 5, characterized in that when selecting a threshold in the second mode from among a plurality of thresholds, the signal processing unit selects a threshold that is smaller than the threshold in the first mode.

[0167] (Configuration 8) 8. The imaging device according to any one of configurations 1 to 7, wherein the signal processing unit is capable of applying a gain to the signal.

[0168] (Configuration 9) 9. The imaging device according to configuration 8, wherein the signal processing unit increases the threshold value when a gain is applied to the signal in the same mode.

[0169] (Configuration 10) The imaging device described in any one of configurations 1 to 9, characterized in that the signal processing unit sets, as pixels to be corrected, pixels whose pixel levels change depending on the mode in the signal acquired in the first mode and the signal acquired in the second mode.

[0170] (Configuration 11) 11. The imaging device according to any one of configurations 1 to 10, wherein the signal processing unit corrects the signal based on pixel levels of pixels arranged around a pixel to be corrected.

[0171] (Configuration 12) a third mode in which the number of pulses of the pulse signal during a period in which one frame of a signal is acquired is different from the first mode and the second mode; The imaging device according to any one of configurations 1 to 4, wherein the signal processing unit determines whether or not to correct the signal acquired in the third mode based on data of pixels to be corrected acquired in the first mode and the second mode.

[0172] (Configuration 13) 13. The imaging device according to configuration 12, further comprising a switching unit that switches between the first mode, the second mode, and the third mode in accordance with the number of pulses of the pulse signal.

[0173] (Configuration 14) The imaging device according to any one of configurations 1 to 13, further comprising an estimation unit that estimates, based on each pixel information of a pixel to be corrected acquired in the first mode and the second mode, each pixel information of a pixel to be corrected in a mode with a clock number different from the number of pulses of the pulse signal in the first mode and the second mode.

[0174] (Configuration 15) The imaging device according to any one of configurations 3 to 11, characterized in that when the signal processing unit determines whether a pixel is a target pixel for correction or not, it makes the determination based on a change in pixel level of pixels arranged around the pixel to be corrected when switching from the first mode to the second mode.

[0175] (Configuration 16) 16. The imaging device according to any one of configurations 1 to 15, wherein the control means changes the number of pulses of the pulse signal in response to illuminance in a signal acquired by the photoelectric conversion element.

[0176] (Configuration 17) A control method for an imaging device, comprising: A correction is performed on a signal acquired by a photoelectric conversion element having an avalanche photodiode based on position information of a pixel to be corrected; Controlling the number of pulses of the pulse signal during a period for acquiring one frame of signals; a threshold value for determining whether or not a signal requires correction when performing the correction differs between a first mode and a second mode in which the number of pulses of the pulse signal is smaller than that of the first mode.

[0177] (Configuration 18) A program for causing a computer to execute a control method for an imaging device, A correction is performed on a signal acquired by a photoelectric conversion element having an avalanche photodiode based on position information of a pixel to be corrected; Controlling the number of pulses of the pulse signal during a period for acquiring one frame of signals; a threshold value for determining whether or not a signal requires correction when performing the correction differs between a first mode and a second mode in which the number of pulses of the pulse signal is smaller than that of the first mode.

[0178] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0179] 1. Imaging device 101 Lens 102 Image sensor 103 Signal Processing Unit 104 Control section 105 RAM 106 ROM 107 Output section 108 Defective pixel detection section 109 Defective pixel correction section 110 Video signal generator 111 Image processing section

Claims

1. A photoelectric conversion element having an avalanche photodiode in each pixel that outputs a photon detection signal by avalanche multiplication; a switch element that applies a voltage equal to or greater than a breakdown voltage to the avalanche photodiode in response to an input control signal; a control unit that controls the number of the control signals per unit time; a determination unit that determines whether or not each pixel of the photoelectric conversion element is a pixel to be subjected to correction processing based on a comparison between an output value of the pixel and a predetermined threshold value; an output unit that outputs defective pixel data including position information of pixels determined by the determination unit to be subject to correction processing, The determination unit When the number of the control signals per unit time is a first number, the predetermined threshold is set to a first threshold; When the number of the control signals per unit time is a second number that is smaller than the first number, the predetermined threshold is set to a second threshold; The photoelectric conversion device is characterized in that the second threshold is smaller than the first threshold.

2. The photoelectric conversion device described in claim 1, characterized in that the control unit switches between a first mode in which the number of control signals is the first number and a second mode in which the number of control signals is the second number depending on at least the illuminance based on the signal acquired by the photoelectric conversion element.

3. 3. The photoelectric conversion device according to claim 2, wherein the first mode is a mode for taking an image at a higher illuminance than the second mode.

4. 2. The photoelectric conversion device according to claim 1, further comprising a storage unit for storing the defective pixel data.

5. The photoelectric conversion device according to claim 1, characterized in that the determination unit makes the predetermined threshold value larger when the gain for the image signal output from the photoelectric conversion element is a second gain higher than the first gain than when the gain for the image signal output from the photoelectric conversion element is a first gain.

6. 3. The photoelectric conversion device according to claim 2, further comprising an estimation unit that estimates the defective pixel data in a third mode, in which the number of control signals in the first mode is different from that in the second mode, based on the defective pixel data output in the first mode and the second mode.

7. An imaging device comprising the photoelectric conversion device according to claim 1, an output section for outputting the defective pixel data to the image pickup device; and a signal processing section for correcting the image signal outputted by the photoelectric conversion element based on the defective pixel data outputted from the output section.

8. 8. The imaging device according to claim 7, wherein the signal processing unit corrects the image signal based on pixel levels of pixels arranged around the pixel to be corrected, which is indicated by the defective pixel data.

Images