Imaging device

Abstract

To provide an imaging apparatus capable of suppressing image deterioration.SOLUTION: The imaging apparatus comprises: an imaging unit which includes pixels each outputting a signal based on an electric charge generated by a photoelectric conversion unit; and a correction unit which corrects a first signal based on the electric charge generated by the photoelectric conversion unit at a first temperature and in a first period while light into the imaging unit is shielded, and a second signal based on the electric charge generated by the photoelectric conversion unit while light enters the imaging unit, on the basis of the first temperature and the first period.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] Generally, when taking pictures with a long exposure time using an electronic camera, fixed pattern noise and random noise appear in the image data. Patent Document 1 discloses a device that can remove these noises. However, conventional devices have not been able to sufficiently remove these noises. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-141583 Summary of the Invention

[0004] According to a first aspect of the present invention, an imaging device includes an imaging unit having a plurality of pixels that output signals based on charges generated in a photoelectric conversion unit, and a correction target pixel among the plurality of pixels. of and a correction unit configured to correct, for each pixel, a first signal based on charges generated in the photoelectric conversion unit at a first temperature and for a first period in a state in which light entering the imaging unit is blocked, and a second signal based on charges generated in the photoelectric conversion unit in a state in which light is incident on the imaging unit, based on the first temperature and the first period, wherein the correction unit determining, among the plurality of pixels, some pixels whose values ​​of the first signal are within a predetermined range as the pixels to be corrected; Among the plurality of pixels, Pixels other than those to be corrected pixels is determined as a normal pixel, and the first signal Value of with a predetermined value, and correcting the second signal based on the first signal after the clipping process. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a diagram illustrating an example of the configuration of an imaging device according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of the configuration of a portion of an imaging element according to a first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of a histogram of dark current of each pixel of the image sensor according to the first embodiment. [Figure 4] FIG. 3 is a diagram showing an example of a histogram of pixel signals of each pixel of the image sensor according to the first embodiment. [Figure 5] 4 is a flowchart showing an example of the operation of the imaging device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0006] (First embodiment) 1 is a diagram showing an example of the configuration of a camera 1, which is an example of an imaging device according to a first embodiment. The camera 1 includes a photographing optical system (imaging optical system) 2, an image sensor 3, a control unit 4, a memory 5, a display unit 6, an operation unit 7, and a temperature detection unit 8. The photographing optical system 2 has multiple lenses, including a focus adjustment lens (focus lens), and an aperture stop, and forms a subject image on the image sensor 3. The photographing optical system 2 may be detachable from the camera 1.

[0007] The imaging element 3 is an imaging element such as a CMOS image sensor or a CCD image sensor. The imaging element 3 receives a light beam that has passed through the photographing optical system 2 and captures an image of a subject formed by the photographing optical system 2. The imaging element 3 has a plurality of pixels, each having a photoelectric conversion unit, arranged two-dimensionally (in the row and column directions). The photoelectric conversion unit is composed of a photodiode (PD).

[0008] The pixels include pixels (R pixels) having filters that separate incident light in a first wavelength range (red (R) light), pixels (G pixels) having filters that separate incident light in a second wavelength range (green (G) light), and pixels (B pixels) having filters that separate incident light in a third wavelength range (blue (B) light). The R, G, and B pixels are arranged in a Bayer array. The image sensor 3 photoelectrically converts the received light to generate signals (pixel signals) and outputs the generated signals to the control unit 4.

[0009] The memory 5 is composed of a non-volatile storage medium. Image signals (image data), control programs, etc. are stored (recorded) in the memory 5. Writing data to the memory 5 and reading data from the memory 5 are controlled by the control unit 4. The display unit 6 displays images based on the image signals, shooting-related information such as shutter speed and aperture value, and menu screens, etc. The operation unit 7 includes various setting switches such as a release button, a power switch, and switches for switching between various modes, and outputs signals to the control unit 4 based on the respective operations.

[0010] The control unit 4 is composed of a processor such as a CPU, FPGA, or ASIC, and memories such as ROM and RAM, and controls each unit of the camera 1 based on a control program. The control unit 4 supplies a signal that controls the image sensor 3 to the image sensor 3, thereby controlling its operation. The control unit 4 controls the period (charge accumulation period) during which charge is accumulated in the photoelectric conversion unit of the image sensor 3, and the gain (amplification factor) used when amplifying pixel signals in the image sensor 3. When capturing still images or videos, or when displaying a through image (live view image) of the subject on the display unit 6, the control unit 4 causes the image sensor 3 to capture an image of the subject and output pixel signals for each pixel. The control unit 4 stores image signals (image data) including the pixel signals for each pixel in the memory 5.

[0011] The control unit 4 has a storage unit 4a and a processing unit (correction unit) 4b. As will be described later, the storage unit 4a stores an image signal (dark image signal) obtained by capturing an image in a state where light to the image sensor 3 is blocked, and information (condition information) relating to the conditions under which the image was captured in the light-blocked state. The dark image signal and the condition information may be stored in the memory 5. Information relating to a plurality of arithmetic expressions, which will be described later, is stored in the storage unit 4a (or the memory 5).

[0012] As will be described later, the processing unit 4b uses the dark image signal and condition information stored in the storage unit 4a to perform correction processing (hereinafter referred to as noise correction processing) to remove dark current components from the image signal obtained by capturing an image of a subject. The processing unit 4b performs various image processing on the image signal after noise correction processing to generate an image signal for a still image or an image signal for a moving image. The image processing includes image processing such as gradation conversion processing and color interpolation processing.

[0013] The temperature detection unit 8 is configured with a temperature sensor such as a thermistor or a thermocouple, and detects the temperature of the image sensor 3. The temperature detection unit 8 generates information (temperature information) related to the temperature of the image sensor 3 and outputs the generated temperature information to the control unit 4. Note that a circuit for detecting the temperature of the image sensor 3 may be provided inside the image sensor 3 as the temperature detection unit. In this case, the image sensor 3 generates and outputs the temperature information to the control unit 4.

[0014] FIG. 2 is a diagram showing an example of the configuration of a portion of an image sensor according to the first embodiment. FIG. 2 shows some of the pixels 10 of the plurality of pixels 10 provided in the image sensor 3, a vertical control unit 25, and a horizontal control unit 22. FIG. 2 also shows a portion of two pixel columns of the plurality of pixels 10 arranged in the column direction (vertical direction) and the row direction (horizontal direction) intersecting the column direction. In the image sensor 3, vertical signal lines 20 are provided for pixel columns, which are columns of pixels arranged in the column direction, i.e., the vertical direction. A current source (not shown) is provided for the vertical signal line 20. The configurations of the other pixel columns are the same as the configuration of the pixel columns in FIG. 2.

[0015] Each pixel 10 includes a photoelectric conversion unit 11, a transfer unit 12, a floating diffusion (FD) 13, a reset unit 14, an amplifier unit 15, and a selection unit 16. The photoelectric conversion unit 11 is a photodiode PD that converts incident light into electric charges and accumulates the photoelectrically converted electric charges. The transfer unit 12 is composed of a transistor M1 controlled by a signal TG, and transfers the electric charges photoelectrically converted by the photoelectric conversion unit 11 to the FD13. The transistor M1 is a transfer transistor. The FD13 accumulates (holds) the electric charges transferred to the FD13.

[0016] The amplifier 15 amplifies and outputs a signal due to the charge accumulated in the FD 13. The amplifier 15 is configured with a transistor M3 whose drain (terminal), gate (terminal), and source (terminal) are connected to a power supply VDD, the FD 13, and the selection unit 16, respectively. The source of the amplifier 15 is connected to a vertical signal line 20 via the selection unit 16. The transistor M3 is an amplifying transistor.

[0017] The reset unit 14 is composed of a transistor M2 controlled by a signal RST, and discharges the charge accumulated in the FD 13 and resets the voltage (potential) of the FD 13. The transistor M2 is a reset transistor. The selection unit 16 is composed of a transistor M4 controlled by a signal SEL, and electrically connects or disconnects the amplifier unit 15 and the vertical signal line 20. When the transistor M4 of the selection unit 16 is in an on state, it outputs a signal from the amplifier unit 15 to the vertical signal line 20. The transistor M4 is a selection transistor. The amplifier unit 15 and the selection unit 16 constitute an output unit that generates and outputs a signal based on the charge generated by the photoelectric conversion unit 11.

[0018] As described above, the charges photoelectrically converted by the photoelectric conversion unit 11 are transferred to the FD 13 by the transfer unit 12. A signal (pixel signal) corresponding to the charges transferred to the FD 13 is output to the vertical signal line 20. The pixel signal output from the pixel 10 is an analog signal generated based on the charges photoelectrically converted by the photoelectric conversion unit 11.

[0019] The vertical control unit 25 is provided in common to multiple pixel columns. The vertical control unit 25 supplies signals TG (TG1 and TG2 in FIG. 2), signals RST (RST1 and RST2 in FIG. 2), and signals SEL (SEL1 and SEL2 in FIG. 2) to each pixel to control the operation of each pixel. The vertical control unit 25 supplies signals to the gates of each transistor in the pixels to turn the transistor on (connected, conductive, short-circuited) or off (disconnected, non-conductive, open, or blocked)

[0020] The horizontal control unit 22 includes an amplifier unit and a signal processing unit. An amplifier unit is provided for each vertical signal line 20, and amplifies pixel signals input via the vertical signal lines 20 at a predetermined gain (amplification factor). In this case, the amplifier unit determines the gain according to the ISO sensitivity automatically selected by the camera 1 or the ISO sensitivity selected by the user through operation of the operation unit 7, and performs processing to multiply the pixel signals by the determined gain. The amplifier unit outputs the amplified pixel signals to the signal processing unit.

[0021] The signal processing unit has an analog / digital conversion unit (AD conversion unit) and converts the pixel signal output from the amplifier unit into a digital signal. Note that instead of or in addition to the amplification process by the amplifier unit, the signal processing unit may determine a gain according to the selected ISO sensitivity and multiply the pixel signal by the determined gain. After performing signal processing on the pixel signal, the signal processing unit outputs the processed pixel signal to the control unit 4 of the camera 1. When the pixel signal is amplified by the horizontal control unit 22, the dark current component included in the pixel signal is also amplified by a gain determined based on the ISO sensitivity. Therefore, the magnitude of the noise component included in the pixel signal depends on the ISO sensitivity set in the camera 1.

[0022] Because dark current may occur in the image sensor 3, the pixel signal output from the image sensor 3 may contain a signal component (dark current component) resulting from the dark current. The dark current component of the pixel signal acts as a noise component in the pixel signal. The magnitude of the dark current component varies depending on the imaging conditions (imaging conditions), such as the charge accumulation period during which charge is accumulated in the photoelectric conversion unit 11, the temperature during charge accumulation in the photoelectric conversion unit 11, and the ISO sensitivity. Furthermore, a pixel 10 with a defect, such as a pixel 10 with a defect factor such as an impurity level due to the inclusion of a foreign element or the like in the photoelectric conversion unit 11 of the pixel 10, tends to have a relatively large dark current compared to other pixels 10, resulting in a large dark current component being mixed into the pixel signal. When an image is displayed using an image signal including a pixel signal mixed with such a large dark current component, defects due to the dark current will appear in the image. Such defective pixels 10 and defects in the image caused by such defective pixels 10 are also called white spots. White spots become noticeable in images when an exposure is performed for a long time or when the temperature of the image sensor 3 is high.

[0023] The image sensor 3 according to this embodiment performs an imaging operation (hereinafter referred to as "main imaging") for acquiring pixel signals used to generate image signals for recording or display, and an imaging operation (hereinafter referred to as "preliminary imaging") for acquiring pixel signals used to correct the image signals obtained by the main imaging. In this embodiment, the preliminary imaging is performed in a state in which light reaching the image sensor 3 is blocked, prior to shipping of the camera 1. The control unit 4 causes the image sensor 3 to perform the preliminary imaging and output pixel signals from each pixel 10. In this case, no light is incident on the photoelectric conversion unit 11 from the outside, and therefore the pixels 10 output pixel signals according to the amount of dark current.

[0024] The processing unit 4b of the control unit 4 performs signal processing, described below, on the pixel signals of each pixel obtained by performing preliminary imaging. The processing unit 4b stores, in the storage unit 4a (or memory 5), a dark image signal including the pixel signals of each pixel after signal processing, and condition information related to the imaging conditions when the preliminary imaging was performed. The condition information when the preliminary imaging was performed includes information related to the temperature of the image sensor 3, the charge accumulation period, and the ISO sensitivity when the preliminary imaging was performed.

[0025] In the actual imaging, the image sensor 3 captures the subject image formed by the photographing optical system 2. The processing unit 4b stores in the memory 5 an image signal including pixel signals of each pixel output from the image sensor 3, and condition information relating to the imaging conditions when the actual imaging was performed. The condition information during the actual imaging includes information relating to the temperature of the image sensor 3 when the actual imaging was performed, the charge accumulation period, and the ISO sensitivity. The image signal during the actual imaging and the condition information during the actual imaging may also be stored in the storage unit 4a.

[0026] The processing unit 4b performs noise correction processing on the image signal obtained by the actual imaging. Specifically, the processing unit 4b estimates an image signal corresponding to the amount of dark current generated during the actual imaging based on the dark image signal obtained by the preliminary imaging, the condition information for the preliminary imaging, and the condition information for the actual imaging. In this embodiment, the processing unit 4b calculates an image signal corresponding to the amount of dark current generated during the actual imaging by multiplying the dark image signal by a gain determined based on the condition information for the preliminary imaging and the actual imaging. The processing unit 4b subtracts the calculated image signal from the image signal obtained by the actual imaging. By subtracting the dark image signal from the preliminary imaging multiplied by the gain from the image signal from the actual imaging, the processing unit 4b can reduce noise components due to dark current contained in the image signal from the actual imaging.

[0027] Differences in the impurities, defects, etc. present in the pixel 10 result in differences in the impurity levels generated in the photoelectric conversion unit 11 (or the interface between the material constituting the photoelectric conversion unit 11 and other materials). In pixels 10 in which metal is mixed into the photoelectric conversion unit 11, forming impurity levels, the amount of dark current tends to be large due to the impurity levels. For this reason, the image sensor 3 may include pixels 10 with relatively low dark current and pixels 10 with relatively high dark current. Generally, when noise correction processing is performed on pixel signals output from pixels 10 with relatively low dark current, the noise components contained in the pixel signals may increase. This may result in deterioration of image quality due to an increase in random noise contained in the pixel signals, or a change in the ratio of the color components (R, G, and B components) of the image signal, causing color shifts in the image.

[0028] In this embodiment, noise correction processing is not performed on pixel signals from pixels with relatively little dark current, but is performed on pixel signals from pixels with relatively much dark current. In the following description, of the pixels 10 provided in the image sensor 3, pixels 10 with relatively much dark current will be referred to as correction target pixels, and pixels 10 with relatively little dark current will be referred to as normal pixels.

[0029] The dark current value of a normal pixel is DC'[e - ] can be expressed by the following formula (1). DC'=DC×{exp[-(q / k×Ea_d) / Temp_d] / exp[-(q / k×Ea_d) / Temp_st)]}×Time_d =DC×exp[(q / k×Ea_d)×(1 / Temp_st-1 / Temp_d)]×Time_d …(1) The dark current value DC_h'[e - ] can be expressed by the following equation (2). DC_h'=DC_h×exp[(q / k×Ea_h)×(1 / Temp_st-1 / Temp_d)]×Time_d …(2) In the above equation (1), DC[e - / s] is the dark current value of a normal pixel when the temperature (absolute temperature) is Temp_st [K], and Ea_d [eV] is the activation energy of the dark current of a normal pixel. - / s] is the dark current value of the pixel to be corrected when the temperature is Temp_st [K], and Ea_h [eV] is the activation energy of the dark current of the pixel to be corrected. q is the elementary charge (elementary charge), k is the Boltzmann constant, Temp_d is the temperature during preliminary imaging, and Time_d is the charge accumulation period (exposure time) during preliminary imaging. Generally, the higher the temperature during imaging and the longer the charge accumulation period, the larger the dark current value. Furthermore, as the temperature during preliminary imaging, Temp_d, increases and the charge accumulation period, Time_d, increases, the dark current value DC_h' of the pixel to be corrected becomes larger than the dark current value DC' of a normal pixel, and the dark current values ​​DC' and DC_h' become separated from each other.

[0030] When charge due to dark current accumulates in a pixel, dark current shot noise occurs due to that charge, and its value is the dark current [e - ]. When the charge accumulation time is long, such as several seconds, and the temperature of the image sensor is 10°C or more higher than room temperature, the dark current shot noise component becomes dominant in the random noise generated in the pixel area. Therefore, for example, if the average dark current value of a normal pixel is N, the random noise is (N)^0.5, and the standard deviation of the signals from multiple normal pixels is (N)^0.5. Hereinafter, this value will be referred to as the standard deviation of the dark current values. The standard deviation of the dark current values ​​of each correction target pixel is given by the square root of the average dark current value of each correction target pixel. The standard deviation of the dark current values ​​of each normal pixel is given by the square root of the average dark current value of each normal pixel. Furthermore, in the case of preliminary imaging, the pixel signals output from the correction target pixel and the normal pixel correspond to the dark current value DC_h' of the correction target pixel and the dark current value DC' of the normal pixel, respectively. Therefore, when the condition expressed by the following equation (3) is met, it is possible to distinguish between the correction target pixel and the normal pixel. DC'+b×sqrt(DC') <DC_h’-b×sqrt(DC_h’) …(3) In the above formula (3), b is the magnification of the standard deviation. The magnification b may be appropriately selected according to the image sensor used, etc. For example, the temperature Temp_d and the charge accumulation period Time_d are determined so as to satisfy the formula (3) with the magnification b = 3.5, and pre-imaging is performed at the determined temperature Temp_d and charge accumulation period Time_d, whereby it is possible to sufficiently distinguish the pixel signal of the pixel to be corrected from the pixel signal of the normal pixel. Such conditions of temperature and charge accumulation period are conditions under which many white dots caused by dark current having a specific temperature dependence occur. The distribution of the dark current of each pixel of the image sensor 3 when pre-imaging is performed under such conditions is as shown in FIG. 3.

[0031] FIG. 3 is a diagram showing an example of a histogram of the dark current of each pixel of the image sensor according to the first embodiment. In FIG. 3, the horizontal axis indicates the magnitude of the dark current, and the vertical axis indicates the frequency (number) of pixels indicating each dark current value. In the example shown in FIG. 3, the difference between the peak value DC’ of the dark current value of the normal pixel and the peak value DC_h’ of the dark current value of the pixel to be corrected is large, and the conditional expression with b = 3.5 in the formula (3), that is, DC’ + 3.5 × sqrt(DC’) < DC_h’ - 3.5 × sqrt(DC_h’) is satisfied. Therefore, the pixel to be corrected and the normal pixel can be identified.

[0032] FIG. 4 is a diagram showing an example of a histogram of the pixel signal of each pixel of the image sensor according to the first embodiment, and shows a histogram of the pixel signal of each pixel included in the dark-time image signal obtained by performing pre-imaging. In FIG. 4, the horizontal axis indicates the signal level of the pixel signal, and the vertical axis indicates the frequency of the pixel signal corresponding to each signal level. FIG. 4(a) shows a histogram of the pixel signal of each pixel before the clip processing described later, and FIG. 4(b) shows a histogram of the pixel signal of each pixel after the clip processing.

[0033] As shown in Fig. 4(a), the processing unit 4b sets a first threshold Thre_d1 and a second threshold Thre_d2 for determining pixels to be corrected. The first threshold Thre_d1 and the second threshold Thre_d2 are determined based on the distribution of pixel signal values ​​of each pixel included in the dark image signal. In this embodiment, the processing unit 4b calculates the first threshold Thre_d1 and the second threshold Thre_d2 using the following equations (4) and (5), respectively. Thre_d1=Thre_st1×G_Temp1×(Time_d / Time_st)×(2 (SV_d-SV_st) ) …(4) Thre_d2=Thre_st2×G_Temp1×(Time_d / Time_st)×(2 (SV_d-SV_st) ) …(5) In the above equations (4) and (5), thresholds Thre_st1 and Thre_st2 are the first and second thresholds, respectively, under the conditions (temperature Temp_st, ISO sensitivity SV_st, and charge accumulation period Time_st) under which the pixel signals of the correction target pixels and the pixel signals of the normal pixels are completely separated. Thresholds Thre_st1 and Thre_st2 are determined in advance by simulation, experiment, etc., before the preliminary capture. SV_d is the ISO sensitivity during the preliminary capture. Furthermore, G_Temp1 is a parameter expressed by the following equation (6). G_Temp1=exp[(q / k×Ea_d)×(1 / Temp_st-1 / Temp_d)] …(6)

[0034] The processing unit 4b determines whether each pixel is a pixel to be corrected based on the pixel signal of each pixel obtained by performing preliminary imaging and the first threshold value Thre_d1 and the second threshold value Thre_d2. Of the pixel signals of each pixel, the processing unit 4b determines pixel signals having a signal level between the first threshold value Thre_d1 and the second threshold value Thre_d2 as pixel signals of correction pixels. The processing unit 4b determines pixel signals that are equal to or less than the first threshold value Thre_d1 and pixel signals that are equal to or greater than the second threshold value Thre_d2 as pixel signals of normal pixels.

[0035] The processing unit 4b performs a process (clipping process) in which the pixel signal values ​​(signal levels) of normal pixels are replaced with a predetermined value (e.g., 0). As a result, as shown in FIG. 4B, the frequency (number) of pixel signals whose signal levels are equal to or less than the first threshold Thre_d1 or equal to or greater than the second threshold Thre_d2 becomes 0. Furthermore, the processing unit 4b retains the values ​​of pixel signals whose signal levels are within the range from the first threshold Thre_d1 to the second threshold Thre_d2. The processing unit 4b stores, in the storage unit 4a, a dark image signal including the pixel signals of each pixel after the clipping process. The processing unit 4b also acquires temperature information output from the temperature detection unit 8 during preliminary image capture as information regarding the temperature Temp_d of the image sensor 3 during preliminary image capture. The processing unit 4b stores, in the storage unit 4a, the temperature information during preliminary image capture, information regarding the charge accumulation period Time_d, and information regarding the ISO sensitivity SV_d, as condition information during preliminary image capture.

[0036] When the actual imaging is performed, the processing unit 4b stores image signals including pixel signals of each pixel obtained by performing the actual imaging in the memory 5. The processing unit 4b also acquires temperature information output from the temperature detection unit 8 during the actual imaging as information related to the temperature Temp_h of the image sensor 3 during the actual imaging. The processing unit 4b stores the temperature information during the actual imaging, information related to the charge accumulation period Time_h, and information related to the ISO sensitivity SV_h in the memory 5 as condition information for the actual imaging.

[0037] The processing unit 4b calculates the gain G based on the condition information for the preliminary imaging and the main imaging and the following equation (7). G=G_adj×G_Temp2×(Time_h / Time_d)×(2 (SV_h-SV_d) ) …(7) In equation (7), G_adj is an adjustment gain that is obtained in advance through experiments or simulations, and G_Temp2 is a parameter expressed by the following equation (8). G_Temp2=exp[(q / k×Ea_h)×(1 / Temp_st-1 / Temp_h)] …(8)

[0038] The processing unit 4b obtains an image signal corresponding to the amount of dark current generated during actual imaging by multiplying the dark image signal after clipping stored in the storage unit 4a by the calculated gain G. The processing unit 4b subtracts the dark image signal after multiplication by the gain G from the image signal during actual imaging stored in the memory 5. This makes it possible to reduce noise components due to dark current contained in the image signal during actual imaging.

[0039] In this embodiment, noise correction processing is performed on the image signal obtained by the actual image capture based on the dark image signal acquired before the actual image capture. Therefore, compared to performing an image capture with the image sensor shielded from light after the actual image capture to acquire a dark image signal and then calculating the difference between the dark image signal and the image signal obtained during the actual image capture, the time required to perform the actual image capture and obtain the image signal after noise correction processing can be shortened. In this embodiment, the actual image capture operation is performed only once, and it is not necessary to acquire a dark image signal for each capture, thereby shortening the time required for capture. Furthermore, in this embodiment, there is no need to provide a light-blocking element (such as a mechanical shutter or an in-lens aperture mechanism) in the camera 1 to block light from reaching the image sensor 3 after the actual image capture, which prevents the camera 1 from becoming larger and more expensive.

[0040] Furthermore, the processing unit 4b calculates an image signal corresponding to the amount of dark current during actual imaging based on the condition information during preliminary imaging and actual imaging and the dark image signal. Therefore, even if the imaging conditions (temperature, exposure time (charge accumulation period), ISO sensitivity) differ between preliminary imaging and actual imaging, the image signal corresponding to the amount of dark current during actual imaging can be accurately determined. The processing unit 4b estimates an image signal corresponding to the amount of dark current generated during actual imaging, taking into account not only the charge accumulation period but also the temperature and ISO sensitivity, and performs correction processing on the image signal during actual imaging. As a result, the processing unit 4b can improve the accuracy of noise correction processing. It is possible to reduce white spots that appear in the image and improve image quality.

[0041] Furthermore, as described above, since the pixel signals of normal pixels included in the dark image signal after clipping have a value of 0, the pixel signals of normal pixels included in the dark image signal after multiplication by the gain G also have a value of 0. When the processing unit 4b performs the subtraction described above, the pixel signals of the correction target pixels included in the image signal during actual imaging are corrected, but the pixel signals of normal pixels are not corrected. In this way, the processing unit 4b can partially (locally) correct the image signal during actual imaging. This makes it possible to suppress an increase in noise components included in the pixel signals of normal pixels and the occurrence of color shifts. Furthermore, since the pixel signals of normal pixels included in the dark image signal have a value of 0, the amount of data stored in the storage unit 4a (or memory 5) can be reduced.

[0042] Fig. 5 is a flowchart showing an example of the operation of the camera 1 according to the first embodiment. The process shown in Fig. 5 is executed when, for example, the user operates the release button to issue an instruction to perform actual imaging.

[0043] In step S100, the control unit 4 causes the image sensor 3 to capture an image of the subject and output a pixel signal for each pixel while light from the subject is incident on the image sensor 3. In step S110, the control unit 4 causes the memory 5 to store image signals including the pixel signals for each pixel output from the image sensor 3. The control unit 4 also causes the memory 5 to store condition information including information regarding the charge accumulation period Time_h during actual imaging, temperature information regarding the temperature Temp_h of the image sensor 3 during actual imaging, and information regarding the ISO sensitivity SV_h during actual imaging.

[0044] In step S120, the control unit 4 determines whether the charge accumulation period Time_h during actual imaging is equal to or greater than a predetermined time. The control unit 4 also determines whether the temperature Temp_h of the image sensor 3 during actual imaging is equal to or greater than a predetermined temperature. If the control unit 4 determines that the charge accumulation period Time_h is equal to or greater than the predetermined time and that the temperature Temp_h is equal to or greater than a predetermined temperature, the control unit 4 proceeds to step S130. Note that if a negative determination is made in step S120, noise correction processing is not performed on the image signal during actual imaging.

[0045] In step S130, the control unit 4 calculates the gain G based on the condition information for the preliminary image capture and the actual image capture and on the above-described equation (7). In step S140, the control unit 4 multiplies the dark image signal after clipping stored in the storage unit 4a by the calculated gain G. In step S150, the control unit 4 subtracts the dark image signal multiplied by the gain G from the image signal for the actual image capture. This makes it possible to remove noise components due to dark current from the image signal for the actual image capture. The control unit 4 stores the image signal for the actual image capture after the subtraction in the memory 5.

[0046] As described above, when the charge accumulation period is equal to or longer than a predetermined time and the temperature is equal to or higher than a predetermined temperature, i.e., when it is assumed that the image signal contains a large amount of dark current, the control unit 4 performs noise correction processing on the image signal during actual imaging. Therefore, when the charge accumulation period is shorter than the predetermined time or the temperature is lower than the predetermined temperature, i.e., when it is assumed that the image signal contains a small amount of dark current, it is possible to prevent an increase in noise components and color shifts from occurring by performing noise correction processing.

[0047] According to the above-described embodiment, the following effects can be obtained. (1) An imaging device (camera 1) includes an imaging unit (image sensor 3) having pixels 10 that output signals based on charges generated by photoelectric conversion units 11, and a correction unit (processing unit 4b) that corrects a first signal based on charges generated by photoelectric conversion unit 11 at a first temperature and for a first period when light entering the imaging unit is blocked, and a second signal based on charges generated by photoelectric conversion unit 11 when light is incident on the imaging unit, based on the first temperature and the first period. In this embodiment, processing unit 4b corrects the image signal during actual imaging based on a dark image signal obtained by preliminary imaging and condition information for the preliminary imaging. This reduces the dark current component included in the image signal during actual imaging, thereby preventing degradation of image quality. (2) The processing unit 4b calculates an image signal corresponding to the amount of dark current generated during the actual image capture based on the temperature and charge accumulation period of the image sensor 3 during the preliminary image capture, and performs noise correction processing on the image signal during the actual image capture, thereby improving the accuracy of the noise correction processing.

[0048] The following modifications are also within the scope of the present invention, and one or more of the modifications may be combined with the above-described embodiment.

[0049] (Variation 1) When the noise component contained in the dark image signal is large and the noise component contained in the image signal during actual imaging is small, the accuracy of the noise correction process may decrease. Therefore, in this modification, the control unit 4 performs preliminary imaging multiple times to obtain multiple dark image signals and averages the multiple dark image signals. The control unit 4 performs clipping on the averaged dark image signals and stores the clipped dark image signals in the storage unit 4a (or memory 5). This allows the control unit 4 to perform noise correction on the image signal during actual imaging using the averaged dark image signals, thereby preventing a decrease in the accuracy of the noise correction process. Because the clipped dark image signals contain many pixel signals with a signal value of 0, the control unit 4 may perform Huffman compression (Huffman coding) on ​​the clipped dark image signals and store the compressed dark image signals in the storage unit 4a.

[0050] (Variation 2) If the temperature of the image sensor 3 changes during actual imaging, an error may occur in the gain G. Therefore, in this modification, the control unit 4 acquires multiple pieces of temperature information repeatedly detected during the charge accumulation period during actual imaging from the temperature detection unit 8. The control unit 4 calculates the gain G using the following equation (9) instead of the above-mentioned equation (7). G=G_adj×2 (SV_h-SV_d) )×1 / Time_d×Σ(G_Temp'×Δt) …(9) In equation (9), Δt is the time interval at which the temperature detection unit 8 detects the temperature, and ΣΔt=Time_h. Furthermore, G_Temp′ is a parameter expressed by the following equation (10). G_Temp'=exp[(q / k×Ea_h)×(1 / Temp_d-1 / Temp_h')] …(10) In equation (10), Temp_h' is the temperature of the image sensor 3 at each time as determined from the temperature information. The control unit 4 calculates the gain G taking into consideration the temperature of the image sensor 3 during the charge accumulation period of the actual image capture, and performs noise correction processing on the image signal during the actual image capture. This allows the noise correction processing to be performed with high accuracy.

[0051] (Variation 3) In the above-described embodiment, an example has been described in which a dark image signal is acquired in a process before shipping of the camera 1. However, it may also be possible to acquire a dark image signal in a process after shipping of the camera 1. By updating the dark image signal and performing noise correction processing using the latest dark image signal, it is possible to improve the accuracy of the noise correction processing.

[0052] Furthermore, defective pixels may occur in the image sensor 3 after the manufacture of the image sensor 3, and the control unit 4 may detect defective pixels based on pixel signals output from each pixel 10 when light to the pixel 10 is blocked. When detecting defective pixels, the control unit 4 may acquire a dark image signal. Furthermore, the control unit 4 may determine whether or not to acquire a dark image signal based on the remaining battery power of the camera 1. If the control unit 4 detects that the remaining battery power of the camera 1 is low, it may not acquire a dark image signal.

[0053] (Variation 4) The noise correction process may be performed only on some of the pixel signals among the pixel signals of each pixel included in the image signal. The control unit 4 may perform the noise correction process only on pixel signals included in the image signal at the time of actual imaging, where noise is likely to be noticeable when the image is displayed.

[0054] (Variation 5) In the above-described embodiment, an example has been described in which subtraction processing is performed on the pixel signal of a pixel to be corrected. However, the control unit 4 may also perform processing to interpolate the pixel signal of the pixel to be corrected during actual imaging using pixel signals of pixels surrounding the pixel to be corrected. In this case, the control unit 4 may replace the pixel signal of the pixel to be corrected with the average value of pixel signals of multiple (e.g., eight) pixels of the same color surrounding the pixel to be corrected. Alternatively, the control unit 4 may interpolate the pixel signal of the pixel to be corrected using the pixel signal with the fourth highest signal level among the pixel signals of the eight pixels of the same color surrounding the pixel to be corrected.

[0055] (Variation 6) The control unit 4 may calculate an average value of pixel signals of the correction target pixels included in the dark image signal, and subtract the calculated average value from each of the pixel signals of the plurality of correction target pixels included in the image signal at the time of actual imaging. As described above, the pixel signals of the correction target pixels include the average dark current [e - ] includes a random noise component due to dark current shot noise of the square root of [], and by subtracting it with a fixed value, it is possible to prevent an increase in random noise due to subtraction between random properties.

[0056] (Variation 7) The control unit 4 may perform dark shading correction using the dark image signal before clipping. In this case, after the preliminary image capture, the control unit 4 stores the dark image signal before clipping in the storage unit 4a (or memory 5). The control unit 4 subtracts the dark image signal before clipping from the image signal during the main image capture. Alternatively, the control unit 4 may divide the pixel signals of each pixel included in the dark image signal into multiple regions within the image and store the average or median value of the pixel signals for each divided region in the storage unit 4a. Since the storage unit 4a stores the dark image signal consisting of the average or median value of the pixel signals for each divided region, the amount of data stored in the storage unit 4a can be reduced. The control unit 4 performs smoothing processing using a Gaussian filter or the like to restore the dark image signal consisting of the average values ​​of the pixel signals for each divided region to the original image size, and then subtracts the processed dark image signal from the image signal obtained during actual capture. Even when no division is performed, smoothing processing using a Gaussian filter or the like may be performed to remove high-frequency components before subtraction. In this modification, dark shading correction is performed on the image signal obtained during actual capture, thereby reducing noise components in the image signal.

[0057] Alternatively, the control unit 4 may multiply the dark image signal, which is the average value of the pixel signals from each divided region, by a gain G' corresponding to a normal dark current, such as the dark current generated in a normal pixel, rather than the relatively large dark current caused by impurities as described above, and subtract the dark image signal after multiplication by the gain G' from the image signal obtained during actual imaging. This reduces the dark current component contained in the image signal obtained during actual imaging. Note that the gain G' is a different value from the gain G described above and is determined in advance by simulation, experiment, or the like.

[0058] (Variation 8) In the above-described embodiment, an example has been described in which the control unit 4 performs noise correction processing on the image signal during actual imaging when the charge accumulation period is equal to or longer than a predetermined time and the temperature is equal to or higher than a predetermined temperature. However, the control unit 4 may also perform noise correction processing on the image signal during actual imaging when at least one of the charge accumulation period during actual imaging is equal to or longer than a predetermined time and the temperature is equal to or higher than a predetermined temperature.

[0059] (Variation 9) In the above-described embodiment and modified example, a photodiode is used as the photoelectric conversion unit, but a photoelectric conversion film (organic photoelectric film) may be used as the photoelectric conversion unit.

[0060] (Variation 10) The imaging elements and imaging devices described in the above-mentioned embodiments and variations may be applied to cameras, smartphones, tablets, cameras built into PCs, in-vehicle cameras, cameras mounted on unmanned aerial vehicles (drones, radio-controlled aircraft, etc.), etc.

[0061] Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments that are conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0062] 1...imaging device, 3...imaging element, 4...control unit, 4a...storage unit, 4b...processing unit (correction unit), 5...memory, 10...pixel

Claims

1. an imaging unit having a plurality of pixels that output signals based on charges generated by the photoelectric conversion unit; a correction unit that corrects, for a pixel to be corrected among the plurality of pixels, a first signal based on charges generated in the photoelectric conversion unit at a first temperature and a first period in a state in which light to the imaging unit is blocked, and a second signal based on charges generated in the photoelectric conversion unit in a state in which light is incident on the imaging unit, based on the first temperature and the first period; Equipped with The correction unit determining, among the plurality of pixels, some pixels whose values ​​of the first signal are within a predetermined range as the pixels to be corrected; determining pixels other than the correction target pixel among the plurality of pixels as normal pixels; performing clipping processing to replace the value of the first signal of the normal pixel with a predetermined value; correcting the second signal based on the first signal after the clipping process; Imaging device.

2. In the imaging device according to claim 1, The correction unit determines, as the pixels to be corrected, some pixels among the plurality of pixels whose first signal values ​​are greater than a threshold value.

3. Original claim 2 3. The imaging device according to claim 1, The imaging device, wherein the predetermined value is 0.

4. 4. The imaging device according to claim 3, the first signal is a signal output at a first sensitivity based on charges generated by the photoelectric conversion unit at the first temperature and during the first period; the second signal is a signal output at a second sensitivity based on charges generated in the photoelectric conversion unit at a second temperature and for a second period; The correction unit corrects the second signal based on the first signal, the first temperature, the first period, and the first sensitivity.

5. 5. The imaging device according to claim 4, The correction unit corrects the second signal when at least one of the second temperature is equal to or higher than a predetermined temperature and the second period is equal to or longer than a predetermined period.

6. 6. The imaging device according to claim 4, The correction unit multiplies the first signal by a value based on the first temperature, the second temperature, the first period, the second period, the first sensitivity, and the second sensitivity, and subtracts the resulting first signal from the second signal.

Images