Imaging device, imaging system, endoscope, and imaging method

Abstract

This imaging device comprises two or more pixels and a column circuit that outputs pixel signals in accordance with charges held in the pixels arranged in at least one column among the two or more pixels. In a first mode, each pixel sets the state of a transfer transistor to the OFF state and sets the state of a reset transistor to the ON state. In the first mode, each pixel sets the state of the reset transistor to the OFF state and sets the state of the transfer transistor to the ON state. In a second mode, each pixel maintains the state of the transfer transistor in the OFF state and executes the same control as the control of the reset transistor in the first mode.

Description

Imaging Device, Imaging System, Endoscope, and Imaging Method

[0001] The present disclosure relates to an imaging device, an imaging system, an endoscope, and an imaging method.

[0002] The image sensor has two or more pixels arranged in a matrix. Each pixel has a photoelectric conversion element, a floating diffusion (FD), a transfer switch, a reset switch, etc. The photoelectric conversion element receives light and generates electric charges according to the amount of the light. The FD holds the electric charges generated by the photoelectric conversion element. The transfer switch transfers the electric charges generated by the photoelectric conversion element to the FD. The reset switch resets the electric charges held in the FD.

[0003] An image sensor having optical black (OB) pixels in addition to effective pixels has been developed. The OB pixels are shielded from light. By using the signals output from the OB pixels in each column, the pixel values of the effective pixels in each column are corrected. Patent Document 1 discloses an image sensor that uses dummy pixels instead of OB pixels.

[0004] On the other hand, Patent Document 2 discloses a method for correcting the pixel values of effective pixels without using OB pixels or dummy pixels. According to that method, the image sensor operates in a normal output mode or a dummy black level output mode. In the normal output mode, correction pixel values are generated when no light is incident on the pixels of the image sensor. In the dummy black level output mode, correction pixel values are generated when light is incident on the pixels.

[0005] In the normal output mode, first the reset switch is turned on and the charges in the FD are reset. After the reset switch is turned off, the transfer switch is turned on and the charges accumulated in the photoelectric conversion element are transferred to the FD. After the transfer switch is turned off, a signal corresponding to the charges in the FD is output.

[0006] In dummy black level output mode, the reset switch is first turned on, resetting the charge on the floppy disk (FD). After the reset switch is turned off, the reset switch is turned on again in place of the transfer switch. After the reset switch is turned off, a signal corresponding to the charge on the FD is output.

[0007] Japanese Patent No. 6064097 Japanese Patent No. 6089716

[0008] Noise is superimposed on the signal depending on whether the reset switch or transfer switch is turned on or off. In normal output mode, the reset switch is turned on only once, but in dummy black level output mode, the reset switch is turned on twice. Therefore, the noise superimposed on the signal in dummy black level output mode may differ from the noise superimposed on the signal in normal output mode. As a result, the correction accuracy may be reduced.

[0009] The present disclosure aims to provide an imaging device, imaging system, endoscope, and imaging method capable of generating signals used to correct signals output from pixels with high precision.

[0010] According to a first aspect of this disclosure, an imaging device has two or more pixels arranged in a matrix and a column circuit. Each of the two or more pixels has a photoelectric conversion element, a floating diffusion, a transfer transistor, and a reset transistor. The transfer transistor transfers the charge generated by the photoelectric conversion element to the floating diffusion. The reset transistor resets the charge held in the floating diffusion. The column circuit is connected to pixels arranged in at least one column of the two or more pixels. The operating mode of the imaging device is switchable between a first mode and a second mode. In the first mode, the pixel resets the charge by turning the transfer transistor to the off state and the reset transistor to the on state. In the first mode, the pixel transfers the charge to the floating diffusion by turning the reset transistor to the off state and the transfer transistor to the on state. In the first mode, the column circuit outputs a first pixel signal according to the charge held in the floating diffusion. In the second mode, the pixel resets the charge by maintaining the state of the transfer transistor in the off state and performing the same control as the reset transistor control in the first mode. In the second mode, the column circuit outputs a second pixel signal according to the reset charge.

[0011] According to a second aspect of the present disclosure, in the first aspect, in the second mode, the pixel may set the state of the reset transistor to the ON state for a period of the same length as the period of time in the first mode for setting the state of the reset transistor to the ON state.

[0012] According to a third aspect of the present disclosure, the imaging system includes the imaging device and a correction circuit. The correction circuit receives the first pixel signal and the second pixel signal, and corrects the first pixel signal by using the second pixel signal.

[0013] According to a fourth aspect of the present disclosure, in a third aspect, the correction circuit may correct the first pixel signal by using the second pixel signal output from pixels arranged in at least one row of the two or more pixels.

[0014] According to a fifth aspect of the present disclosure, in a third aspect, the column circuit may output the first pixel signal and the second pixel signal in each of two or more frame periods. The correction circuit may correct the first pixel signal output in each of the two or more frame periods.

[0015] According to a sixth aspect of the present disclosure, in a third aspect, the correction circuit may correct the first pixel signal when the value of the first pixel signal or the second pixel signal is less than or equal to a threshold.

[0016] According to a seventh aspect of the present disclosure, in a sixth aspect, the imaging device may have a light-shielded second pixel. The second pixel may have the photoelectric conversion element, the floating diffusion, the transfer transistor, and the reset transistor. The column circuit is connected to the second pixel and may output a third pixel signal in accordance with the charge held in the floating diffusion of the second pixel. When the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit may correct the first pixel signal.

[0017] According to an eighth aspect of the present disclosure, in a sixth aspect, the imaging device may have a second pixel. The second pixel may have a floating diffusion and a reset transistor. The column circuit may be connected to the second pixel and output a third pixel signal in accordance with the charge held in the floating diffusion of the second pixel. When the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit may correct the first pixel signal.

[0018] According to a ninth aspect of the present disclosure, in a third aspect, the column circuit may output the first pixel signal and the second pixel signal in each of two or more frame periods. When the value of the first pixel signal or the second pixel signal is greater than a threshold, the correction circuit may correct the first pixel signal by using the second pixel signal output in a frame period prior to the frame period in which the first pixel signal was output.

[0019] According to a tenth aspect of the present disclosure, in a first embodiment, the imaging system may have a memory for storing the second pixel signal output from the imaging device during a first period as a correction signal. If the value of the first pixel signal or the second pixel signal output from the imaging device during a second period following the first period is greater than a threshold, the correction circuit may correct the first pixel signal using the correction signal.

[0020] According to an eleventh aspect of the present disclosure, the endoscope comprises a scope inserted into a living body and an imaging device. The imaging device is positioned at the tip of the scope.

[0021] A twelfth aspect of the present disclosure provides an imaging method using an imaging device having two or more pixels arranged in a matrix. Each of the two or more pixels includes a photoelectric conversion element, a floating diffusion, a transfer transistor, and a reset transistor. The transfer transistor transfers the charge generated by the photoelectric conversion element to the floating diffusion. The reset transistor resets the charge held in the floating diffusion. The operating mode of the imaging device is switchable between a first mode and a second mode. In the first mode, the imaging device resets the charge by turning off the state of the transfer transistor and turning on the state of the reset transistor. In the first mode, the imaging device transfers the charge to the floating diffusion by turning off the state of the reset transistor and turning on the state of the transfer transistor. In the first mode, the imaging device outputs a first pixel signal via a column circuit connected to at least one row of pixels among the two or more pixels, according to the charge held in the floating diffusion. The imaging device resets the charge by maintaining the state of the transfer transistor in the second mode in the off state and performing the same control as the control of the reset transistor in the first mode. In the second mode, the imaging device outputs a second pixel signal via the column circuit according to the reset charge.

[0022] According to each of the above embodiments, the imaging device, imaging system, endoscope, and imaging method can generate signals used to correct the signals output from pixels with high precision.

[0023] This is a schematic diagram showing the configuration of the endoscope system of the first embodiment. This is a block diagram showing the configuration of the camera unit, connector unit, and control unit of the endoscope system of the first embodiment. This is a block diagram showing the configuration of the image sensor of the endoscope system of the first embodiment. This is a diagram showing the pixel configuration in the image sensor of the endoscope system of the first embodiment. This is a timing chart showing the waveform of the signal in the image sensor of the endoscope system of the first embodiment. This is a diagram showing an example of the first pixel signal in the first embodiment. This is a diagram showing an example of the second pixel signal in the first embodiment. This is a diagram showing an example of the corrected first pixel signal in the first embodiment. This is a diagram showing examples of the first and second pixel signals in a modified version of the first embodiment. This is a block diagram showing the configuration of the camera unit, connector unit, and control unit of the endoscope system of the second embodiment. This is a flowchart showing an example of processing performed by the correction circuit of the endoscope system of the second embodiment.

[0024] Embodiments of this disclosure will be described with reference to the drawings. Below, an example of an endoscope system having an imaging device will be described.

[0025] (First Embodiment) Figure 1 shows the configuration of the endoscope system 1 (imaging system) of the first embodiment. The endoscope system 1 shown in Figure 1 has an endoscope insertion section 2, a transmission cable 3, an operation section 4, a connector section 5, a control unit 6, and a display device 7. The endoscope insertion section 2 (scope), the transmission cable 3, the operation section 4, and the connector section 5 constitute the endoscope 8.

[0026] The endoscope insertion unit 2 has an insertion section 2a. The insertion section 2a is part of the transmission cable 3. The insertion section 2a is inserted into the living body of the subject. The endoscope insertion unit 2 generates pixel signals by imaging the inside of the subject. The endoscope insertion unit 2 outputs the generated pixel signals to the control unit 6. The camera unit 9 shown in Figure 2 is positioned at the tip 2b of the insertion section 2a. An operation unit 4 is connected to the end of the insertion section 2a opposite to the tip 2b. The operation unit 4 receives various operations for the endoscope insertion unit 2 from the user.

[0027] The transmission cable 3 connects the camera unit 9 and the connector unit 5. The pixel signals generated by the camera unit 9 are output to the connector unit 5 via the transmission cable 3.

[0028] The connector unit 5 is connected to the transmission cable 3 and the control unit 6. The connector unit 5 performs predetermined processing on the pixel signals output from the endoscope insertion unit 2 to generate a video signal. The connector unit 5 outputs the video signal to the control unit 6.

[0029] The control unit 6 performs image processing on the video signal output from the connector unit 5. Furthermore, the control unit 6 comprehensively controls the entire endoscope system 1.

[0030] The display device 7 displays images based on the video signals processed by the control unit 6. The display device 7 also displays various information related to the endoscope system 1.

[0031] The endoscope system 1 includes a camera unit 9, a connector unit 5, and a control unit 6, as shown in Figure 2. Figure 2 shows the configuration of the camera unit 9, the connector unit 5, and the control unit 6. The camera unit 9 is located at the tip 2b of the endoscope 8. The operating unit 4 and the display device 7 are not shown in Figure 2.

[0032] The endoscope system 1 has a light source device that generates illumination light to be shone on the subject. The light source device is not shown in Figure 2.

[0033] The camera unit 9 has an image sensor 10 and a transmission circuit 11. The connector unit 5 has a receiving circuit 12 and a correction circuit 13. The control unit 6 has an image processing circuit 14.

[0034] The image sensor 10 generates a pixel signal and outputs the pixel signal to the transmission circuit 11. The transmission circuit 11 transmits the pixel signal to the connector 5. The receiving circuit 12 receives the pixel signal and outputs the pixel signal to the correction circuit 13. The correction circuit 13 corrects the pixel signal using a method described later and generates a video signal. The correction circuit 13 outputs the video signal to the control unit 6. The video processing circuit 14 applies predetermined signal processing to the video signal and outputs the video signal to the display device 7.

[0035] The correction circuit 13 or the video processing circuit 14 may be configured as a digital circuit including at least one of a processor and a logic circuit. For example, the processor is a CPU (Central Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). The correction circuit 13 or the video processing circuit 14 may include one or more processors. The correction circuit 13 or the video processing circuit 14 may include one or more logic circuits.

[0036] The computer in the connector unit 5 or the control unit 6 may read and execute a program. The program includes instructions that define the operation of the correction circuit 13 or the video processing circuit 14. In other words, the functions of the correction circuit 13 or the video processing circuit 14 may be implemented by software. The program may be transmitted from the computer holding the program to the connector unit 5 or the control unit 6 via a transmission medium or by transmission waves in the transmission medium. The "transmission medium" for transmitting the program is a medium that has the function of transmitting information. A medium that has the function of transmitting information includes networks such as the Internet and communication lines such as telephone lines. The program described above may implement some of the functions described above. Furthermore, the program described above may be a differential file (differential program). The functions described above may be implemented by a combination of a program already recorded in the computer and a differential program.

[0037] Figure 3 shows the configuration of the image sensor 10. The image sensor 10 includes an imaging unit 20, a timing generator 21, a vertical selection circuit 22, a column circuit unit 23, a horizontal selection circuit 24, and an output unit 25.

[0038] The imaging unit 20 has two or more pixels 26 arranged in a matrix. The two or more pixels 26 form an m-row and n-column array. The number of rows (m) is 2 or more, and the number of columns (n) is 2 or more. The number of rows and columns do not need to be the same. Each pixel 26 outputs a first pixel signal having a signal level and a second pixel signal having a reset level.

[0039] The timing generator 21 generates a timing signal and outputs the timing signal to the vertical selection circuit 22. The vertical selection circuit 22 selects pixels 26 arranged in the row direction in an array of two or more pixels 26. The vertical selection circuit 22 controls the operation of the selected pixels 26. The vertical selection circuit 22 outputs a control signal for each row in the array of two or more pixels 26 to control the two or more pixels 26.

[0040] The column circuit section 23 has two or more column circuits 27. Each column circuit 27 is arranged for each column in the arrangement of two or more pixels 26. Each column circuit 27 is connected to a vertical signal line 30 extending in the vertical direction, that is, the column direction. The vertical signal line 30 is arranged for each column in the arrangement of two or more pixels 26. The vertical signal line 30 is connected to the pixels 26 of each column. Each column circuit 27 is electrically connected to each pixel 26 via the vertical signal line 30. Each column circuit 27 holds the first pixel signal and the second pixel signal output from each pixel 26.

[0041] Each column circuit 27 is connected to a horizontal signal line 28 extending in the horizontal direction, that is, the row direction. Selection pulses are output from the horizontal selection circuit 24 to each column circuit 27. The column circuit 27 selected based on the selection pulse outputs the first pixel signal and the second pixel signal to the horizontal signal line 28.

[0042] One column circuit 27 may be arranged for each of two or more columns in the arrangement of two or more pixels 26, and one column circuit 27 may be used in a time-sharing manner in two or more columns. Therefore, the column circuit 27 only needs to be arranged so as to correspond to one or more columns in the arrangement of two or more pixels 26.

[0043] The horizontal signal line 28 is connected to the output section 25. The horizontal selection circuit 24 sequentially selects the column circuits 27 by sequentially outputting selection pulses to the column circuits 27. The first pixel signal and the second pixel signal output from the column circuit 27 selected by the horizontal selection circuit 24 are transferred to the output section 25. The output section 25 outputs the first pixel signal and the second pixel signal to the transmission circuit 11.

[0044] An operation mode that can be switched between the first mode and the second mode is set in the image sensor 10. When the first mode is set in the image sensor 10, each pixel 26 generates a first pixel signal. When the second mode is set in the image sensor 10, each pixel 26 generates a second pixel signal.

[0045] Figure 4 shows the configuration of each pixel 26. The pixel 26 shown in FIG. 4 includes a photoelectric conversion element 260, a transfer switch 261, a floating diffusion (FD) 262, a reset switch 263, an amplifier circuit 264, and a selection switch 265.

[0046] The photoelectric conversion element 260 is a photodiode. The photoelectric conversion element 260 performs photoelectric conversion on the light incident on the photoelectric conversion element 260 and generates charges according to the amount of the light.The transfer switch 261 transfers the charges generated by the photoelectric conversion element 260 to the FD 262. The FD 262 holds the charges transferred by the transfer switch 261. [[ID=ID=4]]

[0047] The reset switch 263 resets the voltage of the FD 262 to a voltage corresponding to the power supply voltage VDD. Thereby, the reset switch 263 resets the charges held in the FD 262. The amplifier circuit 264 generates a pixel signal by amplifying a signal based on the voltage of the FD 262. The selection switch 265 outputs the pixel signal to the vertical signal line 30. A first pixel signal having a signal level and a second pixel signal having a reset level are output from the pixel 26.

[0048] The vertical selection circuit 22 outputs a reset control signal RS, a transfer control signal TX, and a selection control signal SEL. These signals are output for each row in the array of two or more pixels 26. The reset control signal RS is input to the gate terminal of the reset switch 263. The transfer control signal TX is input to the gate terminal of the transfer switch 261. The selection control signal SEL is input to the gate terminal of the selection switch 265.

[0049] The transfer switch 261, the reset switch 263, the amplifier circuit 264, and the selection switch 265 are transistors. Each state of the transfer switch 261, the reset switch 263, and the selection switch 265 is either an on state or an off state. Each switch can switch between the on state and the off state.

[0050] The state of the reset switch 263 is controlled according to the reset control signal RS. The state of the transfer switch 261 is controlled according to the transfer control signal TX. The state of the selection switch 265 is controlled according to the selection control signal SEL.

[0051] Figure 5 shows the waveforms of the vertical synchronization signal VSYNC, the reset control signal RS, the transfer control signal TX, and the selection control signal SEL. In Figure 5, the horizontal direction represents time, and the vertical direction represents the voltage value of each signal. Figure 5 is used to explain the operation of pixel 26.

[0052] The vertical synchronization signal VSYNC is output from the timing generator 21 to the vertical selection circuit 22. The vertical synchronization signal VSYNC indicates the start and end timings of the vertical scanning period (frame period). Each of the vertical synchronization signal VSYNC, reset control signal RS, transfer control signal TX, and selection control signal SEL has either a high (H) voltage or a low (L) voltage.

[0053] Prior to timing T1, the voltages of the vertical synchronization signal VSYNC, reset control signal RS, transfer control signal TX, and selection control signal SEL are all low. The reset switch 263, transfer switch 261, and selection switch 265 are all in the off state.

[0054] At timing T1, the voltage of the vertical synchronization signal VSYNC changes from a low voltage to a high voltage. At this time, a new vertical scanning period begins. Also, the vertical scanning period before the new vertical scanning period ends. The operating mode of the image sensor 10 is set to the first mode. At timing T2, which is after timing T1, the voltage of the vertical synchronization signal VSYNC changes from a high voltage to a low voltage.

[0055] At timing T3, which is later than timing T2, the voltage of the selection control signal SEL changes from a low voltage to a high voltage. As a result, the state of the selection switch 265 changes from the off state to the on state. At this time, the amplification circuit 264 and the vertical signal line 30 are electrically connected.

[0056] At timing T4, which is later than timing T3, the voltage of the reset control signal RS changes from a low voltage to a high voltage. As a result, the state of the reset switch 263 changes from the off state to the on state. At this time, the reset switch 263 resets the charge held in FD 262. At timing T5, which is later than timing T4, the voltage of the reset control signal RS changes from a high voltage to a low voltage. As a result, the state of the reset switch 263 changes from the on state to the off state.

[0057] At timing T6, which is later than timing T5, the transfer control signal TX changes from an L voltage to an H voltage. As a result, the state of the transfer switch 261 changes from the off state to the on state. At this time, the transfer switch 261 transfers the charge generated by the photoelectric conversion element 260 to the FD 262. The FD 262 holds the charge transferred by the transfer switch 261. At timing T7, which is later than timing T6, the transfer control signal TX changes from an H voltage to an L voltage. As a result, the state of the transfer switch 261 changes from the on state to the off state.

[0058] The amplification circuit 264 generates a first pixel signal by amplifying the signal based on the voltage of the FD 262. The selection switch 265 outputs the first pixel signal to the vertical signal line 30. Since the selection control signal SEL is output to all pixels 26 in one row, each pixel 26 outputs the first pixel signal to the vertical signal line 30 located in the column corresponding to each pixel. The first pixel signal is output by the vertical signal line 30 to the column circuit 27 of each column. The column circuit 27 has a capacitor and holds the first pixel signal in the capacitor.

[0059] At timing T8, which is later than timing T7, the voltage of the selection control signal SEL changes from a high voltage to a low voltage. As a result, the state of the selection switch 265 changes from the on state to the off state.

[0060] The horizontal selection circuit 24 outputs a selection pulse to the column circuit 27 of the first column. At this time, the column circuit 27 of the first column outputs the first pixel signal to the vertical signal line 30. Subsequently, the horizontal selection circuit 24 outputs a selection pulse to the column circuit 27 of a second column different from the first column. At this time, the column circuit 27 of the second column outputs the first pixel signal to the vertical signal line 30. This operation is repeated, and the first pixel signals from all columns are sequentially output to the vertical signal line 30 and sequentially transferred to the output unit 25.

[0061] The output unit 25 outputs the first pixel signal to the transmission circuit 11. The transmission circuit 11 transmits the first pixel signal to the connector unit 5. The receiving circuit 12 receives the first pixel signal and outputs the first pixel signal to the correction circuit 13.

[0062] In the first row of pixels 26 in the imaging unit 20, the operation from timing T3 to timing T8 is performed. Subsequently, the operation from timing T3 to timing T8 is performed in the second row of pixels 26, which is different from the first row. This operation is repeated until the operation from timing T3 to timing T8 is performed in the pixels 26 of all rows.

[0063] After the operation at timing T3 to timing T8 has been performed at all pixels 26 in all rows, the operating mode of the image sensor 10 is changed from the first mode to the second mode.

[0064] After the operating mode of the image sensor 10 is changed to the second mode, at timing T9, the voltage of the selection control signal SEL changes from an L voltage to an H voltage. As a result, the state of the selection switch 265 changes from the off state to the on state. At this time, the amplification circuit 264 and the vertical signal line 30 are electrically connected.

[0065] At timing T10, which is later than timing T9, the voltage of the reset control signal RS changes from an L voltage to an H voltage. As a result, the state of the reset switch 263 changes from the off state to the on state. At this time, the reset switch 263 resets the charge held in the FD 262. When the operating mode of the image sensor 10 is the second mode, the state of the transfer switch 261 is fixed to the off state. As a result, the transfer of charge from the photoelectric conversion element 260 to the FD 262 is stopped.

[0066] When the reset switch 263 is in the ON state, the amplification circuit 264 generates a second pixel signal by amplifying the signal based on the voltage of the FD 262. The selection switch 265 outputs the second pixel signal to the vertical signal line 30. Since the selection control signal SEL is output to all pixels 26 in one row, each pixel 26 outputs the second pixel signal to the vertical signal line 30 located in the column corresponding to each pixel. The second pixel signal is output to the column circuit 27 of each column by the vertical signal line 30. The column circuit 27 holds the second pixel signal in a capacitor.

[0067] At timing T11, which is later than timing T10, the voltage of the reset control signal RS changes from a high voltage to a low voltage. As a result, the state of the reset switch 263 changes from the on state to the off state.

[0068] In the first and second modes, each pixel 26 sets the reset switch 263 to the ON state only once. In the second mode, the timing at which the state of the reset switch 263 changes to the ON state after the state of the selection switch 265 changes to the ON state is the same as the timing at which the state of the reset switch 263 changes to the ON state after the state of the selection switch 265 changes to the ON state in the first mode. In other words, the length of the period from timing T9 to timing T10 is the same as the length of the period from timing T3 to timing T4.

[0069] In the second mode, the length of time the reset switch 263 is in the ON state is the same as the length of time the reset switch 263 is in the ON state in the first mode. That is, the length of time from timing T10 to timing T11 is the same as the length of time from timing T4 to timing T5. The voltage of the reset control signal RS during the period from timing T10 to timing T11 is the same as the voltage of the reset control signal RS during the period from timing T4 to timing T5. As described above, in the second mode, each pixel 26 performs the same control as the reset switch 263 control in the first mode.

[0070] The timing of the operation of each switch in each pixel 26 is synchronized with a clock signal (not shown) and is the same as the rising or falling edge timing of the clock signal. The first timing at which the state of the reset switch 263 changes to the ON state in the first mode is the same as the second timing at which the state of the reset switch 263 changes to the ON state in the second mode. In other words, the period of the clock signal corresponding to the first timing is the same as the period of the clock signal corresponding to the second timing. Even if jitter occurs in the clock signal, the first and second timings, based on the period of the clock signal, do not change.

[0071] At timing T12, which is later than timing T11, the voltage of the selection control signal SEL changes from a high voltage to a low voltage. As a result, the state of the selection switch 265 changes from the on state to the off state.

[0072] The horizontal selection circuit 24 outputs a selection pulse to the column circuit 27 of the first column. At this time, the column circuit 27 of the first column outputs the second pixel signal to the vertical signal line 30. Subsequently, the horizontal selection circuit 24 outputs a selection pulse to the column circuit 27 of a second column different from the first column. At this time, the column circuit 27 of the second column outputs the second pixel signal to the vertical signal line 30. This operation is repeated, and the second pixel signals of all columns are sequentially output to the vertical signal line 30 and sequentially transferred to the output unit 25.

[0073] The output unit 25 outputs the second pixel signal to the transmission circuit 11. The transmission circuit 11 transmits the second pixel signal to the connector unit 5. The receiving circuit 12 receives the second pixel signal and outputs the second pixel signal to the correction circuit 13.

[0074] In the imaging unit 20, the operation from timing T9 to timing T12 is performed on the pixels 26 of the first row. Subsequently, the operation from timing T9 to timing T12 is performed on the pixels 26 of a second row, which is different from the first row. This operation is repeated until the operation from timing T9 to timing T12 is performed on the pixels 26 of all rows.

[0075] The operation shown in Figure 5 is performed during one vertical scanning period. The first pixel signal and the second pixel signal are output from each pixel 26 during one vertical scanning period. The operation shown in Figure 5 is repeated over two or more vertical scanning periods.

[0076] The operating mode of the image sensor 10 may be set to the second mode for only a portion of two or more vertical scanning periods. Alternatively, the operating mode of the image sensor 10 may be set to the first mode only for a portion of two or more vertical scanning periods.

[0077] The correction circuit 13 corrects the first pixel signal by using the second pixel signal. The process performed by the correction circuit 13 will be described below.

[0078] Figure 6 shows an example of a first pixel signal. The value of the first pixel signal output from a 15-row, 15-column pixel 26 is shown in Figure 6. Figure 7 shows an example of a second pixel signal. The value of the second pixel signal output from a 15-row, 15-column pixel 26 is shown in Figure 7. Both the first and second pixel signals contain vertical streaky noise.

[0079] The correction circuit 13 calculates the average value Col1[x] of the second pixel signal values ​​for each column according to the following equation (1). In equation (1), ImageArea2[x,y] represents the value of the second pixel signal of the pixel 26 in the xth column and yth row.

[0080]

[0081] In equation (1), n ​​represents the number of rows. When the number of rows in the imaging unit 20 is K, then n in equation (1) is 1 or greater and 1 or less than or equal to K.

[0082] The correction circuit 13 calculates the correction value ImageArea[x,y] of the first pixel signal of the pixel 26 in the xth column and yth row according to the following equation (2). In equation (2), ImageArea1[x,y] represents the value of the first pixel signal of the pixel 26 in the xth column and yth row. ImageArea[x,y] = ImageArea1[x,y] - Col1[x] (2)

[0083] As shown in equation (2), the correction circuit 13 corrects the value of the first pixel signal by subtracting the value of the second pixel signal from the value of the first pixel signal. The correction circuit 13 generates a video signal based on the corrected first pixel signal and outputs the video signal to the control unit 6.

[0084] Figure 8 shows an example of the corrected first pixel signal. The correction circuit 13 calculates the value of the first pixel signal shown in Figure 8 by subtracting the value of the second pixel signal shown in Figure 7 from the value of the first pixel signal shown in Figure 6. The vertical streaky noise in the first pixel signal shown in Figure 6 is removed.

[0085] The first pixel signal and the second pixel signal used by the correction circuit 13 do not need to be generated in the same vertical scanning period. The correction circuit 13 may use the first pixel signal generated in the first vertical scanning period and the second pixel signal generated in a second vertical scanning period that is different from the first vertical scanning period. The first vertical scanning period and the second vertical scanning period do not need to be consecutive.

[0086] Alternatively, the output unit 25 may correct the first pixel signal by using the second pixel signal instead of the correction circuit 13. Or, the image processing circuit 14 may correct the first pixel signal by using the second pixel signal instead of the correction circuit 13.

[0087] As described above, the image sensor 10 (imaging device) has two or more pixels 26 arranged in a matrix. The photoelectric conversion element 260 of each pixel 26 performs photoelectric conversion and generates an electric charge corresponding to the amount of light received. The FD 262 of each pixel 26 holds the electric charge generated by the photoelectric conversion element 260. The transfer switch 261 (transfer transistor) of each pixel 26 transfers the electric charge generated by the photoelectric conversion element 260 to the FD 262. The reset switch 263 (reset transistor) of each pixel 26 resets the electric charge held in the FD 262.

[0088] Each column circuit 27 is connected to at least one row of pixels 26 from among two or more pixels 26. An operating mode switchable between a first mode and a second mode is set in the image sensor 10. In the first mode, each pixel 26 resets the charge held in the FD 262 by setting the transfer switch 261 to the off state and the reset switch 263 to the on state. In the first mode, each pixel 26 transfers the charge held in the FD 262 to the FD 262 by setting the reset switch 263 to the off state and the transfer switch 261 to the on state. In the first mode, each column circuit 27 outputs a first pixel signal according to the charge held in the FD 262.

[0089] In the second mode, each pixel 26 maintains the state of the transfer switch 261 in the off state and resets the charge held in the FD 262 by performing the same control as the control of the reset switch 263 in the first mode. In the second mode, each column circuit 27 outputs a second pixel signal according to the reset charge.

[0090] In each aspect of the imaging method of this disclosure, the image sensor 10 performs the following processes. In the first mode, the image sensor 10 resets the charge held in the FD 262 by setting the state of the transfer switch 261 to the off state and the state of the reset switch 263 to the on state. In the first mode, the image sensor 10 transfers the charge held in the FD 262 to the FD 262 by setting the state of the reset switch 263 to the off state and the state of the transfer switch 261 to the on state. In the first mode, the image sensor 10 outputs a first pixel signal via a column circuit 27 connected to at least one row of pixels 26 among two or more pixels 26, according to the charge held in the FD 262. In the second mode, the image sensor 10 resets the charge held in the FD 262 by maintaining the state of the transfer switch 261 to the off state and performing the same control as the control of the reset switch 263 in the first mode. In the second mode, the image sensor 10 outputs a second pixel signal via the column circuit 27 according to the reset charge.

[0091] Each aspect of the present disclosure may include the following modifications: In the second mode, each pixel 26 sets the state of the reset switch 263 to the ON state for a period of the same length as the period during which the state of the reset switch 263 is set to the ON state in the first mode.

[0092] Each aspect of this disclosure may include the following modifications: The imaging system (endoscopic system 1) has a correction circuit 13 that receives a first pixel signal and a second pixel signal and corrects the first pixel signal by using the second pixel signal.

[0093] Each aspect of this disclosure may include the following modifications: The correction circuit 13 corrects the first pixel signal by using a second pixel signal output from at least one row of pixels 26 among two or more pixels 26.

[0094] Each aspect of this disclosure may include the following modifications: The column circuit 27 outputs a first pixel signal and a second pixel signal in each of two or more vertical scanning periods (frame periods). The correction circuit 13 corrects the first pixel signal output in each of the two or more vertical scanning periods.

[0095] Each aspect of this disclosure may include the following modifications. The endoscope 8 has an endoscope insertion section 2 (scope) that is inserted into the body and an image sensor 10. The image sensor 10 is located at the tip of the endoscope insertion section 2.

[0096] The operation of the reset switch 263 in the second mode is the same as the operation of the reset switch 263 in the first mode. Therefore, the correlation between the noise superimposed on the second pixel signal in the second mode and the noise superimposed on the first pixel signal in the first mode is increased. The correction circuit 13 can correct the first pixel signal with high precision by using the second pixel signal.

[0097] (Modification of the First Embodiment) A modification of the first embodiment will be described. Generally, in image sensors using a column-parallel readout method, vertical streak noise is generated due to variations in the characteristics of the column circuits. This variation depends on the power supply voltage and temperature. In the modification of the first embodiment, the correction circuit 13 corrects the first pixel signal in real time.

[0098] Figure 9 shows an example of a first pixel signal and a second pixel signal. A pixel signal (frame) generated in one vertical scan period includes a first pixel signal and a second pixel signal. The correction circuit 13 uses at least a portion of the second pixel signal of the (N-1)th frame to correct the first pixel signal of the Nth frame. That is, the correction circuit 13 uses at least a portion of the second pixel signal of the frame immediately preceding the target frame to correct the first pixel signal of the target frame. The correction circuit 13 uses the second pixel signal output from one or more rows of pixels 26. In the example shown in Figure 9, the correction circuit 13 uses the second pixel signal output from n rows of pixels 26, where n is variable.

[0099] The correction circuit 13 may use at least a portion of the second pixel signal of the Nth frame to correct the first pixel signal of the Nth frame. Alternatively, the correction circuit 13 may use at least a portion of the second pixel signal of the (N-1)th frame and at least a portion of the second pixel signal of the Nth frame to correct the first pixel signal of the Nth frame.

[0100] The correction circuit 13 can reduce noise caused by the effects of the power supply voltage VDD and temperature by correcting the first pixel signal in real time.

[0101] (Second Embodiment) A second embodiment will now be described. The operation shown in Figure 5 is repeated. During the period in which the second pixel signal is output, the photoelectric conversion element 260 generates an electric charge corresponding to the amount of light received. When strong light is incident on the imaging unit 20, the electric charge may overflow from the photoelectric conversion element 260 and flow into the FD 262. As a result, an abnormal second pixel signal may be output. The second modification provides a method for detecting when strong light is incident on the imaging unit 20 and for correcting the first pixel signal with high accuracy.

[0102] Figure 10 shows the configuration of the endoscope system 1a of the second embodiment. The endoscope system 1a shown in Figure 10 has a camera unit 9, a connector unit 5a, and a control unit 6. Configurations that are the same as those shown in Figure 2 will not be described.

[0103] The camera unit 9 shown in Figure 10 is the same as the camera unit 9 shown in Figure 2. The control unit 6 shown in Figure 10 is the same as the control unit 6 shown in Figure 2. The connector part 5 shown in Figure 2 is changed to the connector part 5a shown in Figure 10.

[0104] The connector section 5a includes a receiving circuit 12, a correction circuit 13, and a memory 15. The memory 15 stores a second pixel signal used to correct the first pixel signal when strong light is not incident on the imaging unit 20. When strong light is not incident on the imaging unit 20, the correction circuit 13 corrects the first pixel signal by using the second pixel signal received by the receiving circuit 12. When strong light is incident on the imaging unit 20, the correction circuit 13 corrects the first pixel signal by using the second pixel signal stored in the memory 15.

[0105] Figure 11 shows an example of the processing performed by the correction circuit 13. The operation of the correction circuit 13 will be explained using Figure 11.

[0106] During the first vertical scanning period, the correction circuit 13 acquires the first pixel signal received by the receiving circuit 12 (step S100).

[0107] After step S100, the correction circuit 13 acquires the second pixel signal received by the receiving circuit 12 (step S101).

[0108] After step S101, the correction circuit 13 corrects the first pixel signal using the second pixel signal (step S102). The method for correcting the first pixel signal is the same as the method in the first embodiment.

[0109] After step S102, the correction circuit 13 stores Col1[x], shown in equation (1) above, as Col_keep[x] in the memory 15. Col1[x] is the average value of the second pixel signal values ​​of each column. The memory 15 stores Col_keep[x] (step S103).

[0110] In the second vertical scanning period following the first vertical scanning period, the correction circuit 13 acquires the first pixel signal received by the receiving circuit 12 (step S104).

[0111] After step S104, the correction circuit 13 acquires the second pixel signal received by the receiving circuit 12 (step S105).

[0112] After step S105, the correction circuit 13 determines whether the value of the first pixel signal acquired in step S104 is greater than a threshold. For example, the correction circuit 13 compares the average value of the first pixel signals of all pixels 26 in the imaging unit 20 with the threshold. The correction circuit 13 may also compare the maximum value of the first pixel signals of all pixels 26 in the imaging unit 20 with the threshold. The threshold is stored in the memory 15 beforehand (step S106).

[0113] When the value of the first pixel signal is greater than the threshold, the correction circuit 13 can determine that strong light is incident on the imaging unit 20. When the value of the first pixel signal is less than or equal to the threshold, the correction circuit 13 can determine that strong light is not incident on the imaging unit 20.

[0114] If the correction circuit 13 determines in step S106 that the value of the first pixel signal is greater than a threshold, the correction circuit 13 corrects the first pixel signal by using Col_keep[x] stored in memory 15 (step S107). After step S107, step S104 is executed.

[0115] When the correction circuit 13 determines in step S106 that the value of the first pixel signal is below a threshold, the correction circuit 13 corrects the first pixel signal by using the second pixel signal acquired in step S105 (step S108). The method for correcting the first pixel signal is the same as the method in the first embodiment.

[0116] After step S108, the correction circuit 13 stores Col1[x] shown in equation (1) above as Col_keep[x] in memory 15. Memory 15 stores Col_keep[x] (step S109). After step S109, step S104 is executed.

[0117] In the above example, the correction circuit 13 determines in step S106 whether the value of the first pixel signal is greater than a threshold. The first pixel signal is generated in the imaging unit 20 before the second pixel signal is generated in the imaging unit 20. The first pixel signal is generated in the imaging unit 20 during the same vertical scanning period as the vertical scanning period during which the second pixel signal is generated in the imaging unit 20.

[0118] The correction circuit 13 may perform step S106 by using the first pixel signal generated in the imaging unit 20 after the second pixel signal has been generated in the imaging unit 20. In other words, the correction circuit 13 may perform step S106 by using the first pixel signal generated in the imaging unit 20 during a vertical scanning period following the vertical scanning period in which the second pixel signal was generated in the imaging unit 20.

[0119] The correction circuit 13 may determine in step S106 whether the value of the second pixel signal is greater than a threshold. In this example, the correction circuit 13 can determine in real time the degree of influence of strong light on the second pixel signal.

[0120] During the pre-shipment inspection period, the image sensor 10 may be driven while no light is incident on the imaging unit 20, and steps S100 to S103 may be executed in that state. The correction circuit 13 may calculate the summation average of the first pixel signals that have been generated in two or more vertical scanning periods included in the inspection period and corrected using the second pixel signals. The number of first pixel signals used in this process may be variable. It is desirable that this number be set so as to sufficiently remove random noise.

[0121] The Col_keep[x] stored in memory 15 during the inspection period is a value unique to the image sensor 10. The correction circuit 13 performs correction of the first pixel signal using Col_keep[x], thereby performing appropriate correction according to each individual image sensor 10. As a result, image quality is improved.

[0122] Each aspect of this disclosure may include the following modifications: When the value of the first pixel signal or the second pixel signal is less than or equal to a threshold, the correction circuit 13 corrects the first pixel signal.

[0123] Each aspect of the present disclosure may include the following modifications: The column circuit 27 outputs a first pixel signal and a second pixel signal in each of two or more vertical scanning periods (frame periods). When the value of the first pixel signal or the second pixel signal is greater than a threshold, the correction circuit 13 corrects the first pixel signal by using the second pixel signal output in a vertical scanning period prior to the vertical scanning period in which the first pixel signal was output.

[0124] Each aspect of this disclosure may include the following modifications: The memory 15 stores the second pixel signal output from the image sensor 10 during the first period as a correction signal. When the value of the first or second pixel signal output from the image sensor 10 during the second period, which is after the first period, is greater than a threshold, the correction circuit 13 corrects the first pixel signal by using the correction signal.

[0125] In the second embodiment, when strong light is incident on the imaging unit 20, the correction circuit 13 corrects the first pixel signal by using the second pixel signal stored in the memory 15 instead of the second pixel signal that is affected by the light. Therefore, the correction circuit 13 can correct the first pixel signal with high accuracy without being affected by strong light.

[0126] (Modification of the second embodiment) A modification of the second embodiment will be described. Two or more pixels 26 in the imaging unit 20 include the aforementioned OB pixels or dummy pixels. Alternatively, the imaging unit 20 has a region in which two or more pixels 26 are arranged and a region in which OB pixels or dummy pixels are arranged. The configuration of the OB pixels is the same as the configuration of the pixels 26 shown in Figure 4, but the OB pixels are light-shielded. The dummy pixels do not have a photoelectric conversion element 260 and a transfer switch 261, but have an FD 262, a reset switch 263, an amplification circuit 264, and a selection switch 265.

[0127] When the reset switch 263 resets the charge held in the FD 262, the OB pixel or dummy pixel outputs a third pixel signal with a reset level. The image sensor 10 may have two or more OB pixels or dummy pixels.

[0128] Each column circuit 27 holds a third pixel signal output from an OB pixel or dummy pixel. Each column circuit 27 outputs the third pixel signal to a horizontal signal line 28. The output unit 25 outputs the third pixel signal transferred via the horizontal signal line 28 to the transmission circuit 11. The transmission circuit 11 transmits the third pixel signal to the connector unit 5. The receiving circuit 12 receives the third pixel signal and outputs the third pixel signal to the correction circuit 13.

[0129] In step S106, the correction circuit 13 calculates the difference between the second pixel signal and the third pixel signal. In step S106, the correction circuit 13 compares the absolute value of this difference with a threshold. If the correction circuit 13 determines in step S106 that the absolute value of the difference is greater than the threshold, step S107 is executed. If the correction circuit 13 determines in step S106 that the absolute value of the difference is less than or equal to the threshold, step S108 is executed.

[0130] Each aspect of the present disclosure may include the following modifications. The image sensor 10 has a light-shielded OB pixel (second pixel). The OB pixel has a photoelectric conversion element 260, an FD 262, a transfer switch 261, and a reset switch 263. A column circuit 27 is connected to the OB pixel and outputs a third pixel signal according to the charge held in the FD 262 of the OB pixel. When the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit 13 corrects the first pixel signal.

[0131] Each aspect of the present disclosure may include the following modifications. The image sensor 10 has a dummy pixel (second pixel). The dummy pixel has an FD 262 and a reset switch 263. A column circuit 27 is connected to the dummy pixel and outputs a third pixel signal according to the charge held in the FD 262 of the dummy pixel. When the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit 13 corrects the first pixel signal.

[0132] In a modified version of the second embodiment, the correction circuit 13 can correct the first pixel signal with high accuracy without being affected by strong light, similar to the second embodiment.

[0133] While preferred embodiments of the Disclosure have been described above, the Disclosure is not limited to these embodiments or their variations. Additions, omissions, substitutions, and other modifications are permitted without departing from the spirit of the Disclosure. Furthermore, the Disclosure is not limited by the foregoing description, but only by the scope of the attached claims.

[0134] According to each embodiment of the present disclosure, imaging devices, imaging systems, endoscopes, and imaging methods can generate signals used to correct signals output from pixels with high precision.

[0135] 1, 1a Endoscope system 2 Endoscope insertion section 2a Insertion section 2b Tip 3 Transmission cable 4 Operation section 5, 5a Connector section 6 Control unit 7 Display device 8 Endoscope 9 Camera unit 10 Image sensor 11 Transmitting circuit 12 Receiving circuit 13 Correction circuit 14 Image processing circuit 15 Memory 20 Imaging section 21 Timing generator 22 Vertical selection circuit 23 Column circuit section 24 Horizontal selection circuit 25 Output section 26 Pixel 27 Column circuit 260 Photoelectric conversion element 261 Transfer switch 262 FD 263 Reset switch 264 Amplification circuit 265 Selection switch

Claims

1. An imaging device comprising: two or more pixels arranged in a matrix, each having: a photoelectric conversion element, a floating diffusion element, a transfer transistor for transferring the charge generated by the photoelectric conversion element to the floating diffusion element, and a reset transistor for resetting the charge held in the floating diffusion element; a column circuit connected to at least one row of pixels among the two or more pixels; the operating mode of the imaging device is switchable between a first mode and a second mode; in the first mode, the pixel resets the charge by turning off the state of the transfer transistor and turning on the state of the reset transistor; the pixel transfers the charge to the floating diffusion by turning off the state of the reset transistor and turning on the state of the transfer transistor; the column circuit outputs a first pixel signal according to the charge held in the floating diffusion; in the second mode, The image imaging device wherein the pixel resets the charge by maintaining the state of the transfer transistor in the off state and performing the same control as the control of the reset transistor in the first mode, and the column circuit outputs a second pixel signal according to the reset charge.

2. The imaging apparatus according to claim 1, wherein in the second mode, the pixel sets the state of the reset transistor to the ON state for a period of the same length as the period of setting the state of the reset transistor to the ON state in the first mode.

3. An imaging system comprising: an imaging device according to claim 1; and a correction circuit that receives the first pixel signal and the second pixel signal and corrects the first pixel signal by using the second pixel signal.

4. The imaging system according to claim 3, wherein the correction circuit corrects the first pixel signal by using the second pixel signal output from pixels arranged in at least one row of the two or more pixels.

5. The imaging system according to claim 3, wherein the column circuit outputs the first pixel signal and the second pixel signal in each of two or more frame periods, and the correction circuit corrects the first pixel signal output in each of the two or more frame periods.

6. The imaging system according to claim 3, wherein the correction circuit corrects the first pixel signal when the value of the first pixel signal or the second pixel signal is less than or equal to a threshold.

7. The imaging system according to claim 6, wherein the imaging device has a light-shielded second pixel, the second pixel having a photoelectric conversion element, a floating diffusion, a transfer transistor, and a reset transistor, the column circuit is connected to the second pixel and outputs a third pixel signal according to the charge held in the floating diffusion of the second pixel, and when the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit corrects the first pixel signal.

8. The imaging system according to claim 6, wherein the imaging device has a second pixel, the second pixel has a floating diffusion and a reset transistor, the column circuit is connected to the second pixel and outputs a third pixel signal according to the charge held in the floating diffusion of the second pixel, and when the difference between the second pixel signal and the third pixel signal is less than or equal to a threshold, the correction circuit corrects the first pixel signal.

9. The imaging system according to claim 3, wherein the column circuit outputs the first pixel signal and the second pixel signal in each of two or more frame periods, and when the value of the first pixel signal or the second pixel signal is greater than a threshold, the correction circuit corrects the first pixel signal by using the second pixel signal output in a frame period earlier than the frame period in which the first pixel signal was output.

10. The imaging system according to claim 3, which has a memory for storing the second pixel signal output from the imaging device during a first period as a correction signal, and when the value of the first pixel signal or the second pixel signal output from the imaging device during a second period later than the first period is greater than a threshold, the correction circuit corrects the first pixel signal by using the correction signal.

11. An endoscope comprising a scope inserted into a living body and an imaging device according to claim 1, wherein the imaging device is positioned at the tip of the scope.

12. An imaging method using an imaging device having two or more pixels arranged in a matrix, wherein each of the two or more pixels includes: a photoelectric conversion element; a floating diffusion; a transfer transistor for transferring charge generated by the photoelectric conversion element to the floating diffusion; and a reset transistor for resetting the charge held in the floating diffusion, wherein the operating mode of the imaging device is switchable between a first mode and a second mode, in the first mode, the charge is reset by turning the state of the transfer transistor to the off state and turning the state of the reset transistor to the on state, the charge is transferred to the floating diffusion by turning the state of the reset transistor to the off state and turning the state of the transfer transistor to the on state, and a first pixel signal is output via a column circuit connected to at least one row of pixels among the two or more pixels according to the charge held in the floating diffusion, in the second mode, the charge is reset by maintaining the state of the transfer transistor to the off state and performing the same control as the control of the reset transistor in the first mode. An imaging method that outputs a second pixel signal via the aforementioned column circuit in accordance with the reset charge.

Images