Image sensor, endoscope, imaging system, and signal output method

Abstract

An image sensor has a pixel array. The pixel array includes normal pixels that output normal pixel signals and two or more phase difference pixels that output phase difference pixel signals used for detection of image plane phase difference. The normal pixels and the two or more phase difference pixels are arranged in a matrix and share floating diffusions.

Description

Image Sensor, Endoscope, Imaging System, and Signal Output Method

[0001] The present disclosure relates to an image sensor, an endoscope, an imaging system, and a signal output method.

[0002] As disclosed in Patent Documents 1 and 2, in commercially available digital cameras and the like, AF accuracy is improved by performing AF control using phase difference pixels for detecting a phase difference (parallax) on an imaging surface. Distance information can be obtained by using this function. Therefore, application to measurement of concave-convex shapes using phase difference pixels is expected.

[0003] FIG. 11 illustrates the principle of an image plane phase difference for detecting a phase difference. Light emitted from a subject SB10 passes through a lens 500, and then passes through a microlens 510 provided on a pixel array of an image sensor. An image of the pupil of the lens 500 is projected by the microlens 510 onto a light receiving element (PD) of a phase difference pixel. Light emitted from the same position on the subject SB10 is incident on a left region or a right region of the light receiving element depending on the position where the light passes through the pupil of the lens 500. If light incident on the two regions can be separated, two images with parallax, viewed from the right half and the left half of the pupil respectively, can be reproduced. Therefore, by using one lens 500, the same situation as when stereoscopic viewing is performed with two lenses can be realized. A phase difference (parallax) can be detected based on the signals of the two regions.

[0004] Japanese Patent Application Laid-Open No. 2016-015430 U.S. Patent No. 10,412,349

[0005] On the other hand, in an endoscope system, there are cases where the frame rate is increased and the imaging sensitivity is enhanced in order to perform IR (infrared) imaging or the like. To increase the frame rate, it is necessary to reduce the number of pixels in an image. Therefore, two or more signals generated in two or more pixels may be added in a floating diffusion. This makes it possible to increase the frame rate and enhance the imaging sensitivity. However, in the prior art, no image sensor in which a floating diffusion is shared between normal pixels and phase difference pixels has been disclosed.

[0006] This disclosure aims to provide an image sensor, an endoscope, an imaging system, and a signal output method that can share floating diffusion between normal pixels and phase-difference pixels.

[0007] According to a first aspect of this disclosure, the image sensor has a pixel array. The pixel array includes normal pixels that output normal pixel signals and two or more phase difference pixels that output phase difference pixel signals used for detecting image plane phase differences. The normal pixels and the two or more phase difference pixels are arranged in a matrix and share a floating diffusion.

[0008] According to a second aspect of the present disclosure, in the first aspect, the pixel array may have a first pixel group and a second pixel group. The first pixel group includes only two or more of the normal pixels that share a first floating diffusion. The second pixel group includes one or more of the normal pixels and two or more of the phase difference pixels that share a second floating diffusion different from the first floating diffusion.

[0009] According to a third aspect of the present disclosure, in a second aspect, the pixel array may have two or more first pixel groups surrounding the second pixel group.

[0010] According to a fourth aspect of the present disclosure, in a second aspect, the two or more phase difference pixels may include a left pixel into which light passing through the left path due to pupil splitting is incident, and a right pixel into which light passing through the right path due to pupil splitting is incident.

[0011] According to a fifth aspect of the present disclosure, in a second embodiment, the second pixel group may include two or more pixel units. Each of the two or more pixel units may be formed by configuring one or more pixels among the four pixels constituting a Bayer array as phase difference pixels.

[0012] According to a sixth aspect of the present disclosure, in a fifth aspect, the two or more phase difference pixels may include a left pixel into which light passing through the left path due to pupil splitting is incident, and a right pixel into which light passing through the right path due to pupil splitting is incident. The left pixel may be included in a first pixel unit among the two or more pixel units. The right pixel may be included in a second pixel unit among the two or more pixel units. The first pixel unit and the second pixel unit may be different from each other. A green color filter may be placed in each of the left and right pixels.

[0013] According to a seventh aspect of the present disclosure, in a second aspect, the number of normal pixels in the first pixel group may be the same as the sum of the number of normal pixels in the second pixel group and the number of phase difference pixels in the second pixel group.

[0014] According to an eighth aspect of the present disclosure, in a second aspect, the pixel array may include two or more of the first pixel groups. The number of normal pixels included in each of the two or more of the first pixel groups may be the same.

[0015] According to a ninth aspect of the present disclosure, in a second embodiment, the pixel array may include two or more second pixel groups. The number of normal pixels included in each of the two or more second pixel groups may be the same. The number of phase difference pixels included in each of the two or more second pixel groups may be the same.

[0016] According to a tenth aspect of the present disclosure, in a second aspect, the first pixel group may include only four or more of the normal pixels that share the first floating diffusion and constitute a Bayer array.

[0017] According to an eleventh aspect of this disclosure, in the first aspect, only the normal pixels may be adjacent to the two or more phase difference pixels.

[0018] According to a twelfth aspect of the present disclosure, in a first aspect, the floating diffusion may generate an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from two or more phase difference pixels. The image sensor may output the added pixel signal.

[0019] According to a thirteenth aspect of the present disclosure, in a twelfth aspect, the image sensor may output the summation pixel signal in a first mode. In a second mode different from the first mode, the image sensor may output the normal pixel signal and two or more phase difference pixel signals.

[0020] According to a fourteenth aspect of the present disclosure, in a thirteenth aspect, the pixel array may have a first pixel group and a second pixel group. The first pixel group includes only two or more of the normal pixels that share a first floating diffusion. The second pixel group includes one or more of the normal pixels and two or more phase difference pixels that share a second floating diffusion different from the first floating diffusion. The first floating diffusion may generate a first sum pixel signal by adding two or more of the normal pixel signals output from the two or more of the normal pixels. The second floating diffusion may generate a second sum pixel signal by adding pixel signals output from the normal pixels and two or more phase difference pixels. In the first mode, the image sensor may output the first sum pixel signal and the second sum pixel signal.

[0021] According to a fifteenth aspect of the present disclosure, the endoscope comprises a scope inserted into a living body and an image sensor. The image sensor is located at the tip of the scope.

[0022] According to a sixteenth aspect of the present disclosure, the imaging system includes the image sensor and a signal processing circuit. The floating diffusion generates an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from the two or more phase difference pixels. The image sensor outputs the added pixel signal in a first mode. The image sensor outputs the normal pixel signal and the two or more phase difference pixel signals in a second mode different from the first mode. The signal processing circuit detects the image plane phase difference based on the two or more phase difference pixel signals.

[0023] According to a 17th aspect of the present disclosure, in a 16th aspect, the imaging system may have a correction circuit. The pixel array may have a first pixel group and a second pixel group. The first pixel group includes only two or more of the normal pixels that share a first floating diffusion. The second pixel group includes one or more of the normal pixels and two or more phase difference pixels that share a second floating diffusion different from the first floating diffusion. The first floating diffusion may generate a first added pixel signal by adding two or more of the normal pixel signals output from the two or more of the normal pixels. The second floating diffusion may generate a second added pixel signal by adding pixel signals output from the normal pixels and two or more phase difference pixels. In the first mode, the image sensor may output the first added pixel signal and the second added pixel signal. The correction circuit may correct the second added pixel signal.

[0024] According to a 18th aspect of the present disclosure, in a 17th aspect, the correction circuit may correct the second sum pixel signal generated in the second floating diffusion of the second pixel group by using the first sum pixel signal generated in the first floating diffusion of the first pixel group adjacent to the second pixel group.

[0025] According to a 19th aspect of the present disclosure, in a 17th aspect, the correction circuit may correct the second summed pixel signal generated in the second floating diffusion of the second pixel group according to a number calculated according to the number of normal pixels in the first pixel group, the number of normal pixels in the second pixel group, and the number of phase difference pixels in the second pixel group.

[0026] According to a 20th aspect of the present disclosure, in a 17th aspect, the correction circuit may correct each of the two or more phase difference pixel signals by using the normal pixel signals of one or more normal pixels arranged around each of the two or more phase difference pixels.

[0027] According to a 21st aspect of the present disclosure, in a 16th aspect, the imaging system may have a correction circuit. Only the normal pixels may be adjacent to the two or more phase difference pixels. A color filter may be placed on the normal pixels, and a color filter of the same color may be placed on each of the two or more phase difference pixels. The correction circuit may correct each of the two or more phase difference pixel signals by using the normal pixel signal of one or more of the normal pixels, which are arranged around each of the two or more phase difference pixels, and which have a color filter of the same color as the color filter placed on each of the two or more phase difference pixels.

[0028] A 22nd aspect of the present disclosure provides a signal output method using the image sensor. The floating diffusion generates an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from the two or more phase difference pixels. The image sensor outputs the added pixel signal in a first mode. In a second mode different from the first mode, the image sensor outputs the normal pixel signal and the two or more phase difference pixel signals. A signal processing circuit detects the image plane phase difference based on the two or more phase difference pixel signals.

[0029] According to each of the above embodiments, the image sensor, endoscope, imaging system, and signal output method can share floating diffusion between normal pixels and phase-difference pixels.

[0030] This is a schematic diagram showing the configuration of the endoscope system of the embodiment. This is a block diagram showing the configuration of the camera unit and control unit of the endoscope system of the embodiment. This is a diagram showing the pixel arrangement in the pixel array of the image sensor of the endoscope system of the embodiment. This is a diagram showing the circuit that outputs signals in the first pixel group and the second pixel group of the embodiment. This is a diagram showing the correspondence between the summation pixel signal output from the image sensor of the endoscope system of the embodiment and the pixel arrangement. This is a diagram showing the pixel arrangement around the phase difference pixel in the embodiment. This is a flowchart showing the signal output method in the embodiment. This is a flowchart showing the signal output method in the embodiment. This is a diagram showing the pixel arrangement in the pixel array of the image sensor of the endoscope system of the embodiment. This is a diagram showing the pixel arrangement in the pixel array of the image sensor of the endoscope system of the embodiment. This is a diagram showing the principle of detecting phase difference.

[0031] Embodiments of this disclosure will be described with reference to the drawings. Below, an example of an imaging system, specifically an endoscope system, will be described.

[0032] Figure 1 shows the configuration of an endoscope system 1 (imaging system) according to an embodiment. The endoscope system 1 shown in Figure 1 includes an endoscope insertion unit 2, a transmission cable 3, an operation unit 4, a connector unit 5, a control unit 6, and a display device 7. The endoscope insertion unit 2 (scope), the transmission cable 3, the operation unit 4, and the connector unit 5 constitute the endoscope 8.

[0033] The endoscope insertion unit 2 has an insertion section 2a. The insertion section 2a is part of the transmission cable 3. The insertion section 2a is inserted into the living body of the subject. The endoscope insertion unit 2 generates pixel signals by imaging the inside of the subject. The endoscope insertion unit 2 outputs the generated pixel signals to the control unit 6. The camera unit 9 shown in Figure 2 is positioned at the tip 2b of the insertion section 2a. An operation unit 4 is connected to the end of the insertion section 2a opposite to the tip 2b. The operation unit 4 receives various operations for the endoscope insertion unit 2 from the user.

[0034] The transmission cable 3 connects the camera unit 9 and the connector unit 5. The pixel signals generated by the camera unit 9 are output to the control unit 6 via the transmission cable 3 and the connector unit 5.

[0035] The control unit 6 processes the pixel signals output from the endoscope insertion unit 2 to generate an image signal. Furthermore, the control unit 6 controls the entire endoscope system 1.

[0036] The display device 7 displays images based on the video signals processed by the control unit 6. The display device 7 also displays various information related to the endoscope system 1.

[0037] The endoscope system 1 includes a camera unit 9 and a control unit 6, as shown in Figure 2. Figure 2 shows the configuration of the camera unit 9 and the control unit 6. The camera unit 9 is located at the tip 2b of the endoscope 8. The operating unit 4, connector unit 5, and display device 7 are not shown in Figure 2.

[0038] The endoscope system 1 has a light source device that generates illumination light to be shone on the subject. The light source device is not shown in Figure 2.

[0039] The camera unit 9 includes a lens 10, an image sensor 11, and a control circuit 12. The control unit 6 includes a correction circuit 13 and a signal processing circuit 14.

[0040] The lens 10 includes a focusing lens and other components, and forms an optical image of the subject on the image sensor 11. The image sensor 11 operates in a first mode or a second mode. The first and second modes are switchable. In the first mode, the image sensor 11 generates an added pixel signal by adding two or more pixel signals generated at two or more pixels, and outputs the added pixel signal to the control unit 6. In the second mode, the image sensor 11 outputs the pixel signal of each pixel to the control unit 6 without adding the two or more pixel signals generated at two or more pixels. The control circuit 12 performs AF (autofocus) control and adjusts the focus of the lens 10.

[0041] The correction circuit 13 corrects the summed pixel signal or the pixel signal of each pixel output from the image sensor 11 and outputs the corrected pixel signal to the signal processing circuit 14. The signal processing circuit 14 generates a video signal based on the pixel signal output from the correction circuit 13 and outputs the video signal to the display device 7. The signal processing circuit 14 also detects the image plane phase difference based on the pixel signal output from the correction circuit 13 and outputs a control signal to the control circuit 12 for executing AF control based on the detected image plane phase difference. The correction circuit 13 and the signal processing circuit 14 may be integrated.

[0042] The correction circuit 13 or signal processing circuit 14 may be configured as a digital circuit including at least one of a processor and a logic circuit. For example, the processor is a CPU (Central Processing Unit). For example, the logic circuit is at least one of an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array). The correction circuit 13 or signal processing circuit 14 may include one or more processors. The correction circuit 13 or signal processing circuit 14 may include one or more logic circuits.

[0043] A computer of the control unit 6 may read a program and execute the read program. The program includes instructions that define the operation of the correction circuit 13 or the signal processing circuit 14. That is, the functions of the correction circuit 13 or the signal processing circuit 14 may be implemented by software. The program may be transmitted from a computer storing the program to the control unit 6 via a transmission medium or by a transmission wave in a transmission medium. The "transmission medium" for transmitting a program is a medium having a function of transmitting information. The medium having the function of transmitting information includes networks (communication networks) such as the Internet and communication lines (communication wires) such as telephone lines. The above-described program may implement part of the aforementioned functions. Further, the above-described program may be a difference file (difference program). The aforementioned functions may be implemented by a combination of a difference program and a program already recorded in a computer.

[0044] The image sensor 11 has the pixel array 20 shown in FIG. 3. FIG. 3 shows a pixel arrangement in the pixel array 20.

[0045] The pixel array 20 has two or more pixels arranged in a matrix. Both the number of rows and the number of columns are 2 or more. In the example shown in FIG. 3, the pixel array 20 includes pixels arranged in 24 rows and 16 columns.

[0046] Two or more pixels arranged in two or more consecutive rows or columns constitute a pixel group. The pixel array 20 includes a first pixel group 21 and a second pixel group 22. Each of the first pixel group 21 and the second pixel group 22 includes two or more pixels. The first pixel group 21 includes only two or more normal pixels. The second pixel group 22 includes one or more normal pixels and two or more phase difference pixels.

[0047] The first pixel group 21 includes two pixel units. Each pixel unit is formed of four pixels constituting a Bayer array. A color filter is disposed on each pixel. One pixel unit constituting the minimum unit of the Bayer array includes a normal pixel on which a red (R) filter is disposed, two normal pixels on which green (Gr or Gb) filters are disposed, and a normal pixel on which a blue (B) filter is disposed.

[0048] The first pixel group 21 does not include any phase difference pixels, and includes only eight normal pixels. The number of normal pixels included in two or more different first pixel groups 21 is the same. Hereinafter, a normal pixel on which a red (R) filter is disposed is referred to as an R pixel, a normal pixel on which a green (Gr or Gb) filter is disposed is referred to as a Gr pixel or a Gb pixel, and a normal pixel on which a blue (B) filter is disposed is referred to as a B pixel.

[0049] The second pixel group 22 includes two pixel units. Each pixel unit is formed by configuring one of the four pixels constituting a Bayer array as a phase difference pixel. Specifically, one of the two pixels on which green filters are disposed is a phase difference pixel. A phase difference pixel is a left pixel on which light passing through a left-side path after pupil division is incident, or a right pixel on which light passing through a right-side path after pupil division is incident.

[0050] The second pixel group 22 includes six normal pixels and two phase difference pixels. The number of normal pixels included in two or more different second pixel groups 22 is the same. Further, the number of phase difference pixels included in two or more different second pixel groups 22 is the same. The second pixel group 22 includes a phase difference pixel on which a green (Gr) filter is disposed, and a phase difference pixel on which a green (Gb) filter is disposed. Hereinafter, a phase difference pixel on which a green (Gr) filter is disposed is referred to as a ZR pixel, and a phase difference pixel on which a green (Gb) filter is disposed is referred to as a ZL pixel. The ZR pixel is a right pixel, and the ZL pixel is a left pixel.

[0051] One second pixel group 22 is surrounded by two or more first pixel groups 21. For example, the second pixel group 22a is surrounded by eight first pixel groups 21a.

[0052] The eight normal pixels in the first pixel group 21 share one floating diffusion (FD). Additionally, the six normal pixels and two phase-difference pixels in the second pixel group 22 also share one FD.

[0053] The number of ZL pixels in the second pixel group 22 is the same as the number of ZR pixels in the second pixel group 22. The second pixel group 22 contains one ZL pixel and one ZR pixel. The second pixel group 22 may contain two or more ZL pixels and two or more ZR pixels.

[0054] Color filters of the same color are placed on the ZL and ZR pixels. In the example shown in Figure 3, a green color filter is placed on the ZL and ZR pixels.

[0055] ZL pixels and ZR pixels are not adjacent to each other. In the example shown in Figure 3, only eight normal pixels are adjacent to ZL pixels, and only eight normal pixels are adjacent to ZR pixels.

[0056] As mentioned above, the second pixel group 22 contains two pixel units. The pixel unit containing the ZL pixel is different from the pixel unit containing the ZR pixel.

[0057] Figure 4 shows a circuit that outputs signals in the first pixel group 21 and the second pixel group 22. The circuit shown in Figure 4 has eight pixel circuits 30, an FD 33, a reset transistor 34, an amplification transistor 35, and a selection transistor 36. The eight pixels included in the first pixel group 21 or the second pixel group 22 share the FD 33, the reset transistor 34, the amplification transistor 35, and the selection transistor 36.

[0058] The eight pixel circuits 30 correspond to eight pixels included in the first pixel group 21 or the second pixel group 22. Each pixel circuit 30 has a photoelectric conversion element 31 and a transfer transistor 32.

[0059] The photoelectric conversion element 31 is a photodiode. The photoelectric conversion element 31 performs photoelectric conversion on the light incident on it and generates a pixel signal corresponding to the amount of light. The transfer transistor 32 transfers the pixel signal generated by the photoelectric conversion element 31 to the FD 33. The FD 33 holds the pixel signal transferred by the transfer transistor 32.

[0060] The reset transistor 34 resets the voltage of FD33 to a voltage corresponding to the power supply voltage VDD. This resets the signal held in FD33. The amplification transistor 35 amplifies the pixel signal based on the voltage of FD33. The selection transistor 36 outputs the pixel signal amplified by the amplification transistor 35 to the vertical signal line 37.

[0061] The state of each of the transfer transistor 32, reset transistor 34, and selection transistor 36 can be either on or off. Each transistor can switch between the on and off states.

[0062] The state of the transfer transistor 32 is controlled according to the transfer control signal TX1 or TX2. The state of the reset transistor 34 is controlled according to the reset control signal RST. The state of the selection transistor 36 is controlled according to the selection control signal SEL.

[0063] Transfer control signals TX1, TX2, RST, and SEL are supplied for each row. Transfer control signal TX1 is supplied to the transfer transistor 32 of the pixel circuit 30 located in the first column. Transfer control signal TX2 is supplied to the transfer transistor 32 of the pixel circuit 30 located in a second column different from the first column.

[0064] When the first mode is set for the image sensor 11, the signal held in the FD 33 is reset by the reset transistor 34, and then the eight transfer transistors 32 of the eight pixel circuits 30 are simultaneously turned ON. At this time, the pixel signals accumulated in the eight photoelectric conversion elements 31 are simultaneously transferred to the FD 33 and added together. The FD 33 holds the added pixel signals. The pixel signals held in the FD 33 are amplified by the amplification transistor 35 and output to the vertical signal line 37 via the selection transistor 36.

[0065] When the second mode is set for the image sensor 11, the signal held in the FD 33 is reset by the reset transistor 34, and then, for example, the state of the transfer transistor 32 of the pixel circuit 30 located in the first row and first column is turned ON. At this time, the pixel signal stored in the photoelectric conversion element 31 is transferred to the FD 33 and held in the FD 33. The pixel signal held in the FD 33 is amplified by the amplification transistor 35 and output to the vertical signal line 37 via the selection transistor 36.

[0066] After the signal held in FD33 is reset by the reset transistor 34, the state of the transfer transistor 32 of the pixel circuit 30 located in the first row and second column is turned ON. At this time, the pixel signal stored in the photoelectric conversion element 31 is transferred to FD33 and held in FD33. The pixel signal held in FD33 is amplified by the amplification transistor 35 and output to the vertical signal line 37 via the selection transistor 36.

[0067] The above operation is repeated, and the pixel signals accumulated in each photoelectric conversion element 31 are sequentially transferred to the FD 33 and output to the vertical signal line 37.

[0068] When the first mode is set to the image sensor 11, the image sensor 11 generates a first summed pixel signal by adding the eight pixel signals output from the eight normal pixels included in the first pixel group 21. Also, when the first mode is set to the image sensor 11, the image sensor 11 generates a second summed pixel signal by adding the six pixel signals output from the six normal pixels included in the second pixel group 22 and the two pixel signals output from the two phase-difference pixels included in the second pixel group 22. The image sensor 11 outputs the first summed pixel signal and the second summed pixel signal to the correction circuit 13.

[0069] Figure 5 shows the correspondence between the summation pixel signal output from the image sensor 11 and the pixel array when the first mode is set to the image sensor 11. The first summation pixel signal is generated based on eight pixel signals output from eight normal pixels included in the first pixel group 21. The eight normal pixels include two R pixels, two B pixels, two Gr pixels, and two Gb pixels.

[0070] The second summation pixel signal is generated based on six pixel signals output from six normal pixels included in the second pixel group 22 and two pixel signals output from two phase-difference pixels included in the second pixel group 22. The six normal pixels include two R pixels, two B pixels, one Gr pixel, and one Gb pixel. The two phase-difference pixels include one ZL pixel and one ZR pixel.

[0071] The right or left portion of the aperture of a phase-difference pixel is shielded from light. For example, the area of ​​the aperture of a phase-difference pixel is half the area of ​​the aperture of a normal pixel. Therefore, the sensitivity of a phase-difference pixel is lower than that of a normal pixel. The same number of pixels are included in the first pixel group 21 and the second pixel group 22, but the amount of the second summed pixel signal generated in the second pixel group 22 is smaller than the amount of the first summed pixel signal generated in the first pixel group 21 adjacent to the second pixel group 22. In order to use the second summed pixel signal for image generation, the second summed pixel signal needs to be corrected.

[0072] When the first mode is set for the image sensor 11, the correction circuit 13 corrects the second summation pixel signal. Two methods for correcting the summation pixel signal are described below.

[0073] First, the first method will be explained. The correction circuit 13 corrects the second summing pixel signal generated in the second pixel group 22 by using two or more first summing pixel signals generated in two or more first pixel groups 21 adjacent to the second pixel group 22. For example, the correction circuit 13 corrects the second summing pixel signal generated in the second pixel group 22a by using two or more first summing pixel signals generated in two or more first pixel groups 21a shown in Figure 3. For example, the correction circuit 13 calculates the average of the values ​​of two or more first summing pixel signals and replaces the value of the second summing pixel signal with that average.

[0074] The correction circuit 13 may correct the second summing pixel signal by using four or eight first summing pixel signals generated in four or eight first pixel groups 21. The correction circuit 13 may also replace the second summing pixel signal with one first summing pixel signal generated in one first pixel group 21.

[0075] Only two or more first pixel groups 21 are adjacent to the second pixel group 22. The correction circuit 13 can correct the second summing pixel signal by using only two or more first summing pixel signals. The number of first summing pixel signals that can be used to correct the second summing pixel signal can be increased, improving the accuracy of the correction.

[0076] Next, the second method will be explained. The amount of the first summed pixel signal generated in the first pixel group 21 is proportional to the number of normal pixels N1 included in the first pixel group 21. On the other hand, the amount of the second summed pixel signal generated in the second pixel group 22 is proportional to N2 shown in the following equation (1): N2 = Nn + (Nl + Nr) / 2 (1)

[0077] In equation (1), Nn represents the number of normal pixels in the second pixel group 22. In equation (1), Nl represents the number of ZL pixels in the second pixel group 22. In equation (1), Nr represents the number of ZR pixels in the second pixel group 22.

[0078] The position of the light-shielding portion formed in the aperture of the phase-difference pixel differs between the ZL pixel and the ZR pixel. Therefore, the amount of light incident on the aperture of the ZL pixel (first quantity) and the amount of light incident on the aperture of the ZR pixel (second quantity) differ depending on the angle of incidence when the light enters the lens 10. However, the sum of the first and second quantities is approximately constant regardless of the angle of incidence. Therefore, (Nl + Nr) / 2 in equation (1) is approximately constant.

[0079] The correction circuit 13 corrects the second summing pixel signal by multiplying it by N1 / N2. The amount of the first summing pixel signal generated in the first pixel group 21 is the sum of the eight pixel signals generated in the eight normal pixels. N2 in equation (1) is 7. The amount of the second summing pixel signal generated in the second pixel group 22 is approximately the same as the sum of the seven pixel signals generated in the seven pixels.

[0080] In the above example, the correction circuit 13 corrects the second summing pixel signal by multiplying it by 8 / 7. The number of pixels in the second pixel group 22 is the same as the number of pixels in the first pixel group 21, and the number of phase difference pixel pairs in the second pixel group 22 is the same. Therefore, N1 and N2 are fixed values. The correction circuit 13 can easily perform the correction process.

[0081] If the second pixel group 22 includes only one of the ZL pixels or the ZR pixels, the amount of the second summed pixel signal generated in the second pixel group 22 including the ZL pixels may differ from the amount of the second summed pixel signal generated in the second pixel group 22 including the ZR pixels. Therefore, the correction process performed by the correction circuit 13 may become more complex.

[0082] When the second pixel group 22 includes ZL pixels and ZR pixels, the ratio of the amount of the second summation pixel signal to the amount of the first summation pixel signal generated in the first pixel group 21 adjacent to the second pixel group 22 is approximately constant. Therefore, a simple correction process can be implemented in the second method.

[0083] The correction circuit 13 outputs the first summed pixel signal to the signal processing circuit 14 and outputs the corrected second summed pixel signal to the signal processing circuit 14. The signal processing circuit 14 generates a video signal that constitutes an image including the first summed pixel signal and the second summed pixel signal.

[0084] When the second mode is set for the image sensor 11, the image sensor 11 outputs eight pixel signals generated in each of the eight normal pixels included in the first pixel group 21. The image sensor 11 also outputs eight pixel signals generated in each of the six normal pixels and two phase-difference pixels included in the second pixel group 22.

[0085] As described above, the sensitivity of phase-difference pixels is lower than that of normal pixels. In order to use the pixel signals generated by phase-difference pixels for image generation, those pixel signals need to be corrected.

[0086] When the second mode is set for the image sensor 11, the correction circuit 13 corrects the pixel signal generated in each of the two phase difference pixels by using the pixel signals generated in one or more normal pixels arranged around each phase difference pixel.

[0087] Figure 6 shows the pixel arrangement around the phase difference pixel. A green (Gb) color filter is placed on the ZL pixel PIX1. Only two or more normal pixels are adjacent to the ZL pixel PIX1. The correction circuit 13 corrects the pixel signal generated in the ZL pixel PIX1 by using one or more pixel signals generated in one or more normal pixels on which the green color filter is placed.

[0088] When focusing only on pixels with green color filters, ZL pixel PIX1 is adjacent to four normal pixels PIX2 with green (Gr) color filters and four normal pixels PIX3 with green (Gb) color filters. The correction circuit 13 corrects the pixel signal generated in ZL pixel PIX1 by using the four pixel signals generated in the four normal pixels PIX2 and the four pixel signals generated in the four normal pixels PIX3. For example, the correction circuit 13 calculates the average of the four pixel signals generated in the four normal pixels PIX2 and the four pixel signals generated in the four normal pixels PIX3, and replaces the value of the pixel signal generated in ZL pixel PIX1 with that average.

[0089] When the second mode is set for the image sensor 11, the signal processing circuit 14 generates a video signal that constitutes an image, which includes the pixel signals generated in the normal pixels and the pixel signals corrected as described above. The signal processing circuit 14 also detects the image plane phase difference based on the pixel signals generated in the ZL pixels and the pixel signals generated in the ZR pixels. The signal processing circuit 14 outputs a control signal to the control circuit 12 for executing AF control based on the detected image plane phase difference.

[0090] Figure 7 shows the signal output method in the first mode. The signal output method in the first mode will be explained with reference to Figure 7.

[0091] The FD33 included in the first pixel group 21 generates a first summed pixel signal by adding two or more pixel signals output from two or more pixel circuits 30. The FD33 included in the second pixel group 22 generates a second summed pixel signal by adding three or more pixel signals output from three or more pixel circuits 30. The three or more pixel signals used to generate the second summed pixel signal include one or more pixel signals output from one or more normal pixel circuits 30 and two or more pixel signals output from two or more phase-difference pixel circuits 30 (step S100).

[0092] The image sensor 11 outputs a first summing pixel signal and a second summing pixel signal (step S101).

[0093] The correction circuit 13 corrects the second summing pixel signal. In the first method described above, the correction circuit 13 corrects the second summing pixel signal generated at the FD 33 of the second pixel group 22 by using the first summing pixel signal generated at the FD 33 of the first pixel group 21 adjacent to the second pixel group 22. Alternatively, in the second method described above, the correction circuit 13 corrects the second summing pixel signal generated at the FD 33 of the second pixel group 22 according to a number calculated according to the number of normal pixels in the first pixel group 21, the number of normal pixels in the second pixel group 22, and the number of phase difference pixels in the second pixel group 22 (step S102).

[0094] The signal processing circuit 14 generates a video signal that constitutes an image including a first summing pixel signal and a second summing pixel signal, and outputs the video signal to the display device 7 (step S103).

[0095] Figure 8 shows the signal output method in the second mode. The signal output method in the second mode will be explained with reference to Figure 8.

[0096] The image sensor 11 outputs pixel signals generated in each of the two or more pixel circuits 30 in the first pixel group 21. The image sensor 11 also outputs pixel signals generated in each of the pixel circuits 30 of one or more normal pixels and two or more phase-difference pixels in the second pixel group 22 (step S200).

[0097] The correction circuit 13 corrects each of the two or more pixel signals output from two or more phase difference pixels by using the pixel signals generated in one or more normal pixels arranged around each phase difference pixel (step S201).

[0098] The signal processing circuit 14 detects the image plane phase difference based on the two pixel signals generated in the pair of left and right phase difference pixels. Based on the detected image plane phase difference, the signal processing circuit 14 outputs a control signal to the control circuit 12 for executing AF control (step S202).

[0099] The signal processing circuit 14 generates a video signal that constitutes an image including the pixel signal generated in a normal pixel and the corrected pixel signal, and outputs the video signal to the display device 7 (step S203).

[0100] The order in which steps S202 and S203 are executed is not limited to the order described above. Step S202 may be executed after step S203.

[0101] Figure 9 shows another example of the pixel arrangement in the pixel array 20. The explanation of the same parts as shown in Figure 3 is omitted.

[0102] The second pixel group 22b shown in Figure 9 differs from the second pixel group 22 shown in Figure 3. The second pixel group 22b contains two pixel units. Each pixel unit is formed by making two of the four pixels constituting the Bayer array into phase-difference pixels. Specifically, the two pixels on which the green filter is placed are phase-difference pixels.

[0103] The second pixel group 22b includes four normal pixels and four phase-difference pixels. The second pixel group 22b includes two phase-difference pixels with green (Gr) filters and two phase-difference pixels with green (Gb) filters.

[0104] Figure 10 shows another example of the pixel arrangement in the pixel array 20. The explanation of the same parts as shown in Figure 3 is omitted.

[0105] In the pixel array 20 shown in Figure 3, the second pixel group 22 is located in specific column positions. For example, the second pixel group 22 is located in columns 0 and 1, while only the first pixel group 21 is located in columns 2 and 3.

[0106] In the pixel array 20 shown in Figure 10, the second pixel group 22 is arranged at an arbitrary column position. For example, the second pixel group 22 is arranged in two columns, column numbers 0 and 1, and the other second pixel group 22 is arranged in two columns, column numbers 2 and 3.

[0107] As described above, the image sensor 11 has a pixel array 20. The pixel array 20 includes normal pixels that output a normal pixel signal and two or more phase difference pixels that output a phase difference pixel signal used for detecting the image plane phase difference. The normal pixels and the two or more phase difference pixels are arranged in a matrix and share an FD 33. For example, the six normal pixels and two phase difference pixels of the second pixel group 22 share an FD 33.

[0108] Each embodiment of the Endoscope 8 of this Disclosure comprises an endoscope insertion section 2 (scope) that is inserted into the body and an image sensor 11. The image sensor 11 is located at the tip of the endoscope insertion section 2.

[0109] Each embodiment of the imaging system (endoscopic system 1) of this disclosure includes an image sensor 11 and a signal processing circuit 14. The FD 33 generates an added pixel signal by adding a normal pixel signal and two or more phase difference pixel signals output from two or more phase difference pixels. The image sensor 11 outputs the added pixel signal in a first mode. In a second mode different from the first mode, the image sensor 11 outputs a normal pixel signal and two or more phase difference pixel signals. The signal processing circuit 14 detects the image plane phase difference based on the two or more phase difference pixel signals.

[0110] In the signal output method of each aspect of the present disclosure, the FD33 generates an added pixel signal by adding a normal pixel signal and two or more phase difference pixel signals output from two or more phase difference pixels (step S100). The image sensor 11 outputs the added pixel signal in the first mode (step S101). The image sensor 11 outputs a normal pixel signal and two or more phase difference pixel signals in a second mode different from the first mode (step S200). The signal processing circuit 14 detects the image plane phase difference based on the two or more phase difference pixel signals (step S202).

[0111] Each aspect of the present disclosure may include the following modifications: The pixel array 20 has a first pixel group 21 and a second pixel group 22. The first pixel group 21 includes only two or more normal pixels that share an FD 33 (first floating diffusion). The second pixel group 22 includes one or more normal pixels and two or more phase-difference pixels that share an FD 33 (second floating diffusion).

[0112] Each aspect of the present disclosure may include the following modifications: The pixel array 20 has two or more first pixel groups 21 surrounding a second pixel group 22.

[0113] Each aspect of this disclosure may include the following modifications: Two or more phase-difference pixels include a left pixel (ZL pixel) into which light passing through the left path due to pupil splitting is incident, and a right pixel (ZR pixel) into which light passing through the right path due to pupil splitting is incident.

[0114] Each aspect of the present disclosure may include the following modifications: The second pixel group 22 includes two or more pixel units. Each of the two or more pixel units is formed by configuring one or more pixels among the four pixels constituting the Bayer array with phase difference pixels.

[0115] Each aspect of this disclosure may include the following modifications: Two or more phase-difference pixels include a left pixel into which light passing through the left path is incident by pupil splitting, and a right pixel into which light passing through the right path is incident by pupil splitting. The left pixel is included in a first pixel unit of two or more pixel units. The right pixel is included in a second pixel unit of two or more pixel units. The first and second pixel units are distinct from each other. A green color filter is placed in each of the left and right pixels.

[0116] Each aspect of this disclosure may include the following modifications: The number of normal pixels in the first pixel group 21 is the same as the sum of the number of normal pixels in the second pixel group 22 and the number of phase difference pixels in the second pixel group 22.

[0117] Each aspect of this disclosure may include the following modifications: The pixel array 20 includes two or more first pixel groups 21. Each of the two or more first pixel groups 21 contains the same number of normal pixels.

[0118] Each aspect of this disclosure may include the following modifications: The pixel array 20 includes two or more second pixel groups 22. Each of the two or more second pixel groups 22 contains the same number of normal pixels. Each of the two or more second pixel groups 22 contains the same number of phase difference pixels.

[0119] Each aspect of this disclosure may include the following modifications: The first pixel group 21 includes only four or more ordinary pixels that share an FD 33 (first floating diffusion) and constitute a Bayer array.

[0120] Each aspect of this disclosure may include the following modifications: Only normal pixels are adjacent to two or more phase-difference pixels.

[0121] Each aspect of this disclosure may include the following modifications. The FD33 generates an added pixel signal by adding a normal pixel signal and two or more phase difference pixel signals output from two or more phase difference pixels. The image sensor 11 outputs the added pixel signal.

[0122] Each aspect of this disclosure may include the following modifications: In a first mode, the image sensor 11 outputs an added pixel signal. In a second mode different from the first mode, the image sensor 11 outputs a normal pixel signal and two or more phase-difference pixel signals.

[0123] Each aspect of this disclosure may include the following modifications: FD33 (first floating diffusion) generates a first added pixel signal by adding two or more normal pixel signals output from two or more normal pixels of the first pixel group 21. FD33 (second floating diffusion) generates a second added pixel signal by adding pixel signals output from normal pixels and two or more phase-difference pixels of the second pixel group 22. In the first mode, the image sensor 11 outputs the first added pixel signal and the second added pixel signal.

[0124] Each aspect of this disclosure may include the following modifications: The correction circuit 13 corrects the second summation pixel signal.

[0125] Each aspect of this disclosure may include the following modifications: The correction circuit 13 corrects the second summation pixel signal generated in the FD33 (second floating diffusion) of the second pixel group 22 by using the first summation pixel signal generated in the FD33 (first floating diffusion) of the first pixel group 21 adjacent to the second pixel group 22.

[0126] Each aspect of this disclosure may include the following modifications. The correction circuit 13 corrects the second summed pixel signal generated in the FD 33 (second floating diffusion) of the second pixel group 22 according to a number calculated according to the number of normal pixels in the first pixel group 21, the number of normal pixels in the second pixel group 22, and the number of phase difference pixels in the second pixel group 22.

[0127] Each aspect of this disclosure may include the following modifications: The correction circuit 13 corrects each of two or more phase difference pixel signals by using the normal pixel signals of one or more normal pixels arranged around each of the two or more phase difference pixels.

[0128] Each aspect of this disclosure may include the following modifications: Only normal pixels are adjacent to two or more phase difference pixels. A color filter is placed on a normal pixel adjacent to a phase difference pixel, and a color filter of the same color is placed on each of the two or more phase difference pixels. The correction circuit 13 corrects each of the two or more phase difference pixel signals by using the normal pixel signal of one or more normal pixels, which are placed around each of the two or more phase difference pixels, and which have a color filter of the same color as the color filter placed on each of the two or more phase difference pixels.

[0129] As described above, the image sensor 11 can share the FD 33 between normal pixels and phase-difference pixels.

[0130] The correction circuit 13 can correct the second summing pixel signal by using only two or more first summing pixel signals generated in two or more first pixel groups 21 adjacent to the second pixel group 22. Therefore, the accuracy of the correction is improved.

[0131] As described above, the correction circuit 13 corrects the second summing pixel signal by multiplying the second summing pixel signal by N1 / N2. N1 represents the number of normal pixels included in the first pixel group 21. N2 is calculated based on equation (1) described above. The number of pixels included in the second pixel group 22 is the same as the number of pixels included in the first pixel group 21, and the number of phase difference pixel pairs included in the second pixel group 22 is the same. Since N1 and N2 are fixed values, the correction circuit 13 can easily perform the correction process.

[0132] While preferred embodiments of the Disclosure have been described above, the Disclosure is not limited to these embodiments or their variations. Additions, omissions, substitutions, and other modifications are permitted without departing from the spirit of the Disclosure. Furthermore, the Disclosure is not limited by the foregoing description, but only by the scope of the attached claims.

[0133] According to each embodiment of the present disclosure, the image sensor, endoscope, imaging system, and signal output method can share floating diffusion between normal pixels and phase-difference pixels.

[0134] 1 Endoscope system 2 Endoscope insertion section 2a Insertion section 2b Tip 3 Transmission cable 4 Operation section 5 Connector section 6 Control unit 7 Display device 8 Endoscope 9 Camera unit 10 Lens 11 Image sensor 12 Control circuit 13 Correction circuit 14 Signal processing circuit 20 Pixel array 21 First pixel group 22 Second pixel group 30 Pixel circuit 33 FD 34 Reset transistor 35 Amplifier transistor 36 Selection transistor

Claims

1. An image sensor having a pixel array including a normal pixel that outputs a normal pixel signal and two or more phase difference pixels that output a phase difference pixel signal used for detecting the image plane phase difference, wherein the normal pixel and the two or more phase difference pixels are arranged in a matrix and share floating diffusion.

2. The image sensor according to claim 1, wherein the pixel array comprises: a first pixel group comprising only two or more normal pixels that share a first floating diffusion; and a second pixel group comprising one or more normal pixels and two or more phase difference pixels that share a second floating diffusion different from the first floating diffusion.

3. The image sensor according to claim 2, wherein the pixel array has two or more first pixel groups surrounding the second pixel group.

4. The image sensor according to claim 2, wherein the two or more phase difference pixels include a left pixel into which light passing through the left path due to pupil division is incident, and a right pixel into which light passing through the right path due to pupil division is incident.

5. The image sensor according to claim 2, wherein the second pixel group includes two or more pixel units, and each of the two or more pixel units is formed by configuring one or more pixels among the four pixels constituting the Bayer array as phase difference pixels.

6. The image sensor according to claim 5, wherein the two or more phase difference pixels include a left pixel into which light passing through the left path due to pupil division is incident, and a right pixel into which light passing through the right path due to pupil division is incident, the left pixel is included in a first pixel unit among the two or more pixel units, the right pixel is included in a second pixel unit among the two or more pixel units, the first pixel unit and the second pixel unit are different from each other, and a green color filter is disposed in each of the left pixel and the right pixel.

7. The image sensor according to claim 2, wherein the number of normal pixels in the first pixel group is the same as the sum of the number of normal pixels in the second pixel group and the number of phase difference pixels in the second pixel group.

8. The image sensor according to claim 2, wherein the pixel array includes two or more first pixel groups, and the number of normal pixels included in each of the two or more first pixel groups is the same.

9. The image sensor according to claim 2, wherein the pixel array includes two or more second pixel groups, the number of normal pixels included in each of the two or more second pixel groups is the same, and the number of phase difference pixels included in each of the two or more second pixel groups is the same.

10. The image sensor according to claim 2, wherein the first pixel group includes only four or more of the normal pixels that share the first floating diffusion and constitute a Bayer array.

11. The image sensor according to claim 1, wherein only the normal pixels are adjacent to the two or more phase difference pixels.

12. The image sensor according to claim 1, wherein the floating diffusion generates an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from the two or more phase difference pixels, and the image sensor outputs the added pixel signal.

13. The image sensor according to claim 12, wherein in a first mode, it outputs the summation pixel signal, and in a second mode different from the first mode, it outputs the normal pixel signal and two or more of the phase difference pixel signals.

14. The image sensor according to claim 13, wherein the pixel array comprises: a first pixel group comprising only two or more normal pixels that share a first floating diffusion; and a second pixel group comprising one or more normal pixels and two or more phase difference pixels that share a second floating diffusion different from the first floating diffusion, wherein the first floating diffusion generates a first sum pixel signal by adding two or more normal pixel signals output from the two or more normal pixels; and the second floating diffusion generates a second sum pixel signal by adding pixel signals output from the normal pixels and two or more phase difference pixels; and the image sensor outputs the first sum pixel signal and the second sum pixel signal in the first mode.

15. An endoscope comprising a scope inserted into a living body and an image sensor according to claim 1, wherein the image sensor is positioned at the tip of the scope.

16. An imaging system comprising: an image sensor according to claim 1; a signal processing circuit; wherein the floating diffusion generates an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from the two or more phase difference pixels; the image sensor outputs the added pixel signal in a first mode; the image sensor outputs the normal pixel signal and the two or more phase difference pixel signals in a second mode different from the first mode; and the signal processing circuit detects the image plane phase difference based on the two or more phase difference pixel signals.

17. The imaging system according to claim 16, comprising a correction circuit, wherein the pixel array comprises: a first pixel group comprising only two or more of the normal pixels that share a first floating diffusion; and a second pixel group comprising one or more of the normal pixels and two or more phase difference pixels that share a second floating diffusion different from the first floating diffusion, wherein the first floating diffusion generates a first added pixel signal by adding two or more of the normal pixel signals output from the two or more of the normal pixels; the second floating diffusion generates a second added pixel signal by adding the pixel signals output from the normal pixels and the two or more phase difference pixels; the image sensor outputs the first added pixel signal and the second added pixel signal in the first mode; and the correction circuit corrects the second added pixel signal.

18. The imaging system according to claim 17, wherein the correction circuit corrects the second summed pixel signal generated in the second floating diffusion of the second pixel group by using the first summed pixel signal generated in the first floating diffusion of the first pixel group adjacent to the second pixel group.

19. The imaging system according to claim 17, wherein the correction circuit corrects the second summed pixel signal generated in the second floating diffusion of the second pixel group according to a number calculated according to the number of normal pixels in the first pixel group, the number of normal pixels in the second pixel group, and the number of phase difference pixels in the second pixel group.

20. The imaging system according to claim 17, wherein the correction circuit corrects each of the two or more phase difference pixel signals by using the normal pixel signals of one or more normal pixels arranged around each of the two or more phase difference pixels.

21. The imaging system according to claim 16, comprising a correction circuit, wherein only the normal pixels are adjacent to the two or more phase difference pixels, a color filter is placed on the normal pixels, and a color filter of the same color is placed on each of the two or more phase difference pixels, wherein the correction circuit corrects each of the two or more phase difference pixel signals by using the normal pixel signal of one or more of the normal pixels, which are arranged around each of the two or more phase difference pixels, and which have a color filter of the same color as the color filter placed on each of the two or more phase difference pixels.

22. A signal output method using the image sensor described in claim 1, wherein the floating diffusion generates an added pixel signal by adding the normal pixel signal and two or more phase difference pixel signals output from the two or more phase difference pixels, the image sensor outputs the added pixel signal in a first mode, the image sensor outputs the normal pixel signal and the two or more phase difference pixel signals in a second mode different from the first mode, and the signal processing circuit detects the image plane phase difference based on the two or more phase difference pixel signals.

Images