Image sensor and imaging device

Abstract

The present invention provides an image sensor and imaging device that improve the detection accuracy of image plane phase difference, acquire Bayer array image data, and improve the efficiency of light utilization. [Solution] An image sensor is provided comprising: two first pixels arranged continuously in a first direction and detecting light of a first color; two second pixels arranged continuously in a second direction intersecting the first direction, adjacent to the two first pixels, and detecting light of a second color; a plurality of first light-receiving regions arranged in the first pixels and divided in the first direction for receiving light of the first color; and a plurality of second light-receiving regions arranged in the second pixels and divided in the second direction for receiving light of the second color.

Description

Technical Field

[0001] The present invention relates to an imaging device and an imaging apparatus.

Background Art

[0002] There is known an imaging apparatus that performs focus detection by a pupil division type phase difference method based on output signals from a plurality of pixels dedicated to focus detection arranged in a part of an imaging device (see, for example, Patent Document 1). Patent Document 1: Japanese Patent Application Laid-Open No. 2011-77770

Summary of the Invention

Problems to be Solved by the Invention

[0003] Conventionally, since the pixels for focus detection are discretely arranged, the focus detection accuracy is lower than when the pixels for focus detection are continuously arranged. On the other hand, when the pixels for focus detection corresponding to a specific color of color filter are continuously arranged, the pixel arrangement becomes different from a predetermined arrangement such as a Bayer array. In the prior art, when trying to convert to a predetermined arrangement such as a Bayer array by interpolation processing or the like, the calculation becomes complicated.

Means for Solving the Problems

[0004] In a first aspect of the present invention, there is provided an imaging device including: a plurality of light receiving regions that detect light of a first color arranged in a first direction, and a plurality of first light receiving portions arranged in the first direction; and a plurality of light receiving regions that detect light of a second color arranged in a second direction different from the first direction, and a plurality of second light receiving portions arranged in the first direction and arranged at positions different from the plurality of first light receiving portions in the first direction.

[0005] In a second aspect of the present invention, an image sensor is provided comprising: a plurality of first light-receiving units arranged in a first direction, each having a plurality of light-receiving regions for detecting light of a first color arranged in a first direction; and a plurality of second light-receiving units arranged in a second direction different from the first direction, each having a plurality of light-receiving regions for detecting light of a second color arranged in a second direction different from the first direction, each being arranged in a second direction and intersecting with the plurality of first light-receiving units between the two first light-receiving units.

[0006] In a third aspect of the present invention, an image sensor is provided having a first light receiving unit having a plurality of photoelectric conversion units for detecting light of a first color; a second light receiving unit arranged in a first direction from the first light receiving unit and having a plurality of photoelectric conversion units for detecting light of a second color; a third light receiving unit arranged in a second direction from the first light receiving unit and having a plurality of photoelectric conversion units for detecting light of a second color; and a fourth light receiving unit arranged in a second direction from the second light receiving unit and having a plurality of photoelectric conversion units for detecting light of a first color, wherein the plurality of photoelectric conversion units of the first light receiving unit and the plurality of photoelectric conversion units of the fourth light receiving unit are arranged in the direction from the first light receiving unit to the fourth light receiving unit, and the plurality of photoelectric conversion units of the second light receiving unit and the plurality of photoelectric conversion units of the third light receiving unit are arranged in the direction from the second light receiving unit to the third light receiving unit.

[0007] The present invention may also include the following embodiments. [Item 1] Two first pixels are arranged in a continuous manner in a first direction and detect light of a first color, Two second pixels are arranged in a continuous direction intersecting the first direction, adjacent to the two first pixels, and detect light of a second color. A plurality of first light-receiving regions arranged in the first pixel and divided in the first direction for receiving light of the first color, A plurality of second light-receiving regions arranged in the second pixel and divided in the second direction for receiving light of the second color, An image sensor equipped with the following features. [Item 2] The first direction and the second direction are orthogonal. The image sensor described in item 1. [Item 3] The system further includes a focus detection unit that detects the focus state based on the output signal from the first pixel and the output signal from the second pixel. The image sensor described in item 1 or 2. [Item 4] An imaging unit in which the first pixel and the second pixel are arranged, A signal processing unit is stacked with the imaging unit and processes signals from the imaging unit, An image sensor as described in any one of items 1 to 3. [Item 5] Having multiple of the aforementioned first pixels, Two third pixels are arranged in a continuous third direction intersecting the first direction, adjacent to two of the plurality of first pixels, and detect light of a third color. A plurality of third light-receiving regions arranged in the third pixel and receiving light of the third color divided in the third direction, An image sensor as described in any one of items 1 to 4, which includes the following: [Item 6] The second and third directions are parallel. The image sensor described in item 5. [Item 7] Arranged along the first and second directions, a plurality of first pixels corresponding to the first color, Each region surrounded by four adjacent first pixels is provided, and multiple other pixels corresponding to a color different from the first color, Equipped with, An image sensor in which at least some of the plurality of first pixels and the plurality of other pixels have two separate light-receiving regions. [Item 8] The system further includes a focus detection unit that detects the focus state of the image sensor based on the output signals from each light-receiving region of the pixel having the two light-receiving regions. The image sensor described in item 7. [Item 9] The aforementioned plurality of other pixels are, A plurality of second pixels, arranged along the aforementioned second direction and corresponding to the second color, A plurality of third pixels, arranged along the second direction and corresponding to the third color, Includes, The second and third rows of pixels are arranged alternately in the first direction. The array conversion unit further comprises: generating a first converted pixel signal by adding the pixel signals of two adjacent first pixels in the first direction; generating a second converted pixel signal by adding the pixel signals of two adjacent second pixels in the second direction; and generating a third converted pixel signal by adding the pixel signals of two adjacent third pixels in the second direction. The image sensor described in item 7 or 8. [Item 10] At least some of the pixels of the first pixel have a first light-receiving region and a second light-receiving region arranged side by side in the first direction, At least some of the pixels of the second and third pixels have a first light-receiving region and a second light-receiving region arranged side by side in the second direction. The image sensor described in item 9. [Item 11] All of the aforementioned pixels have the two light-receiving regions, The image sensor described in item 10. [Item 12] The array conversion unit performs the following for each of the pixels: A first pixel signal is generated by adding the output signal of the first light-receiving region of the pixel and the output signal of the second light-receiving region. A second pixel signal is generated by adding the output signal of the first light-receiving region of the pixel and the output signal of the second light-receiving region of a pixel adjacent to the first light-receiving region of the pixel. The image sensor described in item 11. [Item 13] For each pixel, with respect to the reset timing for resetting the charge accumulated in the first light-receiving region among the two light-receiving regions, the reset timing of the second light-receiving region is delayed, and output signals corresponding to the amounts of charge accumulated in the first light-receiving region and the second light-receiving region are read out simultaneously, and a pixel signal of the pixel is generated by subtracting the value of the output signal of the second light-receiving region from the value of the output signal of the first light-receiving region, and further comprising a global shutter processing unit. The imaging device according to any one of items 7 to 12. [Item 14] For each pixel, further comprising a readout unit that simultaneously reads out output signals corresponding to the amounts of charge accumulated in each of the two light-receiving regions, and independently for each light-receiving region. The imaging device according to any one of items 7 to 12. [Item 15] The planar shape of each of the pixels is a quadrilateral. Each side of the pixel is inclined at 45 degrees with respect to the first direction and the second direction. The imaging device according to any one of items 7 to 14. [Item 16] Based on lens data indicating the characteristics of the lens through which the light incident on the imaging device passes, further comprising a correction unit that corrects the values of the output signals output from each of the two light-receiving regions. The imaging device according to any one of items 7 to 15. [Item 17] An imaging chip on which each of the pixels is formed, A signal processing chip that is stacked with the imaging chip and processes signals from the imaging chip, The imaging device according to any one of items 7 to 16 having the above. [Item 18] An imaging device comprising the imaging device according to any one of items 1 to 17.

[0008] In a fourth aspect of the present invention, an imaging device comprising the imaging device according to any one of the first to third aspects is provided.

[0009] It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram shows an overview of an image sensor 100 according to one embodiment. [Figure 2A] This figure shows an example of the first pixel 202-1. [Figure 2B] This figure shows an example of the second pixel 202-2 and the third pixel 202-3. [Figure 3] This figure shows an example of the light-receiving unit 200. [Figure 4] This figure shows an example of array conversion processing in the signal processing unit 210. [Figure 5] This figure shows an example of the arrangement of the first conversion pixel 203-1. [Figure 6] This figure shows an example of array conversion processing in the signal processing unit 210. [Figure 7] This figure shows an example of the arrangement of the second conversion pixel 203-2 and the third conversion pixel 203-3. [Figure 8] This figure shows an example of the arrangement of the first conversion pixel 203-1, the second conversion pixel 203-2, and the third conversion pixel 203-3. [Figure 9] This figure shows another example of the light-receiving unit 200. [Figure 10A] This figure shows an example of generating the first converted pixel signal G1. [Figure 10B] This figure shows an example of generating the second converted pixel signal G2. [Figure 10C] This figure shows an example of generating the third conversion pixel signal G3. [Figure 10D] This figure shows an example of generating the fourth transformation pixel signal G4. [Figure 11A] This is a perspective view of microlens 101. [Figure 11B] This figure shows the planar shape of the microlens 101. [Figure 12]This figure shows another example of processing by the signal processing unit 210. [Figure 13] This figure shows an example configuration of the light-receiving unit 200. [Figure 14] This figure shows another example configuration of the light-receiving unit 200. [Figure 15] This figure shows an example of the arrangement of the transfer transistor TX and the charge detection unit in the example shown in Figure 14. [Figure 16] This figure shows an example of a cross-section of the image sensor 100. [Figure 17] This is a block diagram showing some of the functions of the signal processing unit 210. [Figure 18] This diagram illustrates the relationship between lens characteristics and the output signal. [Figure 19] This is a block diagram showing an example configuration of an imaging device 500 according to one embodiment. [Modes for carrying out the invention]

[0011] The present invention will be described below through embodiments, but these embodiments are not intended to limit the scope of the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] Figure 1 shows an overview of an image sensor 100 according to one embodiment. The image sensor 100 includes a light-receiving section 200 in which a plurality of pixels 202 are arranged, and a signal processing section 210 that processes signals from the light-receiving section 200. Each of the plurality of pixels 202 has a light-receiving element such as a photodiode and accumulates charge according to the amount of light received. The signal processing section 210 in this example reads out a signal corresponding to the amount of charge accumulated by each pixel 202 and performs predetermined processing.

[0013] In this example, the multiple pixels 202 are arranged in a matrix. That is, the multiple pixels 202 are arranged along multiple rows and multiple columns. In this specification, the row direction is shown as the x-axis direction and the column direction as the y-axis direction. The row direction is an example of a first direction, and the column direction is an example of a second direction.

[0014] Multiple pixels 202 include multiple first pixels 202-1, multiple second pixels 202-2, and multiple third pixels 202-3. The first pixel 202-1 is the pixel corresponding to the first color filter, the second pixel 202-2 is the pixel corresponding to the second color filter, and the third pixel 202-3 is the pixel corresponding to the third color filter. In this example, the first color is green, the second color is blue, and the third color is red. In this example, the planar shape of each pixel 202 is a rectangle, with each side of the pixel 202 tilted at 45 degrees with respect to the first and second directions. In a more specific example, the planar shape of each pixel 202 is a square.

[0015] Multiple first pixels 202-1 are arranged along both the row and column directions. In this example, each vertex of the first pixels 202-1 is positioned adjacent to one another. This arrangement creates a region enclosed by four closely spaced first pixels 202-1. The second pixel 202-2 and the third pixel 202-3 are located within the region enclosed by the four first pixels 202-1. In this example, the shape of each pixel 202 is identical.

[0016] The second pixel 202-2 is arranged along the column direction. Similarly, the third pixel 202-3 is also arranged along the column direction. The columns of the second pixel 202-2 and the columns of the third pixel 202-3 are arranged alternately in the row direction. Furthermore, the columns of the second pixel 202-2 and the columns of the third pixel 202-3 are offset by half a pixel in the column direction relative to the column of the first pixel 202-1.

[0017] Figure 2A shows an example of a first pixel 202-1. At least a portion of the first pixel 202-1 has two separated light-receiving regions 214. The first light-receiving region 214a and the second light-receiving region 214b of the first pixel 202-1 are arranged side by side in the row direction. In this example, the two light-receiving regions 214 are defined by dividing the region of the first pixel 202-1 into two equal parts with a straight line extending in the column direction. In this example, this straight line is the diagonal of the first pixel 202-1. An element isolation section is provided between the light-receiving regions 214 to prevent the charge generated in response to incident light from moving between the light-receiving regions 214. In Figure 2A, the microlens 101 provided corresponding to the first pixel 202-1 is shown by a dotted line. In this example, the two light-receiving regions 214 are provided at different positions in the row direction with respect to the common microlens 101.

[0018] In the light-receiving unit 200, a plurality of first pixels 202-1, each having two light-receiving regions 214, are arranged adjacently in the row direction. The signal processing unit 210 functions as a focus detection unit that detects the focus state by detecting the image plane phase difference in the row direction between the signal from the first light-receiving region 214a and the signal from the second light-receiving region 214b of the first pixels 202-1 arranged adjacently in the row direction. Since the first pixels 202-1 for detecting the image plane phase difference are arranged adjacently in the row direction, the image plane phase difference in the row direction can be detected with high accuracy. Furthermore, compared to a method that detects the image plane phase difference using light shielding, the efficiency of light utilization can be improved.

[0019] Figure 2B shows an example of a second pixel 202-2 and a third pixel 202-3. At least a portion of the second pixel 202-2 and the third pixel 202-3 has two separate light-receiving regions 214. The first light-receiving region 214a and the second light-receiving region 214b of the second pixel 202-2 and the third pixel 202-3 are arranged side by side in the column direction. In this example, the two light-receiving regions 214 are defined by dividing the region of the second pixel 202-2 or the third pixel 202-3 into two equal parts by a straight line extending in the row direction. The two light-receiving regions 214 of the second pixel 202-2 and the third pixel 202-3 are positioned at different locations in the column direction with respect to a common microlens 101.

[0020] In the light-receiving unit 200, a plurality of second pixels 202-2 or third pixels 202-3, each having two light-receiving regions 214, are arranged adjacently in the column direction. The signal processing unit 210 functions as a focus detection unit that detects the focus state by detecting the image plane phase difference in the column direction between the signal from the first light-receiving region 214a and the signal from the second light-receiving region 214b of the second pixels 202-2 or third pixels 202-3, which are arranged adjacently in the column direction. Since the second pixels 202-2 or third pixels 202-3 for detecting the image plane phase difference are arranged adjacently in the column direction, the image plane phase difference in the column direction can be detected with high accuracy. Furthermore, compared to a method that uses light shielding to detect the image plane phase difference, the efficiency of light utilization can be improved.

[0021] Figure 3 shows an example of the light-receiving unit 200. In this example, all pixels 202 have two light-receiving regions 214. In Figure 3, the boundaries of the light-receiving regions 214 in each pixel 202 are shown by dotted lines. In this example, image data is generated using the output of all pixels 202, and the output of at least some of the pixels 202 is used for image plane phase difference detection. The signal processing unit 210 can use pixels 202 at any position as pixels 202 for image plane phase difference detection.

[0022] The signal processing unit 210 may change the pixels 202 used for image plane phase difference detection at any time. For example, the signal processing unit 210 may use the pixels 202 capturing an image of a specific subject as the pixels 202 for image plane phase difference detection. If the position of the pixels 202 capturing the image of the subject changes over time, the signal processing unit 210 may follow this change and select the pixels 202 for image plane phase difference detection. Alternatively, all pixels 202 may be used for both image signal generation and image plane phase difference detection. In this example, since light shielding is not used for image plane phase difference detection, the efficiency of incident light utilization does not decrease even if the structure is designed so that all pixels 202 can be used for image plane phase difference detection.

[0023] Furthermore, the signal processing unit 210 also functions as an array conversion unit that converts image data based on each pixel signal from the light receiving unit 200 into image data of a predetermined pixel array, such as a Bayer array. When performing array conversion, the signal processing unit 210 adds the signals from the two light receiving areas 214 of each pixel 202 to obtain the pixel signal from each pixel 202.

[0024] Figure 4 shows an example of array transformation processing in the signal processing unit 210. In Figure 4, the column numbers of the multiple pixels 202 are m, m+1, m+2, ..., m+k, ..., and the row numbers are n, n+1, n+2, ..., n+l, ..., where k and l are integers. Figure 4 describes the process of generating the transformed pixel signal of the first transformed pixel 203-1 after array transformation from the pixel signal of the first pixel 202-1. In this example, the first pixel 202-1 is arranged in a column where k is 0 or even, and in a row where l is 0 or even.

[0025] Furthermore, multiple first pixels 202-1 include three or more first pixels 202-1 arranged consecutively in the first direction. For example, three first pixels 202-1 are arranged at positions (m, n+2), (m+2, n+2), and (m+4, n+2). Additionally, multiple second pixels 202-2 (corresponding to the "B" pixel in Figure 4) include two second pixels 202-2 arranged consecutively in a second direction intersecting the first direction, each adjacent to two of the three first pixels 202-1 mentioned above. For example, each of the second pixels 202-2 arranged at positions (m+3, n+1) and (m+3, n+3) intersects with and is adjacent to two first pixels 202-1 arranged at positions (m+2, n+2) and (m+4, n+2).

[0026] Furthermore, each of the multiple third pixels 202-3 is arranged in a continuous third direction intersecting the first direction, and each includes two third pixels 202-3 adjacent to two of the three first pixels 202-1 mentioned above. Note that the second and third directions are parallel directions and refer to directions at different locations. For example, the second direction is the direction from position (m+3, n+1) to position (m+3, n+3), and the third direction is the direction from position (m+1, n+1) to position (m+1, n+3). Also, the two first pixels 202-1 adjacent to the two third pixels 202-3 are different from the two first pixels 202-1 adjacent to the two second pixels 202-2 mentioned above, in that at least one of the first pixels 202-1 is different. For example, each of the two third pixels 202-3 positioned at (m+1, n+1) and (m+1, n+3) intersects with and is adjacent to the two first pixels 202-1 positioned at (m, n+2) and (m+2, n+2).

[0027] The signal processing unit 210 adds the pixel signals of two adjacent first pixels 202-1 in the row direction to generate a converted pixel signal for a first converted pixel 203-1 that is virtually positioned between the two first pixels 202-1. In Figure 4, the two first pixels 202-1 whose pixel signals are added are connected by a double-headed arrow.

[0028] More specifically, the signal processing unit 210 groups the first pixels 202-1 of each row so that they form pairs of two adjacent first pixels 202-1. The signal processing unit 210 adds the pixel signals of the two paired first pixels 202-1 to generate the converted pixel signal of the first converted pixel 203-1. At this time, the first pixels 202-1 of each row are grouped so that the row-direction position of the first converted pixel 203-1 alternates for each row of first pixels 202-1. For example, in the n+s (where s is 0, 4, 8, ...) row, the first pixels 202-1 at the column positions (m, m+2), (m+4, m+6), and (m+8, m+10) are grouped. In contrast, in the n+s+2th row, the first pixels 202-1 at the column positions (m+2, m+4), (m+6, m+8), and (m+10, m+12) are grouped together.

[0029] Figure 5 shows an example of the arrangement of the first transformed pixels 203-1. Due to the transformation process described in Figure 4, the first transformed pixels 203-1 are arranged as shown in Figure 5. In other words, the row-direction positions of the first transformed pixels 203-1 are arranged alternately for each row of the first transformed pixels 203-1. Specifically, in the n+s row, the first transformed pixels 203-1 are placed in the column positions m+1, m+5, and m+9. Also, in the n+s+2 row, the first transformed pixels 203-1 are placed in the column positions m+3, m+7, and m+11.

[0030] Figure 6 shows an example of array transformation processing in the signal processing unit 210. In Figure 6, the process of generating the transformed pixel signals of the second transformed pixel 203-2 and the third transformed pixel 203-3 after array transformation from the pixel signals of the second pixel 202-2 and the third pixel 202-3 is explained. In this example, the second pixel 202-2 and the third pixel 202-3 are arranged in columns where k is odd. In this example, the second pixel 202-2 is arranged in the columns m+3, m+7, m+11, ... and the third pixel 202-3 is arranged in the columns m+1, m+5, m+9, ...

[0031] The signal processing unit 210 adds the pixel signals of two adjacent second pixels 202-2 in the column direction to generate a converted pixel signal for a second converted pixel 203-2 virtually positioned between the two second pixels 202-2. The signal processing unit 210 also adds the pixel signals of two adjacent third pixels 202-3 in the column direction to generate a converted pixel signal for a third converted pixel 203-3 virtually positioned between the two third pixels 202-3. In Figure 6, the two pixels 202 whose pixel signals are added are connected by a double-headed arrow.

[0032] Furthermore, the pairs of second pixels 202-2 and third pixels 202-3 that add the pixel signals are selected so that the double-headed arrows connecting the two first pixels 202-1 explained in Figure 4, the double-headed arrows connecting the two second pixels 202-2 explained in Figure 6, and the double-headed arrows connecting the two third pixels 202-3 do not overlap. In other words, the pairs of second pixels 202-2 and third pixels 202-3 that add the pixel signals are selected so that the positions of the first conversion pixel 203-1, the second conversion pixel 203-2, and the third conversion pixel 203-3 do not overlap.

[0033] More specifically, the second pixel 202-2 is grouped with the second pixels 202-2 at row positions (n+3, n+5), (n+7, n+9), and (n+11, n+13). Conversely, the third pixel 202-3 is grouped with the third pixels 202-3 at column positions (n+1, n+3), (n+5, n+7), and (n+9, n+11).

[0034] Figure 7 shows an example of the arrangement of the second transformation pixel 203-2 and the third transformation pixel 203-3. Due to the transformation process described in Figure 6, the second transformation pixel 203-2 and the third transformation pixel 203-3 are arranged as shown in Figure 7. Specifically, in columns m+3, m+7, and m+11, the second transformation pixel 203-2 is placed at row positions n+4, n+8, and n+12. Also, in rows m+1, m+5, and m+9, the third transformation pixel 203-3 is placed at row positions n+2, n+6, and n+10.

[0035] Figure 8 shows an example of the arrangement of the first conversion pixel 203-1, the second conversion pixel 203-2, and the third conversion pixel 203-3. The arrangement shown in Figure 8 is an arrangement in which the arrangements of each conversion pixel 203 shown in Figures 5 and 7 are superimposed. Through the processing described in Figures 4 to 7, the signal processing unit 210 can acquire Bayer array image data as shown in Figure 8.

[0036] As described above, the image sensor 100 allows for the continuous arrangement of pixels for image plane phase difference detection in both the row and column directions, thereby improving the detection accuracy of image plane phase difference. Furthermore, Bayer array image data can be acquired through a simple calculation of adding the pixel signals of adjacent pixels 202. In addition, since light shielding is not used for image plane phase difference detection, the efficiency of light utilization can be improved.

[0037] Figure 9 shows another example of the light-receiving unit 200. In this example of the light-receiving unit 200, some first pixels 202-1, some second pixels 202-2, and some third pixels 202-3 each have two light-receiving regions 214. However, the first pixels 202-1 having two light-receiving regions 214 are arranged consecutively in the row direction. Also, the second pixels 202-2 having two light-receiving regions 214 are arranged consecutively in the column direction. Also, the third pixels 202-3 having two light-receiving regions 214 are arranged consecutively in the column direction. The other configurations are the same as the light-receiving unit 200 described in Figures 1 to 8.

[0038] This configuration allows for the continuous arrangement of pixels for image plane phase difference detection in both the row and column directions, thereby improving the detection accuracy of image plane phase difference. Furthermore, Bayer array image data can be acquired simply by adding the pixel signals of adjacent pixels 202. In addition, since light shielding is not used for image plane phase difference detection, the efficiency of light utilization can be improved.

[0039] Figures 10A to 10D illustrate other processing examples of the signal processing unit 210. In this example, the signal processing unit 210 generates first to fourth converted pixel signals that are shifted in position in the row direction as converted pixel signals for the first pixel 202-1. Figure 10A shows an example of generating the first converted pixel signal G1. The processing in this example is the same as the processing described in Figure 4. That is, for each first pixel 202-1, the signal processing unit 210 adds the output signals of the first light-receiving area 214a and the second light-receiving area 214b within the pixel to generate the first pixel signal S1. Then, it adds the first pixel signals S1 of two adjacent first pixels 202-1 to generate the first converted pixel signal G1. In this example, the first converted pixel signal G1 is the signal of a virtual converted pixel at positions m+1, m+5, ...

[0040] Figure 10B shows an example of generating a second conversion pixel signal G2. In this example, the second conversion pixel signal G2 is the signal of a conversion pixel at a different position from the first conversion pixel signal G1. In this example, for each first pixel 202-1, the output signal of the first light-receiving area 214a of that pixel and the output signal of the second light-receiving area 214b of the first pixel 202-1 adjacent to the first light-receiving area of ​​that pixel are added together to generate a second pixel signal S2. The signal processing unit 210 then adds the adjacent second pixel signals S2 to generate a second conversion pixel signal G2. In this example, the second conversion pixel signal G2 is the signal of a virtual conversion pixel at positions m+2, m+6, ...

[0041] Figure 10C shows an example of generating a third transformed pixel signal G3. In this example, the third transformed pixel signal G3 is the signal of a transformed pixel at a different position from the first transformed pixel signal G1 and the second transformed pixel signal G2. First, the third transformed pixel signal S3 is generated using the same process as the first transformed pixel signal S1. Then, the signal processing unit 210 adds the adjacent third transformed pixel signals S3 to generate the third transformed pixel signal G3. In this example, the third transformed pixel signal G3 is the signal of a virtual transformed pixel at the positions m+3, m+7, ...

[0042] Figure 10D shows an example of generating the fourth transformed pixel signal G4. In this example, the fourth transformed pixel signal G4 is the signal of a transformed pixel at a different position from the first transformed pixel signal G1, the second transformed pixel signal G2, and the third transformed pixel signal G3. First, the fourth transformed pixel signal S4 is generated using the same process as the second transformed pixel signal S2. Then, the signal processing unit 210 adds the adjacent fourth transformed pixel signals S4 to generate the fourth transformed pixel signal G4. In this example, the fourth transformed pixel signal G4 is the signal of a virtual transformed pixel at the positions m, m+4, ...

[0043] Through this process, the signal processing unit 210 can generate multiple types of converted pixel signals G1 to G4 at different positions. The signal processing unit 210 may use the multiple types of converted pixel signals as image data for one frame, or as image data for different frames. In other words, images with multiple types of converted pixel signals may be displayed approximately simultaneously, or at different frame timings. Furthermore, the signal processing unit 210 may generate the above-mentioned multiple types of converted pixel signals from pixel signals captured approximately simultaneously, or from pixel signals acquired at different imaging timings. This process can improve the spatial resolution of the image data. In Figures 10A to D, the first pixel 202-1 was used as an example, but the second pixel 202-2 and the third pixel 202-3 can also be generated using similar processing.

[0044] Figures 11A and 11B show examples of the structure of the microlens 101. Figure 11A is a perspective view of the microlens 101. Here, curved grid lines indicate curved surfaces, and straight grid lines indicate planes. Figure 11B shows the planar shape of the microlens 101. As shown in Figures 11A and 11B, the microlens 101 has a shape like a spherical lens with all four sides cut off. This allows the use of a spherical lens with a larger diameter, thereby increasing the effective aperture of the microlens 101. Furthermore, by aligning the positions of the four sides of the microlens 101 with the positions of the four sides of the pixels 202, the microlenses 101 can be efficiently arranged.

[0045] Figure 12 shows another example of processing by the signal processing unit 210. In this example, the signal processing unit 210 selects pixels 202 to read the output signals of the light-receiving area 214 row by row. The signal processing unit 210 simultaneously reads the output signals of the pixels 202 belonging to the selected row. In this case, the timing of reading the output signals differs from row to row, and the charge accumulation time also differs from row to row. In this example, the signal processing unit 210 compensates for the difference in charge accumulation time by correcting the output signal of the first light-receiving area 214a using the output signal of the second light-receiving area 214b of each pixel 202. In this example, all pixels 202 in the light-receiving unit 200 have two light-receiving areas 214.

[0046] In Figure 12, a1 indicates the charge accumulation time of the first light-receiving region 214a of pixel 202 belonging to the first row, and b1 indicates the charge accumulation time of the second light-receiving region 214b. Similarly, a2 indicates the charge accumulation time of the first light-receiving region 214a of pixel 202 belonging to the second row, and b2 indicates the charge accumulation time of the second light-receiving region 214b. The same applies to the other rows. In Figure 12, the ADC indicates the time it takes to digitally convert the output signal of each light-receiving region 214.

[0047] As shown in Figure 12, the signal processing unit 210 delays the reset timing B of the second light-receiving region 214b for each pixel 202 relative to the reset timing A, which resets the charge accumulated in the first light-receiving region 214a. For this reason, the light-receiving unit 200 has reset lines that independently control the reset timings of the first light-receiving region 214a and the second light-receiving region 214b of each pixel 202. Reset timings A and B are common to all pixels 202.

[0048] The signal processing unit 210 then simultaneously reads out output signals for each pixel 202 corresponding to the amount of charge accumulated in the first light-receiving area 214a and the second light-receiving area 214b. For this reason, the light-receiving unit 200 has readout lines that transmit the output signals of the first light-receiving area 214a and the second light-receiving area 214b of each pixel 202 in parallel. The signal processing unit 210 also has a processing circuit that processes the output signals of the first light-receiving area 214a and the second light-receiving area 214b of each pixel 202 in parallel.

[0049] The signal processing unit 210 generates a pixel signal for each pixel 202 by subtracting the output signal value of the second light-receiving area 214b from the output signal value of the first light-receiving area 214a. This allows for the generation of a pixel signal for all pixels 202 corresponding to the charge accumulation time from reset timing A to reset timing B. Through this process, a pixel signal generated by a global shutter can be simulated from the output signal read out by rolling readout. The signal processing unit 210 also functions as a global shutter processing unit that performs the processing described in Figure 12.

[0050] Figure 13 shows an example of the configuration of the light-receiving unit 200. Although Figure 13 shows the configuration for only one pixel 202, the light-receiving unit 200 has a similar configuration for all pixels 202. As described above, the light-receiving unit 200 has a reset line 221-1 that controls the reset timing of the first light-receiving area 214a and a reset line 221-2 that controls the reset timing of the second light-receiving area 214b. Reset lines 221-1 and 221-2 are provided for each row of pixels 202. Pixels 202 included in the same row are connected to the common reset line 221-1 and reset line 221-2.

[0051] Furthermore, the light-receiving unit 200 has a readout line 224-1 for reading the output signal of the first light-receiving area 214a and a readout line 224-2 for reading the output signal of the second light-receiving area 214b. Readout lines 224-1 and 224-2 are provided for each row of pixels 202. Pixels 202 included in the same row are connected to a common readout line 224-1 and readout line 224-2. The readout lines 224 transmit their respective output signals to the signal processing unit 210.

[0052] The signal processing unit 210 selects the row from which to read the output signal using the row selection signal SEL. The signal processing unit 210 also selects the light-receiving area 214 to which the output signal should be transferred using the transfer signals Tx1 and Tx2.

[0053] With this configuration, the signal processing unit 210 functions as a readout unit that simultaneously reads out output signals corresponding to the amount of charge accumulated in the first light-receiving area 214a and the second light-receiving area 214b for each pixel 202, and independently for each light-receiving area. Furthermore, the signal processing unit 210 can pseudo-generate pixel signals using a global shutter from the output signals read out by rolling readout. Note that the signal processing unit 210 may perform array conversion processing using the pixel signals described in Figures 11A, B and 12 instead of the pixel signals described in Figures 1 to 10D. In other words, the signal processing unit 210 may generate pixel signals by subtracting the output signal of the second light-receiving area 214b from the output signal of the first light-receiving area 214a, without adding the output signals of the first light-receiving area 214a and the second light-receiving area 214b.

[0054] Figure 14 shows another example of the configuration of the light-receiving unit 200. In this example, the global shutter processing described in Figures 12 and 13 is not performed. In this example, each light-receiving region 214 is a photodiode. In the light-receiving unit 200 of this example, a reset transistor R, a source follower transistor SF, and a selection transistor S are provided in common for the four photodiodes. For example, a reset transistor R, etc., are provided in common for the four photodiodes included in region 240.

[0055] Furthermore, a transfer transistor TX is provided for each photodiode. These four photodiodes are each located in different pixels 202. For example, four photodiodes sharing a reset transistor R are located in two first pixels 202-1 and two second pixels 202-2.

[0056] The transfer transistor TX switches whether or not to transfer the charge accumulated by the photodiode to the charge detection unit. The charge detection unit is, for example, a capacitance connected between the wiring and the reference potential (not shown). This charge detection unit is also shared by the four photodiodes.

[0057] The reset transistor R switches whether or not to reset the charge transferred to the charge detection unit. The source follower transistor SF outputs an output signal corresponding to the charge accumulated in the charge detection unit. The selection transistor S switches whether or not to output the output signal to the readout line 224.

[0058] Figure 15 shows an example of the arrangement of the transfer transistor TX and the charge detection unit in the example shown in Figure 14. In this example, the pixel 202 and the transistor are located on different layers. Therefore, the pixel 202 and the transistor can be arranged on top of each other. As described above, the charge detection unit and the reset transistor R are shared by four photodiodes PD. A transfer transistor TX is provided on each photodiode PD. In Figure 15, the gate electrode of the transfer transistor TX is shown in the shaded area.

[0059] The four photodiodes are contained within two first pixels 202-1 and two second pixels 202-2 or third pixels 202-3. Since the first pixels 202-1 and the second and third pixels 202-2 and 202-3 are divided in different directions, a region is created surrounded by the four transfer transistors TX. This region functions as a charge detection unit. Note that in Figure 15, the reset transistor R and other components are omitted, but as shown in Figure 14, the reset transistor R and other components are also shared by the four photodiodes.

[0060] Figure 16 shows an example of a cross-section of the image sensor 100. In this example, a back-illuminated image sensor 100 is shown, but the image sensor 100 is not limited to a back-illuminated type. The image sensor 100 in this example includes an imaging chip 113 that outputs a signal corresponding to incident light, a signal processing chip 111 that processes the signal from the imaging chip 113, and a memory chip 112 that stores the image data processed by the signal processing chip 111. These imaging chip 113, signal processing chip 111, and memory chip 112 are stacked and electrically connected to each other by conductive bumps 109 made of Cu or the like.

[0061] As shown in the figure, incident light mainly enters in the direction indicated by the white arrow. In this embodiment, the side of the imaging chip 113 that receives incident light is referred to as the back surface. An example of the imaging chip 113 is a back-illuminated MOS image sensor. The imaging chip 113 corresponds to the light receiving unit 200. The PD (photodiode) layer 106 is located on the back side of the wiring layer 108. The PD layer 106 is arranged two-dimensionally and has a plurality of PD units 104 that accumulate charge according to the incident light, and transistors 105 provided corresponding to the PD units 104. One PD unit 104 is provided for one pixel 202. That is, the PD unit 104 has a first light receiving region 214a and a second light receiving region 214b.

[0062] A color filter 102 is provided on the incident light side of the PD layer 106 via a passivation film 103. The color filter 102 has multiple types that transmit different wavelength regions and has a specific arrangement corresponding to each of the PD units 104. The set of the color filter 102, PD unit 104, and multiple transistors 105 forms one pixel. By controlling the on / off state of the multiple transistors 105, the readout timing, light reception start timing (reset timing), etc., of each light receiving area 214 are controlled.

[0063] Microlenses 101 are provided on the incident light side of the color filter 102, corresponding to each pixel. The microlenses 101 focus the incident light toward the corresponding PD section 104.

[0064] The wiring layer 108 has wiring 107 that transmits signals from the PD layer 106 to the signal processing chip 111. The wiring 107 corresponds to, for example, the readout line 224 shown in Figure 12. The gate electrodes of the transistors shown in Figures 13 and 14 may also be formed on the wiring layer 108. The transistors shown in Figures 13 and 14 may also be formed on the signal processing chip 111. In this case, the wiring 107 corresponds to the wiring connecting the PD layer 106 and each transistor. The wiring 107 may be multilayer and may also contain passive and active elements. The signal processing chip 111 in this example includes a signal processing unit 210.

[0065] Multiple bumps 109 are arranged on the surface of the wiring layer 108. These multiple bumps 109 are aligned with multiple bumps 109 provided on the opposing surface of the signal processing chip 111, and when the imaging chip 113 and the signal processing chip 111 are pressed together, the aligned bumps 109 are joined together and electrically connected.

[0066] Similarly, multiple bumps 109 are arranged on the opposing surfaces of the signal processing chip 111 and the memory chip 112. When these bumps 109 are aligned with each other and pressure is applied to the signal processing chip 111 and the memory chip 112, the aligned bumps 109 are joined together and electrically connected.

[0067] Furthermore, the joining of bumps 109 is not limited to Cu bump joining by solid-phase diffusion; microbump joining by solder melting may also be employed. Also, it is sufficient to provide about one bump 109 per unit block, as will be described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PD section 104. In addition, in peripheral areas other than the imaging area where pixels are arranged, bumps larger than the bumps 109 corresponding to the imaging area may also be provided.

[0068] The signal processing chip 111 has TSVs (Through-Silicon Electrodes) 110 that connect circuits provided on its front and back surfaces. The TSVs 110 are preferably provided in the peripheral region. The TSVs 110 may also be provided in the peripheral region of the imaging chip 113 and the memory chip 112.

[0069] Figure 17 is a block diagram showing some of the functions of the signal processing unit 210. In this example, the signal processing unit 210 includes a correction unit 260 and a lookup table 270. As explained in Figures 1 to 16, the signal processing unit 210 adds or subtracts the output signals of the two light-receiving regions 214 in each pixel 202. However, the output signals of the two light-receiving regions 214 may vary depending on the characteristics of the lens through which the light incident on the image sensor passes.

[0070] For example, the ratio of the output value of the first light-receiving area 214a to the output value of the second light-receiving area 214b in each pixel 202 varies depending on the EPD value and F-number of the lens. The EPD value is a value that indicates the distance from the image plane (the surface of the image sensor 100) to the exit pupil of the lens. The F-number is a value obtained by dividing the focal length of the lens by the effective aperture. The lookup table 270 stores a table that associates correction values ​​for correcting the output value of each light-receiving area 214 with lens characteristic values ​​such as the EPD value and F-number. The table of lens characteristic values ​​and correction values ​​may be set for each position of the pixel 202.

[0071] The correction unit 260 receives lens data from the imaging device regarding the lens through which the light incident on the image sensor has passed, and receives an output signal from the light receiving unit 200. For example, the imaging device may detect lens characteristics from identification information of the lens unit being used. Alternatively, the imaging device may detect lens characteristics based on the operation of the imaging device by the user or the like. The correction unit 260 also receives information indicating the position of the pixel 202 of the output signal. This position information may be generated by the signal processing unit 210 based on a row selection signal SEL or the like.

[0072] The correction unit 260 extracts correction values ​​corresponding to the lens data from the lookup table 270. These correction values ​​may differ for each light-receiving area 214. The correction unit 260 uses the extracted correction values ​​to generate a correction signal by correcting the output signals of the two light-receiving areas 214. The signal processing unit 210 uses this correction signal to generate a pixel signal.

[0073] Figure 18 illustrates the relationship between lens characteristics and output signals. In Figure 18, the horizontal axis represents the distance of the pixel 202 to the optical axis, and the vertical axis represents the magnitude of the output signal of the light-receiving region 214 at each pixel 202. In Figure 18, the output signals of the two light-receiving regions 214 are shown by solid and dotted lines.

[0074] Typically, the microlenses 101 in the image sensor 100 are positioned offset from the pixels 202 according to their position relative to the optical axis. This design ensures that for a given EPD value, the light spot is positioned at the center of each pixel 202, regardless of its position. This EPD value, where the light spot is centered on each pixel 202 regardless of its position, is referred to as "EPD just."

[0075] In contrast, with lenses that have a shorter EPD (Electronic Printed Diode) or a longer EPD than lenses that have an EPD that is just the same length, the light spot will be shifted from the center of the pixel 202 depending on the position of the pixel 202. Since the pixel 202 is divided into two light-receiving regions 214 by the center line, if the light spot is shifted from the center of the pixel 202, a difference in the magnitude of the output signal will occur in the two light-receiving regions 214. For example, at a position far from the optical axis, most of the light spot will be included in one of the light-receiving regions 214, resulting in a very large output signal in that region, while the output signal in the other light-receiving region 214 will be very small.

[0076] Furthermore, when the F-number changes, the diameter of the light spot on the image plane changes. For example, a smaller F-number results in a larger spot diameter. In this case, the difference in the magnitude of the output signals of the two light-receiving regions 214 decreases. On the other hand, at positions far from the optical axis, the light spot extends outside the area of ​​the pixel 202, and the magnitude of the output signal for the entire pixel 202 decreases.

[0077] Thus, the magnitude of the output signals from the two light-receiving regions 214 varies depending on the lens characteristics such as the EPD value and F-number. The signal processing unit 210 in this example is pre-programmed with a table that associates correction values ​​to compensate for these variations with lens characteristic values. This table can be created by actually detecting the output signals while changing the lens characteristics. With this configuration, pixel signals can be generated with greater accuracy.

[0078] Figure 19 is a block diagram showing an example configuration of an imaging device 500 according to one embodiment. The imaging device 500 includes an imaging lens 520 as an imaging optical system, which guides the subject light beam incident along the optical axis OA to the image sensor 100. The imaging lens 520 may be an interchangeable lens that can be attached to and detached from the imaging device 500. The imaging device 500 mainly comprises an image sensor 100, a system control unit 501, a drive unit 502, a photometering unit 503, a work memory 504, a recording unit 505, a display unit 506, and a drive unit 514.

[0079] The photographic lens 520 is composed of multiple optical lens groups and forms an image of the subject light beam from the scene near its focal plane. In Figure 19, the photographic lens 520 is represented by a single hypothetical lens positioned near the pupil.

[0080] The drive unit 514 drives the photographic lens 520. More specifically, the drive unit 514 moves the optical lens group of the photographic lens 520 to change the focus position, and also drives the iris diaphragm inside the photographic lens 520 to control the amount of light beam incident on the image sensor 100.

[0081] The drive unit 502 is a control circuit that performs charge accumulation control, such as timing control and area control, of the image sensor 100 according to instructions from the system control unit 501. The drive unit 502 operates the light receiving unit 200 and the signal processing unit 210 of the image sensor 100 as described in relation to Figures 1 to 18. The operation unit 508 also receives instructions from the imager via a release button or the like.

[0082] The image sensor 100 is the same as the image sensor 100 described in relation to Figures 1 to 18. The image sensor 100 passes pixel signals to the image processing unit 511 of the system control unit 501. The image processing unit 511 uses the work memory 504 as a workspace to perform various image processing operations and generate image data. For example, when generating image data in JPEG file format, it generates a color video signal from the signals obtained by the Bayer array and then performs compression processing. The image processing unit 511 may have a signal processing unit 210. In this case, the image sensor 100 does not need to have a signal processing unit 210. The generated image data is recorded in the recording unit 505 and converted into a display signal and displayed in the display unit 506 for a preset time.

[0083] The photometering unit 503 detects the brightness distribution of the scene prior to a series of shooting sequences that generate image data. The photometering unit 503 includes, for example, an AE sensor of about 1 million pixels. The calculation unit 512 of the system control unit 501 receives the output of the photometering unit 503 and calculates the brightness of each area of ​​the scene. The calculation unit 512 determines the shutter speed, aperture value, and ISO sensitivity according to the calculated brightness distribution. The photometering unit 503 may also be used in conjunction with the image sensor 100. The calculation unit 512 also performs various calculations for operating the imaging device 500. The drive unit 502 may be partially or entirely mounted on the signal processing chip 111 of the image sensor 100. Part of the system control unit 501 may also be mounted on the signal processing chip 111 of the image sensor 100.

[0084] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0085] It should be noted that the execution order of operations, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before," "prior to," etc., and that these can be performed in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," "next," etc. for convenience, this does not mean that it is mandatory to perform the operations in that order. [Explanation of Symbols]

[0086] 100 Image sensor, 101 Microlens, 102 Color filter, 103 Passivation film, 104 PD section, 105 Transistor, 106 PD layer, 107 Wiring, 108 Wiring layer, 109 Bump, 110 TSV, 111 Signal processing chip, 112 Memory chip, 113 Imaging chip, 200 Light receiving section, 202 Pixel, 203 Conversion pixel, 210 Signal processing section, 214 Light receiving area, 214a First light receiving area, 214b Second light receiving area, 222-1, 222-2 Reset line, 224 Readout line, 260 Correction section, 270 Look-up table, 500 Imaging device, 501 System control section, 502 Drive section, 503 Photometer section, 504 Work memory, 505 Recording section, 506 Display section, 508 511 Control unit, 512 Image processing unit, 514 Calculation unit, 514 Drive unit, 520 Shooting lens

Claims

[Claim 1] A first photoelectric conversion unit that converts light transmitted through a first microlens and a first filter having first spectral characteristics into an electric charge, A second photoelectric conversion unit that converts light transmitted through a second microlens positioned next to the first microlens in the row direction and a second filter having the first spectral characteristics into an electric charge, A third photoelectric conversion unit that converts light transmitted through a third microlens and a third filter having second spectral characteristics different from the first spectral characteristics into electric charge, A fourth photoelectric conversion unit that converts light transmitted through a fourth microlens, which is positioned next to the third microlens in the column direction, and a fourth filter having the second spectral characteristics, into an electric charge, A photoelectric conversion unit that converts light transmitted through the first microlens and the first filter into an electric charge, comprising a fifth photoelectric conversion unit disposed next to the first photoelectric conversion unit in the row direction, A photoelectric conversion unit that converts light transmitted through the second microlens and the second filter into an electric charge, comprising a sixth photoelectric conversion unit disposed next to the second photoelectric conversion unit in the row direction, A photoelectric conversion unit that converts light transmitted through the third microlens and the third filter into an electric charge, comprising a seventh photoelectric conversion unit arranged next to the third photoelectric conversion unit in the column direction, A photoelectric conversion unit that converts light transmitted through the fourth microlens and the fourth filter into an electric charge, comprising an eighth photoelectric conversion unit arranged next to the fourth photoelectric conversion unit in the column direction, A first transfer unit that transfers the charge converted by the first photoelectric conversion unit, A second transfer unit that transfers the charge converted by the second photoelectric conversion unit, A third transfer unit that transfers the charge converted by the third photoelectric conversion unit, A fourth transfer unit that transfers the charge converted by the fourth photoelectric conversion unit, A first charge detection unit that detects the charge transferred from the first photoelectric conversion unit by the first transfer unit, the charge transferred from the second photoelectric conversion unit by the second transfer unit, the charge transferred from the third photoelectric conversion unit by the third transfer unit, and the charge transferred from the fourth photoelectric conversion unit by the fourth transfer unit. An image sensor equipped with the following features.

Images