Solid-state imaging device and electronic device

Abstract

The present disclosure relates to a solid-state imaging device which makes it possible to reduce degradation of phase difference information, and an electronic apparatus. Provided is a solid-state imaging device comprising a pixel array unit in which a plurality of pixels are two-dimensionally arrayed, wherein the plurality of pixels include a phase difference pixel for detecting a phase difference. The pixel array unit has an array pattern in which pixel portions including proximate pixels of a same color are regularly arrayed. During reading of a plurality of pixels, the phase difference pixel is prevented from being added in some portions when pixel signals of predetermined horizontal pixels and pixel signals of predetermined vertical pixels are added horizontally and vertically. The present disclosure is applicable, for example, to a CMOS image sensor having a phase difference pixel.

Description

[Technical field]

[0001] The present disclosure relates to a solid-state imaging device and an electronic device, and more particularly to a solid-state imaging device and an electronic device that are capable of suppressing deterioration of phase difference information. [Background technology]

[0002] In recent years, in order to improve the speed of autofocus, solid-state imaging devices in which pixels for detecting phase differences (hereinafter referred to as phase difference pixels) are arranged have been used.

[0003] As a configuration of this type of phase difference pixel, the techniques disclosed in Patent Documents 1 and 2 are known. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-216647 A [Patent Document 2] JP 2011-250325 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, when a configuration including phase difference pixels is used for a plurality of pixels two-dimensionally arranged in a pixel array section, it is required to suppress deterioration of phase difference information.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress deterioration of phase difference information. [Means for solving the problem]

[0007] According to one aspect of the present disclosure, there is provided a solid-state imaging device including a pixel array section in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels including phase difference pixels for detecting a phase difference, and the pixel array section includes a plurality of adjacent phase difference pixels. Supports the same wavelength bandA pixel section including pixels is regularly arranged in an array pattern, and when reading out the pixels, pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically. In this case, the phase difference pixel is included in a predetermined pixel in the horizontal direction and a predetermined pixel in the vertical direction. When the phase difference pixels are not added, the solid-state imaging device is configured such that the phase difference pixels are partially not added.

[0008] According to one aspect of the present disclosure, there is provided an electronic device including a pixel array section in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels including phase difference pixels for detecting a phase difference, and the pixel array section includes a neighboring Supports the same wavelength band A pixel section including pixels is regularly arranged in an array pattern, and when reading out the pixels, pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically. In this case, the phase difference pixel is included in a predetermined pixel in the horizontal direction and a predetermined pixel in the vertical direction. In this case, the electronic device is equipped with a solid-state imaging device in which the phase difference pixels are partially non-added.

[0009] A solid-state imaging device and an electronic device according to one aspect of the present disclosure include a pixel array section in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels including phase difference pixels for detecting a phase difference, and the pixel array section includes a neighboring Supports the same wavelength band The pixel portion includes pixels and has an array pattern in which the pixels are regularly arranged. When reading out the plurality of pixels, pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added together horizontally and vertically. In this case, the phase difference pixel is included in a predetermined pixel in the horizontal direction and a predetermined pixel in the vertical direction. At this time, the phase difference pixels are partially non-added.

[0012] The solid-state imaging device or electronic device according to one aspect of the present disclosure may be an independent device or an internal block constituting a single device. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of a solid-state imaging device to which the technology according to the present disclosure is applied. [Diagram 2] 2 is a diagram showing a planar layout of a plurality of pixels arranged in a pixel array portion; FIG. [Diagram 3]FIG. 13 is a diagram showing an example of readout driving in a 3-exposure HDR·H2V2 addition method. [Figure 4] FIG. 13 is a schematic diagram showing the state when the entire area is uniformly added with H2V2. [Diagram 5] 11 is a diagram illustrating a state where phase difference pixels are thinned out when performing H2V2 addition. FIG. [Figure 6] FIG. 13 is a diagram showing an example of a configuration in which a selection control line is doubled; [Figure 7] FIG. 13 is a diagram showing an example of drive control using two selection control lines in a three-exposure HDR·H2V2 addition method. [Figure 8] FIG. 13 is a diagram showing an example of drive control using two selection control lines in a three-exposure HDR·H2V2 addition method. [Figure 9] FIG. 13 is a diagram showing an example of readout driving in the near-adjacent four-pixel addition / H2V2 addition method. [Figure 10] FIG. 13 is a diagram showing an example of drive control using two selection control lines in the near-adjacent four-pixel addition / H2V2 addition method. [Figure 11] FIG. 13 is a diagram showing the pixel drive operation when the near-adjacent four-pixel addition / H2V2 addition method is used. [Figure 12] FIG. 13 is a diagram showing the pixel drive operation when the near-adjacent four-pixel addition / H2V2 addition method is used. [Figure 13] FIG. 13 is a diagram showing another example of a configuration in which the selection control line is doubled; [Figure 14] FIG. 13 is a diagram showing the correspondence between selection control lines and readout driving. [Figure 15] FIG. 13 is a diagram showing an example of read driving. [Figure 16] FIG. 2 is a diagram illustrating an example of the configuration of a light-shielding pixel. [Figure 17] 13 is a diagram showing another example of readout driving of a pixel unit arranged in a pixel array unit. FIG. [Figure 18] 13 is an enlarged view showing another example of readout driving of a pixel portion arranged in the pixel array portion. FIG. [Figure 19] 1 is a block diagram showing an example of the configuration of an electronic device equipped with a solid-state imaging device to which the technology according to the present disclosure is applied. [Figure 20] 1A and 1B are diagrams illustrating examples of use of a solid-state imaging device to which the technology according to the present disclosure is applied. [Figure 21] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Figure 22] 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit; FIG. [Figure 23] 1 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system. [Figure 24] 2 is a block diagram showing an example of a functional configuration of a camera head and a CCU. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the technology according to the present disclosure (the present technology) will be described with reference to the drawings. The description will be given in the following order.

[0015] 1. First embodiment 2. Second embodiment 3. Variations 4. Configuration of electronic devices 5. Examples of use of solid-state imaging devices 6. Mobile application examples 7. Application example to endoscopic surgery system

[0016] <1. First embodiment>

[0017] (Example of the configuration of a solid-state imaging device) FIG. 1 is a block diagram showing an example of the configuration of an embodiment of a solid-state imaging device to which the technology according to the present disclosure is applied.

[0018] 1 is composed of an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The solid-state imaging device 10 captures incident light (image light) from a subject via an optical lens system (not shown), converts the amount of incident light imaged on an imaging surface into an electrical signal on a pixel-by-pixel basis, and outputs the electrical signal as a pixel signal.

[0019] 1, a solid-state imaging device 10 includes a pixel array section 11, a vertical drive circuit 12, a column signal processing circuit 13, a horizontal drive circuit 14, an output circuit 15, a control circuit 16, and an input / output terminal 17.

[0020] A plurality of pixel sections 200 are arranged two-dimensionally (in a matrix) in the pixel array section 11. The pixel section 200 is composed of four pixels 100 (2×2=4 pixels) of the same color.

[0021] Each pixel unit 200 is configured with four pixels 100 of the same color, that is, a red (R) pixel, a green (G) pixel, or a blue (B) pixel, depending on a color filter that transmits light of the wavelength of the red (R) component, the green (G) component, or the blue (B) component. Also, the pixel unit 200 is configured as a shared pixel in which a pixel circuit is shared by the four pixels 100 of the same color.

[0022] In the pixel section 200, each pixel 100 of the same color is configured to have a photodiode (PD) as a photoelectric conversion element and a transfer transistor TR-Tr. In addition, in the pixel section 200, four pixels 100 of the same color share a reset transistor RST-Tr, an amplification transistor AMP-Tr, and a selection transistor SEL-Tr as a pixel circuit.

[0023] Although details will be described later, in the pixel array section 11, as the pixels 100 included in the pixel section 200, pixels for detecting a phase difference (hereinafter, referred to as phase difference pixels) are arranged in a scattered manner (in a repeating pattern).

[0024] The vertical drive circuit 12 is composed of a shift register or the like, selects a specific pixel drive line 121, and supplies a drive signal (pulse) to the selected pixel drive line 121 for driving the pixel 100 or pixel section 200, thereby driving the pixel 100 or pixel section 200 on a row-by-row basis.

[0025] That is, the vertical drive circuit 12 selects and scans each pixel 100 or each pixel section 200 of the pixel array section 11 in the vertical direction in a row-by-row manner, and supplies a pixel signal based on the charge (signal charge) generated in the photodiode of each pixel 100 according to the amount of light received to the column signal processing circuit 13 via the vertical signal line 131.

[0026] The column signal processing circuit 13 is arranged for each column of the pixel units 200, and performs signal processing such as noise removal for each pixel column on signals output from one row of the pixel units 200. For example, the column signal processing circuit 13 performs signal processing such as correlated double sampling (CDS) for removing fixed pattern noise specific to each pixel, and AD (Analog-Digital) conversion.

[0027] The horizontal drive circuit 14 is composed of a shift register or the like, and sequentially outputs horizontal scanning pulses to select each of the column signal processing circuits 13 in turn, and cause each of the column signal processing circuits 13 to output a pixel signal to a horizontal signal line 141.

[0028] The output circuit 15 processes and outputs signals sequentially supplied from each of the column signal processing circuits 13 through the horizontal signal line 141. Note that the output circuit 15 may perform only buffering, or may perform black level adjustment, column variation correction, various digital signal processing, and the like.

[0029] The control circuit 16 controls the operation of each part of the solid-state imaging device 10 .

[0030] Furthermore, the control circuit 16 generates clock signals and control signals that serve as a reference for the operations of the vertical drive circuit 12, the column signal processing circuit 13, the horizontal drive circuit 14, etc., based on the vertical synchronization signal, the horizontal synchronization signal, and the master clock signal. The control circuit 16 outputs the generated clock signals and control signals to the vertical drive circuit 12, the column signal processing circuit 13, the horizontal drive circuit 14, etc.

[0031] The input / output terminal 17 exchanges signals with the outside.

[0032] The solid-state imaging device 10 in Fig. 1 configured as above is a CMOS image sensor called a column AD type in which a column signal processing circuit 13 performing CDS processing and AD conversion processing is arranged for each pixel column. The solid-state imaging device 10 in Fig. 1 may be a back-illuminated CMOS image sensor or the like.

[0033] (Plane layout of pixels) FIG. 2 illustrates, as an example, 32 rows and 32 columns of pixels 100 arranged in a predetermined area when viewed from the light incident side, among the multiple pixels 100 arranged two-dimensionally in the pixel array section 11 of FIG.

[0034] In the following description, a color filter that transmits light of a red (R) component wavelength is provided as the color filter, and a pixel that obtains an electric charge corresponding to red (R) component light from light that has transmitted through this R color filter is referred to as an R pixel 100. The R pixel 100 is provided with a pattern of diagonal lines slanting upward to the right, and four adjacent pixels form an R pixel section 200A.

[0035] Also, as shown in the lower right frame in the figure, in the R pixel section 200A, the exposure time is adjusted for each of the four R pixels 100. Here, the exposure time is adjusted in three stages, T1, T2, and T3 (T1 > T2 > T3), and T1 is referred to as the long accumulation exposure time, T2 is referred to as the medium accumulation exposure time, and T3 is referred to as the short accumulation exposure time.

[0036] In the R pixel section 200A, of the four pixels, the upper left is the R L The pixel is 100, and the upper right and lower left are medium R. M The pixel becomes 100, and the bottom right is the R of the short accumulation. S The pixel becomes 100.

[0037] Moreover, a pixel that obtains an electric charge corresponding to green (G) component light from light that has passed through a color filter that transmits light of a green (G) component wavelength is referred to as a G pixel 100. A dot pattern is applied to the G pixel 100, and four adjacent pixels form a G pixel section 200A.

[0038] The G pixel section 200A has an exposure time adjusted in three stages, similar to the R pixel section 200A. Of the four pixels, the upper left is a G pixel with a long accumulation time. L The pixel is 100, and the upper right and lower left are medium-sized G M The pixel becomes 100, and the bottom right is G for short accumulation. S The pixel becomes 100.

[0039] Furthermore, a pixel that obtains an electric charge corresponding to blue (B) component light from light that has passed through a color filter that transmits light of a blue (B) component wavelength is referred to as a B pixel 100. The B pixel 100 is provided with a pattern of diagonal lines slanting downward to the right, and four adjacent pixels form a B pixel section 200A.

[0040] The B pixel section 200A has three exposure time adjustments, similar to the R pixel section 200A. Of the four pixels, the upper left is a B pixel with a long accumulation time. L The pixel is 100, and the upper right and lower left are medium B. M The pixel is 100, and the bottom right is B for short accumulation. S The pixel becomes 100.

[0041] Now, if attention is paid to the R pixel section 200A, the G pixel section 200A, and the B pixel section 200A in the pixel array section 11 in FIG. 2, these pixel sections 200A are regularly arranged in a Bayer array.

[0042] In other words, each pixel in the Bayer array is divided into 2 x 2 pixels, and each pixel is composed of four pixels of the same color. The Bayer array is an array pattern in which G pixels are arranged in a checkerboard pattern, and R and B pixels are arranged alternately in rows in the remaining area.

[0043] Furthermore, in the pixel array unit 11 of FIG. 2, pairs of the phase difference pixel 100L and the phase difference pixel 100R are arranged according to a predetermined rule.

[0044] Here, one on-chip lens (OCL: On Chip Lens) is provided for two pixels (2×1 pixels) in the horizontal direction as a set of phase difference pixels 100L and 100R (2×1 OCL structure). In other words, a plurality of photoelectric conversion elements such as photodiodes are embedded in one on-chip lens. Hereinafter, a pixel having a 2×1 OCL structure is also referred to as a 2×1 OCL pixel.

[0045] In this case, since the four pixels constituting the R, G, B pixel unit 200A have different exposure times for two pixels arranged in the horizontal direction, it is considered difficult to provide one on-chip lens for these two pixels (2 × 1 pixels) due to the characteristics of the phase difference pixels.

[0046] Therefore, here, an on-chip lens is disposed across the pixel units 200A adjacent to each other on the left and right, and the adjacent pixels 100 including either one of the two photoelectric conversion elements formed for the on-chip lens have a color filter of the same color and have the same exposure time. By using such a structure, it is possible to provide one on-chip lens for two pixels (2×1 pixels) in the horizontal direction.

[0047] For example, in the case of adjacent B pixel section 200A and G pixel section 200A, S The photoelectric conversion element of the pixel 100 and the G pixel section 200A at the lower left M When one on-chip lens is provided for the photoelectric conversion element of the pixel 100, the following structure is adopted.

[0048] That is, at the time of manufacturing, the B pixel section 200A at the lower right S The color filter of the pixel 100 is configured as a G color filter instead of a B color filter, so that the pixel 100 is configured as a G pixel 100. S The pixel 100 was a short accumulation pixel (S), but the exposure time is changed to a medium accumulation exposure time to become a medium accumulation pixel (M).

[0049] As a result, the G M The pixel 100 and the G pixel unit 200A at the lower left M One on-chip lens is provided for each pixel 100, and these G M Pixels 100 have the same G color filter and the same medium intensity exposure time.

[0050] Therefore, the G at the bottom right of the B pixel section 200A M The pixel 100 is a phase difference pixel 100L, and the G pixel unit 200A is a M The pixel 100 is configured as a phase difference pixel 100R, and the phase difference between two images can be detected based on a pixel signal obtained from a pair of the phase difference pixels 100L and 100R.

[0051] In addition, in the B pixel section 200A including the phase difference pixel 100L, S Pixel 100, G M If you change it to pixel 100, the short accumulation pixel (S) will no longer exist, but the B M Pixel 100 is B S By changing to pixel 100, the four pixels are made to include a short accumulation pixel (S) in addition to a long accumulation pixel (L) and a medium accumulation pixel (M).

[0052] In this manner, in the pixel array section 11 of FIG. 2, pairs of the phase difference pixel 100L and the phase difference pixel 100R included in the adjacent B pixel section 200A and G pixel section 200A are regularly arranged at predetermined pixel intervals in the horizontal and vertical directions.

[0053] (3-exposure HDR H2V2 additive method) FIG. 3 illustrates a case where a 3-exposure HDR·H2V2 addition method is implemented as readout driving of the pixel section 200A arranged in a Bayer array in the pixel array section 11 of FIG.

[0054] When using this 3-exposure HDR·H2V2 addition method, H2V2 addition is performed first. In H2V2 addition, pixel signals from pixels 100 of the same color and exposure are added together in two pixels in the horizontal direction and two pixels in the vertical direction (H2V2 addition) (A in Figure 3).

[0055] Specifically, the upper left R pixel of the R pixel section 200A L The pixel signal of pixel 100 is added for four pixels, and four R L The pixel signal of pixel 100 is added together to obtain a sum signal (FIG. 3B).

[0056] Similarly, the remaining three pixels in the R pixel section 200A, the four pixels in the G pixel section 200A, and the four pixels in the B pixel section 200A are added together by H2V2 to obtain an added signal for pixels 100 of the same color and exposure (B in Figure 3).

[0057] Then, following the H2V2 addition, 3-exposure HDR is performed. That is, since the pixel signal obtained by the H2V2 addition (analog addition) includes pixel signals from the long accumulation pixels (L), the medium accumulation pixels (M), and the short accumulation pixels (S) for each of the colors R, G, and B, in the 3-exposure HDR, the pixel signals having different exposure times are selected or synthesized to obtain a pixel signal of HDR (High Dynamic Range) (C in FIG. 3).

[0058] This triple-exposure HDR is a technology that expands the dynamic range by arranging three pixels with different exposure times within the same color pixel, and selecting or synthesizing a signal with the appropriate exposure from these three types of pixels.

[0059] Furthermore, in the pixel array of FIG. 2, one on-chip lens is formed to straddle the B pixel section 200A and the G pixel section 200A, and a pair of phase difference pixels 100L and 100R are disposed to perform phase difference detection.

[0060] Here, when the nearest pixel 100 of the same color and exposure is added to this pixel array by H2V2 addition, if it is assumed that the entire region is added uniformly, many normal pixels are added to the phase difference pixel (A in FIG. 4).

[0061] Therefore, the pixel array is a schematic representation of the pixel array after H2V2 addition in B of Fig. 4, and the density of the phase difference pixels 100L and 100R among the pixels included in the pixel array (all areas) is higher than that of the pixel array before H2V2 addition in A of Fig. 4. On the other hand, the signals of the normal pixels and the phase difference pixels are mixed, making it difficult to detect the phase difference.

[0062] Therefore, in the technology disclosed herein, when performing horizontal and vertical addition (H2V2 addition), the phase difference pixels 100L and 100R are thinned out so that the density of the phase difference pixels after the horizontal and vertical addition is the same or approximately the same density as, or lower than, the density of the phase difference pixels before the horizontal and vertical addition, thereby preventing the density of the phase difference pixels 100L and 100R from becoming too high (B in FIG. 5).

[0063] That is, when performing the H2V2 addition shown in A of FIG. 3, if at least one of the four selected pixels includes a phase difference pixel, then in some pixels, only the phase difference pixel is selected and added, or in other pixels, only normal pixels excluding the phase difference pixel are selected and added.

[0064] This makes it possible to adjust the density of the phase difference pixels and prevent deterioration of the phase difference signal. In addition, by controlling the density of the phase difference pixels before and after horizontal and vertical addition to be the same or approximately the same, it is possible to share the same signal processing pipeline.

[0065] Here, three patterns of addition methods are shown as examples.

[0066] The first pattern is a pattern in which the phase difference pixels (2×1 OCL pixels) are left during H2V2 addition. In this case, in order to leave the phase difference pixels, the pixel signals of the other normal pixels (12 normal pixels) are excluded by not transferring them to the vertical signal line 131 (not transferring them to VSL).

[0067] Hereinafter, in controlling the addition method of this first pattern, of the four pixel units 200A of the same color that are the subject of H2V2 addition, three pixel units 200A (12 normal pixels) excluding the pixel unit 200A that includes the phase difference pixel are not transferred to VSL and are not added, and therefore are referred to as "non-added pixel units."

[0068] The second pattern is a pattern in which phase difference pixels (2×1 OCL pixels) are discarded during H2V2 addition. That is, due to restrictions on the correction circuit of the phase difference pixels for the RAW pixels, the density of the phase difference pixels is appropriately eliminated.

[0069] In this case, the pixel unit 200A including the phase difference pixel and the pixel unit 200A located diagonally therefrom are excluded by not performing VSL transfer. Here, since the former pixel unit 200A includes a phase difference pixel, it is referred to as a "pixel unit discarding phase difference pixels," and the diagonal pixel unit 200A is referred to as a "pixel unit discarding phase difference pixels."

[0070] The reason for excluding the pixel unit 200A diagonally opposite the pixel unit 200A including the phase difference pixel, i.e., the “companying pixel unit” diagonally opposite the “pixel unit discarding phase difference pixels”, is to align the centers of gravity of the pixels after H2V2 addition.

[0071] The third pattern is a pattern in which four pixel units 200A (16 normal pixels) are added together. That is, here, normal H2V2 addition is performed, so that 16 normal pixels are transferred to the VSL.

[0072] In order to control the on / off of the above three patterns of VSL transfer, a selection control line (SEL line) for controlling the connection between the vertical signal line 131 and the floating diffusion region (FD: Floating Diffusion) is doubled to perform drive control.

[0073] Specifically, as shown in FIG. 6, a single selection control line (SEL) 122 (A in FIG. 6) for controlling the connection between the vertical signal line (VSL) 131 and the floating diffusion region (FD) 111 is doubled into two selection control lines (SEL) 122-1 and 122-2 (B in FIG. 6), thereby controlling the on / off of the VSL transfer according to the above three patterns.

[0074] The selection control lines 122-1 and 122-2 are included in the pixel drive line 121 (FIG. 1), and a drive signal is applied from the vertical drive circuit 12 or the control circuit 16 or an external control device (not shown). The selection transistors SEL-Tr connected to the selection control lines 122-1 and 122-2 operate in accordance with the drive signal, thereby controlling the on / off of the VSL transfer.

[0075] Specifically, when performing H2V2 addition, as shown in FIG. 7, it is assumed that, among the phase difference pixels (2×1 OCL pixels) represented by white squares in the figure, a pair of phase difference pixels (100L, 100R) surrounded by a frame 210 is kept, and the other phase difference pixels (2×1 OCL pixels) are discarded.

[0076] In this case, by controlling the driving of the selection control lines 122-1 and 122-2 so that the VSL transfer is turned off at the positions indicated by the symbols in the figure (symbols of circles with a line through them), it is possible to leave only the phase difference pixels within the frame 210.

[0077] That is, as shown in Fig. 8, "1" in the figure indicates that the control of the first pattern addition method is performed, and the phase difference pixels in the frame 210 are left, but the three pixel units 200A (12 normal pixels) excluding the pixel unit 200A including the phase difference pixels are not transferred to the VSL and are not added. That is, "C" in the figure indicates a "non-added pixel unit."

[0078] In addition, the phase difference pixels here are selected on a pixel-by-pixel basis by driving control of the selection control lines 112-1 and 112-2. In addition, in FIG. 7 and FIG. 8, a driving signal is applied to the transfer control line (TRG line) to select the pixel 100 in the pixel unit 200A.

[0079] 8, "2" in the figure indicates that the control of the addition method of the second pattern is performed, and the pixel unit 200A including the phase difference pixel is not read, and furthermore, the pixel unit 200A located diagonally therefrom is not read either. In other words, "A" in the figure is a "pixel unit discarding phase difference pixels," and "B" in the figure is a "pixel unit including phase difference pixels," and both of these pixel units are not added.

[0080] Furthermore, in FIG. 8, "3" in the drawing indicates that control of the third pattern of addition method is performed, and normal H2V2 addition is performed in the four pixel units 200A (16 normal pixels).

[0081] In this way, by double-tracking the selection control line (SEL line) to control the on / off of the VSL transfer and controlling the addition method of the first pattern to the third pattern, it is possible to reduce the density of the phase difference pixels while preventing the phase difference pixels and normal pixels from being added together.

[0082] Furthermore, in the technology disclosed herein, two selection control lines (SEL lines) are provided, and the connection of the selection control line is changed for each floating diffusion region (FD) 111 to control the drive signals applied to the selection control lines 122-1 and 122-2, thereby avoiding averaging of the signal level in the vertical signal line (VSL) 131. Note that there is usually one selection control line, and two floating diffusion regions (FD) 111 in the vertical direction are connected to the same vertical signal line (VSL) 131, so that the signal level is averaged in the vertical signal line (VSL) 131.

[0083] Note that, although a configuration having two rows of AD converters each including comparators 151 has been illustrated here as an example, a similar approach can be used to achieve high phase difference detection performance and image quality during H2V2 addition when using the three-exposure HDR·H2V2 addition method even when other configurations are used.

[0084] Here, in order to realize low power consumption driving, it is desirable to satisfy three requirements in terms of implementation.

[0085] The first requirement is that the problem must be solved using the existing defective pixel circuit currently in use. In other words, if this requirement cannot be met, it will be necessary to implement a new countermeasure circuit in the existing defective pixel circuit, which will lead to an increase in circuit size or a decrease in yield.

[0086] In this regard, in the technology disclosed herein, the density of the phase difference pixels 100L, 100R (the density of 2×1 OCL pixels) is reduced by controlling the drive signals applied to the two selection control lines through multiplication by H2V2 addition, so that it can be handled by an existing defective pixel circuit and the first requirement is met.

[0087] The second requirement is that the necessary phase difference pixels 100L and 100R (2×1 OCL pixels) must be left alone to perform image plane phase difference AF (PDAF: Phase Detection Auto Focus).

[0088] In this regard, the technology disclosed herein controls the drive signals applied to two selection control lines to select the pixels to be added, thereby making it possible to leave the phase difference pixels 100L, 100R (2 × 1 OCL pixels) used in image-plane phase difference AF as single pixels without adding them, and to add multiple pixels for the other surrounding pixels, thereby satisfying the second requirement.

[0089] The third requirement is that the center of gravity position must be aligned after analog addition.

[0090] In this regard, the technology disclosed herein achieves readout drive with lower power consumption by pixel addition using H2V2 addition, but by controlling the drive signals applied to two selection control lines, it is possible to select the pixels to be added so that the centers of gravity are aligned after addition (such as ``pixel section that discards phase difference pixels'' versus ``pixel section that takes phase difference pixels with it''), and it is possible to incorporate center of gravity shift correction in the added pixels into the drive, thereby satisfying the third requirement.

[0091] As described above, while there is usually only one selection control line, the technology disclosed herein provides two selection control lines and introduces a physical specification (switching of contacts to the selection control line) that can be controlled for each floating diffusion region (FD), thereby simultaneously solving three problems: phase difference pixel correction, phase difference detection, and center of gravity correction.

[0092] <2. Second embodiment>

[0093] (Adding 4 adjacent pixels, H2V2 adding method) 9 shows a case where the adjacent four-pixel addition / H2V2 addition method is implemented as the readout drive of the pixel section 200B arranged in a Bayer array in the pixel array section 11. However, the exposure time is the same for all four pixels in each pixel section 200B in the pixel array section 11, i.e., all pixels 100 of the same color (such as a medium exposure time).

[0094] When using this near-adjacent four-pixel addition / H2V2 addition method, near-adjacent four-pixel addition is performed first. In near-adjacent four-pixel addition, the four pixels (shared pixels) constituting the R, G, B pixel section 200B are each considered to be near-adjacent four pixels, and the pixel signals from these four pixels are added in the floating diffusion region 111 (FD addition) to generate sum signals of the R component, G component, and B component, respectively.

[0095] Then, following the near-adjacent four-pixel addition, H2V2 addition is performed, in which sum signals from pixel units 200B of the same color are further added together at two pixels in the horizontal direction and two pixels in the vertical direction.

[0096] Specifically, in the R pixel section 200B, the four R pixels 100 corresponding to the four white circles at the four corners of the grid 311R in A of Fig. 9 are treated as four neighboring adjacent pixels, and pixel addition (analog addition) is performed. Also, by H2V2 addition, the sum signals obtained from the four neighboring adjacent pixels at the four corners of the grid 311R in A of Fig. 9 are further added (analog addition) in the horizontal and vertical directions. As a result, a sum signal of the R component is obtained (the sum signal at the bottom left of B of Fig. 9).

[0097] Similarly, the G pixel section 200B performs near-adjacent four-pixel addition and H2V2 addition in response to the white circle in grid 311G in A of Fig. 9, and obtains a G component sum signal (the upper left or lower right sum signal in B of Fig. 9). Also, the B pixel section 200B performs near-adjacent four-pixel addition and H2V2 addition in response to the white circle in grid 311B in A of Fig. 9, and obtains a B component sum signal (the upper right sum signal in B of Fig. 9).

[0098] Here, also in the pixel array of FIG. 9, one on-chip lens is formed to straddle the B pixel section 200B and the G pixel section 200B, and a pair of phase difference pixels 100L and 100R are arranged to perform phase difference detection.

[0099] Assuming that the entire region of the nearest pixel unit 200B of the same color is uniformly added to this pixel array when performing H2V2 addition, many normal pixels are added to the phase difference pixels. Therefore, after H2V2 addition, the density of the phase difference pixels 100L and 100R among the pixels included in the pixel array (entire region) becomes high. On the other hand, the signals of the normal pixels and the phase difference pixels are mixed, making it difficult to detect the phase difference.

[0100] Therefore, in the near-adjacent four-pixel addition·H2V2 addition method, similar to the above-mentioned three-exposure HDR·H2V2 addition method, when performing H2V2 addition, the phase difference pixels 100L, 100R are thinned out to prevent the density of the phase difference pixels 100L, 100R from becoming too high.

[0101] That is, when performing H2V2 addition shown in A of Fig. 9, if at least one of the four selected pixels includes a phase difference pixel, only the phase difference pixel is selected and added in some pixels, or only normal pixels excluding the phase difference pixel are selected and added in other pixels. This makes it possible to adjust the density of the phase difference pixels and prevent deterioration of the phase difference signal.

[0102] In addition, in the adjacent four pixel addition / H2V2 addition method, similar to the above-mentioned three-exposure HDR / H2V2 addition method, in order to control the on / off of VSL transfer in three patterns, the first pattern, the second pattern, and the third pattern, the selection control line 122 for controlling the connection between the vertical signal line (VSL) 131 and the floating diffusion region (FD) 111 is doubled.

[0103] Specifically, as shown in FIG. 10, it is assumed that phase difference pixels (2×1 OCL pixels) surrounded by a frame 210 are left and the other phase difference pixels (2×1 OCL pixels) are discarded.

[0104] In this case, "1" in the figure indicates that the control of the first pattern addition method is performed, and the phase difference pixels in the frame 210 are left, but the three pixel units 200B (12 normal pixels) excluding the pixel unit 200B including the phase difference pixels are not transferred to the VSL and are not added. In other words, "C" in the figure indicates a "non-added pixel unit."

[0105] 10, "2" in the figure indicates that the second pattern of the addition method is controlled, and the pixel unit 200B including the phase difference pixel is not read, and the pixel unit 200B located diagonally therefrom is not read either. In other words, "A" in the figure indicates a "pixel unit that discards phase difference pixels," and "B" in the figure indicates a "pixel unit that includes the phase difference pixels."

[0106] Furthermore, in Fig. 10, "3" in the figure indicates that the control of the third pattern of addition method is performed, and normal H2V2 addition is performed in four pixel units 200B (16 normal pixels). In Fig. 10, a drive signal is applied to the transfer control line (TRG line) and a pixel 100 in the pixel unit 200B is selected.

[0107] In this way, by double-tracking the selection control line to control the on / off of the VSL transfer and controlling the addition method of the first pattern to the third pattern, it is possible to reduce the density of the phase difference pixels while preventing the phase difference pixels and normal pixels from being added together.

[0108] Here, in the near-adjacent four-pixel addition / H2V2 addition method, when the near-adjacent four-pixel addition is performed, the driving shown in FIG. 11 and FIG. 12 is performed to support readout of the phase difference pixels.

[0109] 11 and 12, among the four pixel units 200B arranged in the horizontal direction, the two central B and G pixel units 200B each include phase difference pixels 100L and 100R in addition to normal pixels. On the other hand, in the left and right G and B pixel units 200B, four pixels are normal pixels.

[0110] When performing adjacent four pixel addition, as shown in FIG. 11, the charge accumulated in the photoelectric conversion element of one of the four pixels constituting each pixel unit 200B is transferred to a different floating diffusion region 111 for each pixel unit 200B (S1 in FIG. 11).

[0111] At this time, the pixel signals obtained from the phase difference pixels 100L and 100R of the central B / G pixel unit 200B are held (S2 in FIG. 11).

[0112] Thereafter, as shown in FIG. 12, the charges accumulated in the photoelectric conversion elements of the four pixels constituting each pixel unit 200B are transferred to the floating diffusion region 111 (S4-1 to S4-3 in FIG. 12).

[0113] In FIG. 12, the pixels that were the subject of transfer in FIG. 11 are indicated by black squares, and these pixels will also be retransferred.

[0114] At this time, of the four pixel sections 200B arranged horizontally, the left and right G and B pixel sections 200B are composed only of normal pixels, so four pixels are added together (S5 in FIG. 12), and the resulting added signal is subjected to subsequent processing (S6 in FIG. 12).

[0115] In addition, the central B / G pixel section 200B includes the phase difference pixels 100L and 100R in addition to the normal pixels, which are B and G pixels. Therefore, the sum signal obtained by adding four pixels is not used (S5 in FIG. 12), and subsequent processing is performed on the pixel signals from the held phase difference pixels 100L and 100R (S3 in FIGS. 11 and 12).

[0116] In this way, when performing the near-adjacent four-pixel addition, it is also possible to handle readout of phase difference pixels, and then H2V2 addition is performed.

[0117] <3. Modifications>

[0118] (Extended selection control line) In the above explanation, a configuration was shown in which the selection control line is doubled by increasing the selection control line to two to control the on / off of VSL transfer, but the number of selection control lines is not limited to two and more selection control lines, such as three, can be provided.

[0119] Specifically, as shown in FIG. 13, one selection control line (SEL) 122 (A in FIG. 13) can be doubled into three selection control lines (SEL) 122-1, 122-2, and 122-3 (B in FIG. 13) to control the connection between the vertical signal line (VSL) 131 and the floating diffusion region (FD) 111.

[0120] Here, by doubling the selection control line 122 to three lines, it is possible to realize seven types of readout corresponding to Case 0 to Case 6, as shown in FIG.

[0121] That is, in Figure 14, three selection control lines are respectively associated with Case 0 to Case 6, with an "O" indicating that the target selection control line 122 is driven, and an "X" indicating that the target selection control line 122 is not driven.

[0122] Specifically, when all three selection control lines 122-1 to 122-3 are driven, four pixel units 200 of the same color in frame 401 of A in Fig. 15 can be read out and their pixel signals can be added together as Case 0. Also, for Cases 1 to 6, similarly to Case 0, pixel units 200 of the same color are read out in response to the driving of the three selection control lines 122-1 to 122-3 and are added together (FD addition) in the floating diffusion region 111 (B to G in Fig. 15).

[0123] As described above, in the current technology, the connection from the floating diffusion region (FD) 111 to the vertical signal line (VSL) 131 is determined by the physical configuration, so the SF (Source Follower) addition method can only be set in one way. In the technology according to the present disclosure, the selection control line is multi-tracked and the driving thereof is controlled, so that the variation of SF addition can be controlled.

[0124] (Example of light-shielding pixel configuration) In addition, in the above description, a case has been described in which 2×1 OCL pixels are used as phase difference pixels, but other phase difference pixels such as light-shielding pixels may also be used.

[0125] Here, a light-shielding pixel 100L and a light-shielding pixel 100R whose light-shielding regions are symmetrical can be used as shown in Fig. 16. That is, the light-shielding pixel 100L and the light-shielding pixel 100R are paired as a phase difference pixel, and a phase difference is detected based on light-shielding pixel signals obtained from the light-shielding pixels 100L and 100R.

[0126] (Example of left-right interleaved structure) Incidentally, in order to realize a high frame rate, there is a horizontal interleave structure that is used as a method for simultaneously reading out twice as many rows as the vertical signal lines (VSL) in parallel. In addition, in the above description, in the pixel array section 11 in which the pixel section 200 having four pixels of the same color is regularly arranged, when the phase difference pixels 100L and 100R are arranged according to a predetermined rule, a method for adjusting the density of the phase difference pixels 100L and 100R when performing drive control by the H2V2 addition method has been shown.

[0127] A configuration that satisfies these two methods simultaneously and is capable of reading out pixel signals in predetermined pixel units such as 2×2 pixels will be described below.

[0128] Fig. 17 shows another example of readout driving of pixel units 200 arranged in a Bayer array in the pixel array section 11. For ease of explanation, Fig. 18 shows an enlarged view of a part of the region in which the pixel units 200 shown in Fig. 17 are arranged.

[0129] In the pixel array unit 11, all four pixels constituting the pixel unit 200 are the same color pixels 100. In the pixel array unit 11, the phase difference pixels 100L and 100R are periodically arranged in a predetermined pixel unit such as a 16×16 pixel unit.

[0130] The pixel array unit 11 employs a left-right interleaved structure. The left-right interleaved structure is a structure in which pixel signals are read out by dividing the array into left and right parts, and a vertical signal line (VSL) 131 to which two specific floating diffusion regions (FD) 111 are connected is shared. Since the vertical signal line (VSL) 131 is shared, which column is to be read out is selected by switch control according to a drive signal (SEL signal) applied to a selection control line (SEL line).

[0131] 18, for multiple vertical signal lines (VSL) 131, there are floating diffusion regions (FD) 111 shared by the upper and lower pixel units 200 on the left side and floating diffusion regions (FD) 111 shared by the upper and lower pixel units 200 on the right side, and the vertical signal line (VSL) 131 to which two floating diffusion regions (FD) 111 in different columns on the left and right sides are connected is common. However, in the floating diffusion regions (FD) 111 in the left and right columns connected to the common vertical signal line (VSL) 131, the shared upper and lower pixel units 200 are each made the same color and are arranged in the same row.

[0132] As described above, in the pixel array section 11, the floating diffusion regions (FD) 111 of two columns (the left column and the right column) share the vertical signal line (VSL) 131. Therefore, in the above-mentioned driving control, in a mode (pattern) in which SF addition is not performed, the left and right floating diffusion regions (FD) 111 are read out separately.

[0133] Here, assume that the above-mentioned drive control is performed on a type having two vertical signal lines (VSL) 131 for each column of the floating diffusion region (FD) 111. In this case, the number of rows that can be read simultaneously is four rows (FD rows) when a non-left-right interleaved structure is used, whereas the number of rows that can be read simultaneously is eight rows (FD rows) when a left-right interleaved structure is used, which is twice as many rows as the non-left-right interleaved structure. In other words, the left-right interleaved structure is a structure that is devised to read out at a high frame rate.

[0134] In the pixel array section 11, among the pixel sections 200 sharing the same floating diffusion region (FD) 111, the pixel section 200 consisting of four pixels, that is, R, G, or B pixels 100 corresponding to the upper R, G, or B color filter, is referred to as the top pixel section 200, and the pixel section 200 consisting of four pixels, that is, R, G, or B pixels 100 corresponding to the lower R, G, or B color filter, is referred to as the bottom pixel section 200.

[0135] When reading out the top pixel section 200, the floating diffusion region (FD) 111 that is not to be added is connected to a selection control line that is SELQBINSF, and the other floating diffusion regions (FD) 111 are connected to a selection control line that is SEL. When using a left-right interleaved structure, it is additionally necessary to control the left column and the right column independently, so selection control lines that are SEL and SELQBINSF are prepared for the left column and the right column, respectively.

[0136] 17 and 18, "*_EVN" is written in the selection control line for the left column, and "*_ODD" is written in the selection control line for the right column. In other words, four types of selection control lines (SEL lines) are prepared: SEL_EVN, SELQBINSF_EVN, SEL_ODD, and SELQBINSF_ODD.

[0137] Furthermore, since the transfer control lines (TRG lines) that select the pixels 100 in the pixel section 200 that share the floating diffusion region (FD) 111 need to be controlled independently for the left and right columns, they are prepared separately for the left and right columns, similar to the selection control lines (SEL lines).

[0138] 17 and 18, "*_T" is written on the transfer control line for the top pixel unit, "*_B" is written on the transfer control line for the bottom pixel unit, and "*_ZAF" is written on the transfer control line for the adjacent pixels of the phase difference pixels 100L and 100R. In other words, six types of transfer control lines (TRG lines) are prepared: TRG_EVN_T, TRG_EVN_B, TRG_EVN_ZAF, TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF.

[0139] These transfer control lines and selection control lines can be summarized as follows:

[0140] (Transfer control line) A drive signal (TRG signal) for transfer control (TRG control) of the pixel 100 in the top pixel unit 200 in the left column is applied (output) to TRG_EVN_T. A drive signal for transfer control of the pixel 100 in the bottom pixel unit 200 in the left column is applied to TRG_EVN_B.

[0141] A drive signal for transfer control of the pixel 100 in the top pixel unit 200 in the right column is applied to TRG_ODD_T. A drive signal for transfer control of the pixel 100 in the bottom pixel unit 200 in the right column is applied to TRG_ODD_B.

[0142] A drive signal for transfer control of the pixel 100 adjacent to the phase difference pixels 100L and 100R on the left column is applied to TRG_EVN_ZAF. A drive signal for transfer control of the pixel 100 adjacent to the phase difference pixels 100L and 100R on the right column is applied to TRG_ODD_ZAF.

[0143] (Selection control line) A drive signal (SEL signal) that is established by logic according to the signal levels of the transfer control lines TRG_EVN_T, TRG_EVN_B, and TRG_EVN_ZAF is applied (output) to SEL_EVN. That is, in SEL_EVN, when "TRG_EVN_*" is in the Read state, an output is made according to the input signal that is RSEL, while when "TRG_EVN_*" is in a state other than Read, the output is fixed to a Low level.

[0144] A drive signal that is established by logic corresponding to the signal levels of the transfer control lines TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF is applied to SEL_ODD. That is, in SEL_ODD, when "TRG_ODD_*" is in a Read state, an output is made according to the input signal RSEL, while when "TRG_ODD_*" is in a state other than Read, the output is fixed to a Low level.

[0145] A drive signal that is established by logic corresponding to the signal levels of the transfer control lines TRG_EVN_T, TRG_EVN_B, and TRG_EVN_ZAF is applied to SELQBINSF_EVN. That is, in SELQBINSF_EVN, when "TRG_EVN_*" is in a Read state, an output is made according to the input signal RSELQBINSF, while when "TRG_EVN_*" is in a state other than Read, the output is fixed to a Low level. Note that the input signal RSELQBINSF is required to be fixed to a Low level when, for example, the floating diffusion region (FD) 111 that is not the object of addition is read by the drive control described above.

[0146] A drive signal that is established by logic corresponding to the signal levels of the transfer control lines TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF is applied to SELQBINSF_ODD. That is, in SELQBINSF_ODD, when "TRG_ODD_*" is in a Read state, an output is made according to the input signal RSELQBINSF, while when "TRG_ODD_*" is in a state other than Read, the output is fixed to a Low level. Note that the input signal RSELQBINSF is required to be fixed to a Low level when, for example, the floating diffusion region (FD) 111 that is not the target of addition is read by the drive control described above.

[0147] In this way, when performing the above-mentioned drive control, the drive signals (TRG signal, SEL signal) applied to the transfer control lines (TRG_EVN_B, TRG_EVN_T, TRG_EVN_ZAF, TRG_ODD_B, TRG_ODD_T, TRG_ODD_ZAF) and the selection control lines (SEL_EVN, SELQBINSF_EVN, SEL_ODD, SELQBINSF_ODD) are controlled so that the phase difference pixels 100L, 100R, which cause defects in the captured image, are thinned out and read out at a certain period so that the density of the phase difference pixels 100L, 100R is not too high.

[0148] At this time, by controlling the driving signal applied to the selection control line to thin out the pixels 100 diagonally opposite to the thinned out phase difference pixels 100L and 100R, the centers of gravity of the pixels after H2V2 addition are aligned, so that the defect density can be suppressed while keeping the color center in the center. The method of thinning out is as described in the above-mentioned drive control.

[0149] 17 and 18, the above-mentioned drive control can be realized at a high frame rate with fewer vertical signal lines (VSL), and further, the density of point defects caused by phase difference pixels in the captured image can be controlled. Furthermore, pixel signals can be read out in a predetermined pixel unit such as 2×2 pixels.

[0150] (Examples of other methods) In the above description, the 3-exposure HDR·H2V2 addition method and the adjacent 4 pixel addition·H2V2 addition method are described as examples, but other methods using H2V2 addition may be used. Furthermore, H2V2 addition is an example of horizontal / vertical addition, and is not limited to addition of two pixels in the horizontal direction and two pixels in the vertical direction, but may be any method that adds a pixel signal of a predetermined pixel in the horizontal direction and a pixel signal of a predetermined pixel in the vertical direction horizontally and vertically.

[0151] (Achieving both high frame rate driving and PDAF) To achieve a high frame rate, it is necessary to reduce the total number of pixels by pixel addition and increase the number of simultaneous AD conversions in the horizontal direction. On the other hand, to achieve both high frame rate (multiple pixel addition) driving and image plane phase difference AF (PDAF), it is necessary to drive in a way that leaves the phase difference pixels (2 x 1 OCL pixels) when adding pixels. The current method of achieving this is to use one selection control line for each FD during SF addition, and use Mux control to select only the phase difference pixels (2 x 1 OCL pixels) before AD conversion.

[0152] This implementation method imposes a constraint on increasing the number of simultaneous AD conversions in the horizontal direction, and this issue needs to be resolved; however, the technology disclosed herein can resolve this issue by duplexing the selection control line.

[0153] As described above, according to the technology disclosed herein, in the case where a configuration including the phase difference pixels 100L, 100R is used for the multiple pixels 100 (including the pixel unit 200) arranged two-dimensionally in the pixel array section 11, when reading out the multiple pixels 100, when pixel signals of predetermined pixels 100 in the horizontal direction and pixel signals of predetermined pixels 100 in the vertical direction are added horizontally and vertically, degradation of phase difference information can be suppressed by partially not adding the phase difference pixels 100L, 100R.

[0154] In particular, image sensors for mobile devices in recent years have become increasingly multi-pixel, and image reduction technology is required to support video formats at high speed and with low power consumption. When performing image reduction techniques such as adjacent 4-pixel addition and H2V2 addition, phase difference pixels are currently added to normal pixels, resulting in the loss of phase difference information and adversely affecting image quality. In addition, when further reduction is performed after using 3-exposure HDR, pixel thinning (H2V2 thinning) is performed, which causes problems in image quality such as degradation of the signal-noise ratio (SN ratio) and aliasing.

[0155] In contrast, in the technology disclosed herein, when performing H2V2 addition using the 3-exposure HDR·H2V2 addition method or the adjacent 4-pixel addition·H2V2 addition method, the density of phase difference pixels is prevented from increasing too much even when the image is reduced by mixing an area where only phase difference pixels are added or selected with an area where normal pixels are added or selected together. This makes it possible to achieve high-speed, low-power operation while minimizing image quality degradation without losing phase difference information even when the image is reduced.

[0156] In addition, in the technology disclosed herein, when using the 3-exposure HDR·H2V2 addition method, the same exposure signals are added in analog while maintaining the same shutter control as the current 3-exposure HDR, which makes it possible to improve the signal-to-noise ratio, increase the readout speed, and further reduce power consumption.

[0157] However, if the above-mentioned H2V2 addition is performed uniformly for all pixels, when phase difference pixels are embedded, the phase difference information is deteriorated by adding the phase difference pixels and normal pixels, and the density of pixels affected by the phase difference pixels increases, and there is a concern that the correction may have a negative effect on image quality. In order to avoid this problem, the above-mentioned double-lined selection control line is driven and controlled to suppress the deterioration of phase difference information and image quality.

[0158] In addition, the above-mentioned Patent Document 1 discloses a phase difference detection and an addition method for achieving HDR when detecting phase differences over the entire surface, but does not disclose or even suggest a method for selectively adding embedded phase difference pixels.

[0159] In addition, the above-mentioned Patent Document 2 discloses arranging phase difference pixels in accordance with the summation method, but does not disclose or suggest a configuration for selecting a summation pixel by wiring and drive control of a selection control line as in the technology disclosed herein.

[0160] <4. Configuration of electronic devices>

[0161] FIG. 19 is a block diagram showing an example configuration of an electronic device having a solid-state imaging device to which the technology according to the present disclosure is applied.

[0162] The electronic device 1000 is, for example, an imaging device such as a digital still camera or a video camera, or an electronic device having an imaging function such as a mobile terminal device such as a smartphone or a tablet terminal.

[0163] The electronic device 1000 comprises a lens unit 1011 , an imaging unit 1012 , a signal processing unit 1013 , a control unit 1014 , a display unit 1015 , a recording unit 1016 , an operation unit 1017 , a communication unit 1018 , a power supply unit 1019 , and a drive unit 1020 .

[0164] In the electronic device 1000 , the signal processing unit 1013 , the control unit 1014 , the display unit 1015 , the recording unit 1016 , the operation unit 1017 , the communication unit 1018 , and the power supply unit 1019 are connected to each other via a bus 1021 .

[0165] The lens unit 1011 is composed of a zoom lens, a focus lens, etc., and collects light from a subject. The light collected by the lens unit 1011 (subject light) is incident on the imaging unit 1012.

[0166] The imaging unit 1012 includes a solid-state imaging device to which the technology according to the present disclosure is applied (for example, the solid-state imaging device 10 in FIG. 1). The imaging unit 1012 photoelectrically converts light (subject light) received by the solid-state imaging device through the lens unit 1011 into an electrical signal, and supplies the resulting signal to the signal processing unit 1013.

[0167] The pixel array section of this solid-state imaging device includes a plurality of pixels regularly arranged in a predetermined array pattern, including pixels (normal pixels) that generate signals for generating an image corresponding to subject light, and pixels (phase difference pixels) that generate signals for performing phase difference detection.

[0168] For example, in the above-mentioned solid-state imaging device 10 (FIG. 1), the normal pixels correspond to the R pixel 100 (R pixel section 200), the G pixel 100 (G pixel section 200), and the B pixel 100 (B pixel section 200), and the phase difference pixels correspond to the phase difference pixels 100L, 100R and the light-shielding pixels 100L, 100R.

[0169] The signal processing unit 1013 is a signal processing circuit that processes a signal supplied from the imaging unit 1012. For example, the signal processing unit 1013 is configured as a DSP (Digital Signal Processor) circuit or the like.

[0170] The signal processing unit 1013 processes the signal from the imaging unit 1012 to generate image data of a still image or a video, and supplies the image data to the display unit 1015 or the recording unit 1016. The signal processing unit 1013 also generates data for detecting a phase difference (phase difference detection data) based on a signal from the imaging unit 1012 (phase difference pixels of the image sensor), and supplies the data to the control unit 1014.

[0171] The control unit 1014 is configured as, for example, a CPU (Central Processing Unit), a microprocessor, etc. The control unit 1014 controls the operation of each unit of the electronic device 1000.

[0172] The display unit 1015 is configured as a display device such as a liquid crystal panel, an organic EL (Electro Luminescence) panel, etc. The display unit 1015 processes image data supplied from the signal processing unit 1013, and displays a still image or a moving image captured by the imaging unit 1012.

[0173] The recording unit 1016 is configured as a recording medium such as a semiconductor memory, a hard disk, etc. The recording unit 1016 records image data supplied from the signal processing unit 1013. The recording unit 1016 also provides the recorded image data under the control of the control unit 1014.

[0174] The operation unit 1017 is configured as a touch panel in combination with the display unit 1015 in addition to physical buttons. The operation unit 1017 outputs operation commands for various functions of the electronic device 1000 in response to an operation by a user. The control unit 1014 controls the operation of each unit based on the operation commands supplied from the operation unit 1017.

[0175] The communication unit 1018 is configured as, for example, a communication interface circuit, etc. The communication unit 1018 exchanges data with an external device through wireless communication or wired communication in accordance with a predetermined communication standard.

[0176] The power supply unit 1019 appropriately supplies various types of power to the signal processing unit 1013, control unit 1014, display unit 1015, recording unit 1016, operation unit 1017, and communication unit 1018 to these power sources.

[0177] Furthermore, the control unit 1014 detects the phase difference between the two images based on the phase difference detection data supplied from the signal processing unit 1013. Then, the control unit 1014 determines whether or not the object to be focused on (focus target object) is in focus based on the phase difference detection result. If the focus target object is not in focus, the control unit 1014 calculates the amount of focus deviation (defocus amount) and supplies it to the driving unit 1020.

[0178] The driving unit 1020 is composed of, for example, a motor and drives the lens unit 1011 which is composed of a zoom lens, a focus lens and the like.

[0179] The driving unit 1020 calculates the driving amount of the focus lens of the lens unit 1011 based on the defocus amount supplied from the control unit 1014, and moves the focus lens according to the driving amount. When the focus is on the object to be focused, the driving unit 1020 maintains the current position of the focus lens.

[0180] The electronic device 1000 is configured as described above.

[0181] <5. Examples of use of solid-state imaging devices>

[0182] FIG. 20 is a diagram showing an example of use of a solid-state imaging device to which the technology according to the present disclosure is applied.

[0183] The solid-state imaging device 10 can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as described below. That is, as shown in Fig. 20, the solid-state imaging device 10 can be used not only in the field of appreciation for capturing images for appreciation, but also in devices used in the fields of transportation, home appliances, medicine and health care, security, beauty, sports, agriculture, and the like.

[0184] Specifically, in the field of appreciation, the solid-state imaging device 10 can be used in devices for capturing images for appreciation purposes (e.g., electronic device 1000 in FIG. 19), such as digital cameras, smartphones, and mobile phones with camera functions.

[0185] In the field of transportation, for example, the solid-state imaging device 10 can be used in devices used in transportation, such as on-board sensors that capture images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and for recognition of the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles, etc.

[0186] In the field of home appliances, for example, the solid-state imaging device 10 can be used in devices provided for home appliances such as television receivers, refrigerators, and air conditioners to capture user gestures and operate the appliances according to the gestures. Also, in the field of medicine and healthcare, for example, the solid-state imaging device 10 can be used in devices provided for medical and healthcare purposes such as endoscopes and devices for capturing blood vessel images by receiving infrared light.

[0187] In the field of security, the solid-state imaging device 10 can be used in devices for security purposes, such as surveillance cameras for crime prevention, cameras for person authentication, etc. In the field of beauty, the solid-state imaging device 10 can be used in devices for beauty purposes, such as skin measuring devices for photographing the skin, and microscopes for photographing the scalp.

[0188] In the field of sports, the solid-state imaging device 10 can be used in devices provided for sports, such as action cameras and wearable cameras for sports applications, etc. In the field of agriculture, the solid-state imaging device 10 can be used in devices provided for agricultural purposes, such as cameras for monitoring the conditions of fields and crops.

[0189] <6. Application examples for mobile devices>

[0190] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.

[0191] FIG. 21 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.

[0192] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 21, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.

[0193] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0194] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0195] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.

[0196] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0197] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.

[0198] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0199] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0200] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0201] The audio / video output unit 12052 transmits at least one output signal of audio and video to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 21, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

[0202] FIG. 22 is a diagram showing an example of the installation position of the imaging unit 12031.

[0203] In FIG. 22, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0204] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0205] 22 shows an example of the imaging ranges of the imaging units 12101 to 12104. An imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and an imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.

[0206] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.

[0207] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.

[0208] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0209] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

[0210] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 among the configurations described above. Specifically, the solid-state imaging device 10 can be applied to the imaging units 12101 to 12105. By applying the technology according to the present disclosure to the imaging unit 12031, for example, deterioration of phase difference information and deterioration of image quality can be suppressed, thereby improving visibility and enabling more accurate recognition of objects such as people, cars, obstacles, signs, or characters on the road surface.

[0211] <7. Application examples for endoscopic surgery systems>

[0212] FIG. 23 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0213] 23 shows a state in which an operator (doctor) 11131 is performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 is composed of an endoscope 11100, other surgical tools 11110 such as an insufflation tube 11111 and an energy treatment tool 11112, a support arm device 11120 that supports the endoscope 11100, and a cart 11200 on which various devices for endoscopic surgery are mounted.

[0214] The endoscope 11100 is composed of a lens barrel 11101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 11132, and a camera head 11102 connected to the base end of the lens barrel 11101. In the illustrated example, the endoscope 11100 is configured as a so-called rigid scope having a rigid lens barrel 11101, but the endoscope 11100 may be configured as a so-called flexible scope having a flexible lens barrel.

[0215] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 11101. A light source device 11203 is connected to the endoscope 11100, and light generated by the light source device 11203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 11101, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 11132. The endoscope 11100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0216] An optical system and an image sensor are provided inside the camera head 11102, and reflected light (observation light) from an observation target is collected on the image sensor by the optical system. The observation light is photoelectrically converted by the image sensor to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The image signal is transmitted to a camera control unit (CCU) 11201 as RAW data.

[0217] The CCU 11201 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0218] Under the control of the CCU 11201, the display device 11202 displays an image based on an image signal that has been subjected to image processing by the CCU 11201.

[0219] The light source device 11203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 11100 with irradiation light when photographing an operation site or the like.

[0220] The input device 11204 is an input interface for the endoscopic surgery system 11000. A user can input various information and instructions to the endoscopic surgery system 11000 via the input device 11204. For example, the user inputs an instruction to change the imaging conditions (type of irradiated light, magnification, focal length, etc.) of the endoscope 11100.

[0221] The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 for cauterizing tissue, incising, sealing blood vessels, etc. The insufflation device 11206 sends gas into the body cavity of the patient 11132 via the insufflation tube 11111 to inflate the body cavity for the purpose of securing the field of view of the endoscope 11100 and securing the working space of the surgeon. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various formats such as text, image, or graph.

[0222] The light source device 11203 that supplies irradiation light to the endoscope 11100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 11203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation object with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 11102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0223] The light source device 11203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. The driving of the image sensor of the camera head 11102 may be controlled in synchronization with the timing of the change in the light intensity to obtain images in a time-division manner, and the images may be synthesized to generate an image with a high dynamic range that is free of so-called blackout and whiteout.

[0224] The light source device 11203 may be configured to supply light of a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, by utilizing the wavelength dependency of light absorption in body tissue, a narrow band light is irradiated compared to the irradiated light (i.e., white light) during normal observation, and a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast, so-called narrow band imaging is performed. Alternatively, in the special light observation, a fluorescent observation may be performed in which an image is obtained by fluorescence generated by irradiating an excitation light. In the fluorescent observation, it is possible to irradiate an excitation light to a body tissue and observe the fluorescence from the body tissue (autofluorescence observation), or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 11203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0225] FIG. 24 is a block diagram showing an example of the functional configuration of the camera head 11102 and the CCU 11201 shown in FIG.

[0226] The camera head 11102 has a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 has a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected to each other by a transmission cable 11400 so as to be able to communicate with each other.

[0227] The lens unit 11401 is an optical system provided at the connection portion with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and enters the lens unit 11401. The lens unit 11401 is configured by combining a plurality of lenses including a zoom lens and a focus lens.

[0228] The imaging unit 11402 is composed of an imaging element. The imaging element constituting the imaging unit 11402 may be one (so-called single-plate type) or multiple (so-called multi-plate type). When the imaging unit 11402 is composed of a multi-plate type, for example, each imaging element may generate an image signal corresponding to each of RGB, and a color image may be obtained by combining the image signals. Alternatively, the imaging unit 11402 may be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to 3D (Dimensional) display. By performing 3D display, the surgeon 11131 can more accurately grasp the depth of the biological tissue in the surgical site. Note that when the imaging unit 11402 is composed of a multi-plate type, multiple lens units 11401 may be provided corresponding to each imaging element.

[0229] Furthermore, the imaging unit 11402 does not necessarily have to be provided in the camera head 11102. For example, the imaging unit 11402 may be provided inside the lens barrel 11101, immediately behind the objective lens.

[0230] The driving unit 11403 is configured by an actuator, and moves the zoom lens and focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera head control unit 11405. This allows the magnification and focus of the image captured by the imaging unit 11402 to be appropriately adjusted.

[0231] The communication unit 11404 is configured by a communication device for transmitting and receiving various information to and from the CCU 11201. The communication unit 11404 transmits the image signal obtained from the imaging unit 11402 to the CCU 11201 via the transmission cable 11400 as RAW data.

[0232] Furthermore, the communication unit 11404 receives a control signal for controlling the driving of the camera head 11102 from the CCU 11201, and supplies the control signal to the camera head control unit 11405. The control signal includes information on the imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value at the time of capturing the image, and / or information specifying the magnification and focus of the captured image.

[0233] The image capturing conditions such as the frame rate, exposure value, magnification, and focus may be appropriately specified by a user, or may be automatically set by the control unit 11413 of the CCU 11201 based on an acquired image signal. In the latter case, the endoscope 11100 is equipped with a so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.

[0234] The camera head control unit 11405 controls the driving of the camera head 11102 based on a control signal received from the CCU 11201 via the communication unit 11404.

[0235] The communication unit 11411 is configured with a communication device for transmitting and receiving various information to and from the camera head 11102. The communication unit 11411 receives an image signal transmitted from the camera head 11102 via the transmission cable 11400.

[0236] Furthermore, the communication unit 11411 transmits, to the camera head 11102, a control signal for controlling the driving of the camera head 11102. The image signal and the control signal can be transmitted by electrical communication, optical communication, or the like.

[0237] The image processing unit 11412 performs various types of image processing on the image signal, which is RAW data sent from the camera head 11102 .

[0238] The control unit 11413 performs various controls related to imaging of the surgical site etc. by the endoscope 11100 and display of the captured image obtained by imaging the surgical site etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.

[0239] Further, the control unit 11413 causes the display device 11202 to display the captured image showing the surgical site, etc., based on the image signal that has been image-processed by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition techniques. For example, the control unit 11413 can recognize surgical tools such as forceps, specific living body parts, bleeding, mist when the energy treatment tool 11112 is used, etc., by detecting the shape and color of the edge of an object included in the captured image. When the control unit 11413 causes the display device 11202 to display the captured image, it may use the recognition result to superimpose various types of surgery support information on the image of the surgical site. By superimposing and presenting the surgery support information to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can proceed with the surgery reliably.

[0240] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 is an electrical signal cable for communication of electrical signals, an optical fiber for optical communication, or a composite cable of these.

[0241] Here, in the illustrated example, communication is performed wired using a transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may also be performed wirelessly.

[0242] An example of an endoscopic surgery system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the endoscope 11100 among the configurations described above. Specifically, the solid-state imaging device 10 can be applied to the imaging unit 11402 of the camera head 11102. By applying the technology according to the present disclosure to the imaging unit 11402, for example, deterioration of phase difference information and image quality can be suppressed, and a clearer image of the surgical site can be obtained, allowing the surgeon to reliably confirm the surgical site.

[0243] Although an endoscopic surgery system has been described as an example here, the technology disclosed herein may also be applied to other systems, such as a microsurgery system.

[0244] The embodiments of the technology according to the present disclosure are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the technology according to the present disclosure.

[0245] Furthermore, the technology according to the present disclosure can have the following configuration.

[0246] (1) A pixel array unit in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels include phase difference pixels for detecting a phase difference, the pixel array unit has an arrangement pattern in which pixel units including adjacent pixels of the same color are regularly arranged, When reading out the plurality of pixels, when a pixel signal of a predetermined pixel in the horizontal direction and a pixel signal of a predetermined pixel in the vertical direction are added horizontally and vertically, the phase difference pixels are partially not added. Solid-state imaging device. (2) The density of the phase difference pixels after the horizontal and vertical addition is the same as or approximately the same as, or lower than, the density of the phase difference pixels before the horizontal and vertical addition. The solid-state imaging device according to (1) above. (3) a plurality of selection control lines are provided in a horizontal direction for selecting a connection from a floating diffusion region formed for each of a plurality of pixel portions in the vertical direction to a vertical signal line; During the horizontal and vertical addition, driving of a plurality of selection control lines is controlled. The solid-state imaging device according to (1) or (2). (4) The exposure time of each pixel included in the pixel unit is adjusted, When reading out the plurality of pixels, pixel signals of the same color but different exposure times obtained by the horizontal and vertical addition are synthesized to generate a high dynamic range (HDR) signal. The solid-state imaging device according to any one of (1) to (3). (5) Among the plurality of pixels, the phase difference pixel including any of a plurality of photoelectric conversion elements formed for one on-chip lens is an adjacent pixel of the same color and is included in an adjacent different pixel portion. The solid-state imaging device according to (4) above. (6) The pixel unit is composed of four pixels (2×2), The phase difference pixel is configured of a first phase difference pixel and a second phase difference pixel as two pixels of 2×1 included in pixel units adjacent to each other on the left and right, The first phase difference pixel and the second phase difference pixel are adjusted to have the same exposure time. The solid-state imaging device according to (5) above. (7) The phase difference pixel is configured as a light-shielding pixel. The solid-state imaging device according to (4) above. (8) All pixels of the same color included in the pixel unit are exposed to the same exposure time, When reading out the plurality of pixels, pixel signals of pixels of the same color included in the pixel unit are added together, and then the horizontal and vertical addition is performed. The solid-state imaging device according to any one of (1) to (3). (9) Among the plurality of pixels, the phase difference pixel including any of a plurality of photoelectric conversion elements formed for one on-chip lens is an adjacent pixel of the same color and is included in an adjacent different pixel portion. The solid-state imaging device according to (8) above. (10) The pixel unit is composed of four pixels (2×2), The phase difference pixel is composed of a first phase difference pixel and a second phase difference pixel as two pixels of 2×1 included in each of the pixel portions adjacent to each other on the left and right. The solid-state imaging device according to (9) above. (11) The phase difference pixel is configured as a light-shielding pixel. The solid-state imaging device according to (8) above. (12) The horizontal / vertical addition adds pixel signals of two predetermined pixels in the horizontal direction and pixel signals of two predetermined pixels in the vertical direction; In the horizontal and vertical addition, driving of a plurality of selection control lines is controlled so as to select pixel signals of phase difference pixels included in a specific pixel unit and not add pixel signals of normal pixels included in other pixel units excluding the specific pixel unit. The solid-state imaging device according to (3) above. (13) In horizontal / vertical addition for adding pixel signals of two predetermined pixels in the horizontal direction and pixel signals of two pixels in the vertical direction, driving of a plurality of selection control lines is controlled so that pixel signals of pixels included in a pixel section including a specific phase difference pixel and pixel signals of normal pixels included in a pixel section diagonally opposite the pixel section are not added. The solid-state imaging device according to (3) or (12) above. (14) In horizontal / vertical addition for adding pixel signals of two predetermined pixels in the horizontal direction and pixel signals of two pixels in the vertical direction, driving of a plurality of selection control lines is controlled so as to add pixel signals of pixels in a pixel section including only normal pixels. The solid-state imaging device according to any one of (3), (12) and (13). (15) A particular floating diffusion region is connected to a common vertical signal line; In the horizontal and vertical addition, driving of a plurality of selection control lines is controlled in order to read out pixel signals from specific floating diffusion regions. The solid-state imaging device according to any one of (3), (12) and (13). (16) Particular floating diffusion regions connected to a common vertical signal line are formed in different vertical columns; Each of the pixel units sharing a particular floating diffusion region is arranged in the same row in the horizontal direction and includes pixels of the same color. The solid-state imaging device according to (15) above. (17) A pixel array unit in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels include phase difference pixels for detecting a phase difference, the pixel array unit has an arrangement pattern in which pixel units including adjacent pixels of the same color are regularly arranged, A plurality of selection control lines are provided in the horizontal direction for selecting connections from floating diffusion regions formed for each of a plurality of pixel portions in the vertical direction to vertical signal lines. Solid-state imaging device. (18) The exposure time of each pixel included in the pixel unit is adjusted, When reading out the plurality of pixels, pixel signals of the same color but different exposure times obtained by horizontally and vertically adding pixel signals of predetermined pixels in the horizontal direction and predetermined pixels in the vertical direction are synthesized to generate an HDR signal. The solid-state imaging device according to (17) above. (19) All pixels of the same color included in the pixel unit are exposed to the same exposure time, When reading out the plurality of pixels, pixel signals of pixels of the same color included in the pixel section are added, and then pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically. The solid-state imaging device according to (17) above. (20) A pixel array unit in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels include phase difference pixels for detecting a phase difference, the pixel array unit has an arrangement pattern in which pixel units including adjacent pixels of the same color are regularly arranged, When reading out the plurality of pixels, when a pixel signal of a predetermined pixel in the horizontal direction and a pixel signal of a predetermined pixel in the vertical direction are added horizontally and vertically, the phase difference pixels are partially not added. An electronic device equipped with a solid-state imaging device. [Explanation of symbols]

[0247] 10 solid-state imaging device, 11 pixel array section, 12 vertical drive circuit, 111 floating diffusion region (FD), 121 pixel drive line, 122, 122-1, 122-2, 122-3 selection control line, 131 vertical signal line (VSL), 100 pixel, 200, 200A, 200B pixel section, 1000 electronic device, 1012 imaging section, 1014 control section, 1020 drive section, 12031 imaging section, 11402 imaging section

Claims

1. A pixel array unit in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels include phase difference pixels for detecting a phase difference, the pixel array unit has an arrangement pattern in which pixel units including pixels corresponding to the same wavelength band and adjacent to each other are regularly arranged, When reading out the plurality of pixels, in a case where pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically, if the phase difference pixel is included in the predetermined pixels in the horizontal direction and the predetermined pixels in the vertical direction, the phase difference pixel is partially not added. Solid-state imaging device.

2. The density of the phase difference pixels after the horizontal and vertical addition is the same as or approximately the same as, or lower than, the density of the phase difference pixels before the horizontal and vertical addition. The solid-state imaging device according to claim 1 .

3. a plurality of selection control lines are provided in a horizontal direction for selecting a connection from a floating diffusion region formed for each of a plurality of pixel portions in the vertical direction to a vertical signal line; During the horizontal and vertical addition, driving of a plurality of selection control lines is controlled. The solid-state imaging device according to claim 1 .

4. The exposure time of each pixel included in the pixel unit is adjusted, When reading out the plurality of pixels, pixel signals having different exposure times in the same wavelength band obtained by the horizontal and vertical addition are synthesized to generate a high dynamic range (HDR) signal. The solid-state imaging device according to claim 1 .

5. Among the plurality of pixels, the phase difference pixel including any of a plurality of photoelectric conversion elements formed for one on-chip lens is a pixel corresponding to the same adjacent wavelength band and is included in adjacent different pixel units. The solid-state imaging device according to claim 4 .

6. The pixel unit is composed of four pixels (2×2), The phase difference pixel is configured of a first phase difference pixel and a second phase difference pixel as two pixels of 2×1 included in pixel units adjacent to each other on the left and right, The first phase difference pixel and the second phase difference pixel are adjusted to have the same exposure time. The solid-state imaging device according to claim 5 .

7. The phase difference pixel is configured as a light-shielding pixel. The solid-state imaging device according to claim 4 .

8. The pixels included in the pixel unit are exposed to the same exposure time for all pixels corresponding to the same wavelength band, When reading out the plurality of pixels, pixel signals of pixels corresponding to the same wavelength band included in the pixel unit are added together, and then the horizontal and vertical addition is performed. The solid-state imaging device according to claim 1 .

9. Among the plurality of pixels, the phase difference pixel including any of a plurality of photoelectric conversion elements formed for one on-chip lens is a pixel corresponding to the same adjacent wavelength band and is included in adjacent different pixel units. The solid-state imaging device according to claim 8 .

10. The pixel unit is composed of four pixels (2×2), The phase difference pixel is composed of a first phase difference pixel and a second phase difference pixel as two pixels of 2×1 included in each of the pixel portions adjacent to each other on the left and right. The solid-state imaging device according to claim 9 .

11. The phase difference pixel is configured as a light-shielding pixel. The solid-state imaging device according to claim 8 .

12. The horizontal / vertical addition adds pixel signals of two predetermined pixels in the horizontal direction and pixel signals of two predetermined pixels in the vertical direction; In the horizontal and vertical addition, driving of a plurality of selection control lines is controlled so as to select pixel signals of phase difference pixels included in a specific pixel unit and not add pixel signals of normal pixels included in other pixel units excluding the specific pixel unit. The solid-state imaging device according to claim 3 .

13. During the horizontal and vertical addition, driving of a plurality of selection control lines is controlled so that pixel signals of pixels included in a pixel section including a specific phase difference pixel and pixel signals of normal pixels included in a pixel section diagonally opposite the specific pixel section are not added. The solid-state imaging device according to claim 12.

14. During the horizontal and vertical addition, driving of a plurality of selection control lines is controlled so as to add up pixel signals of pixels in a pixel section including only normal pixels. The solid-state imaging device according to claim 13.

15. A particular floating diffusion region is connected to a common vertical signal line; In the horizontal and vertical addition, driving of a plurality of selection control lines is controlled in order to read out pixel signals from specific floating diffusion regions. The solid-state imaging device according to claim 3 .

16. Particular floating diffusion regions connected to a common vertical signal line are formed in different vertical columns; Each of the pixel units sharing a particular floating diffusion region is configured to include pixels arranged in the same row in the horizontal direction and corresponding to the same wavelength band. The solid-state imaging device according to claim 15.

17. A pixel array unit in which a plurality of pixels are arranged two-dimensionally, the plurality of pixels include phase difference pixels for detecting a phase difference, the pixel array unit has an arrangement pattern in which pixel units including pixels corresponding to the same wavelength band and adjacent to each other are regularly arranged, When reading out the plurality of pixels, in a case where pixel signals of predetermined pixels in the horizontal direction and pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically, if the phase difference pixel is included in the predetermined pixels in the horizontal direction and the predetermined pixels in the vertical direction, the phase difference pixel is partially not added. An electronic device equipped with a solid-state imaging device.

Images