Solid-state imaging device and electronic apparatus

Abstract

The present disclosure relates to a solid-state imaging device and an electronic apparatus that can suppress deterioration of phase difference information. A solid-state imaging device is provided, which includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner, wherein the plurality of pixels include a phase difference pixel for phase difference detection. The pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are regularly arranged. When pixel signals of predetermined pixels in a horizontal direction and pixel signals of predetermined pixels in a vertical direction are subjected to horizontal / vertical addition at the time of reading the plurality of pixels, the phase difference pixel is partially not added. The present disclosure can be applied to, for example, a CMOS image sensor having a phase difference pixel.

Description

Technical Field

[0001] This disclosure relates to solid-state camera devices and electronic devices, and more particularly to solid-state camera devices and electronic devices capable of suppressing phase difference information degradation. Background Technology

[0002] In recent years, in order to improve the speed of autofocus, solid-state camera devices with pixels for phase difference detection (hereinafter referred to as phase difference pixels) have been used.

[0003] As a construction method for this type of phase difference pixel, the techniques disclosed in Patent Document 1 and Patent Document 2 are known.

[0004] List of cited references

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-216647

[0007] Patent Document 2: Japanese Patent Application Publication No. 2011-250325 Summary of the Invention

[0008] The problem to be solved by the present invention

[0009] At the same time, when using a structure in which multiple pixels arranged in two dimensions in a pixel array unit include phase difference pixels, it is necessary to suppress the degradation of phase difference information.

[0010] This disclosure is made in view of this situation, and the purpose of this disclosure is to suppress the degradation of phase difference information.

[0011] Technical solutions to the problem

[0012] According to one aspect of this disclosure, a solid-state camera device includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner. The plurality of pixels includes phase difference pixels for phase difference detection. The pixel array unit has an array pattern in which pixel units comprising adjacent pixels of the same color are regularly arranged, and when reading the plurality of pixels, the phase difference pixels are partially not added when the pixel signals of predetermined pixels in the horizontal direction and predetermined pixels in the vertical direction are horizontally / vertically added.

[0013] According to one aspect of this disclosure, an electronic device is equipped with a solid-state camera device, the solid-state camera device including a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner. The plurality of pixels includes phase difference pixels for phase difference detection. The pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are regularly arranged, and when reading the plurality of pixels, when the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are horizontally / vertically added, the phase difference pixels are partially not added.

[0014] In a solid-state imaging device and electronic device according to one aspect of the present disclosure, a pixel array unit is provided, in which a plurality of pixels are arranged in a two-dimensional manner. The plurality of pixels includes phase difference pixels for phase difference detection, and the pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are regularly arranged. Furthermore, when reading the plurality of pixels, when the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are horizontally / vertically added, the phase difference pixels are partially not added.

[0015] According to one aspect of this disclosure, a solid-state camera device includes a pixel array unit in which a plurality of pixels are arranged in a two-dimensional manner. The plurality of pixels includes phase difference pixels for phase difference detection. The pixel array unit has an array pattern in which pixel units comprising adjacent pixels of the same color are regularly arranged, and a plurality of selection control lines are provided in the horizontal direction for selecting connections from a floating diffusion region formed for each of the plurality of pixel units in the vertical direction to a vertical signal line.

[0016] In a solid-state imaging device according to one aspect of this disclosure, a pixel array unit is provided, in which a plurality of pixels are arranged in a two-dimensional manner. The plurality of pixels includes phase difference pixels for phase difference detection, and the pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are regularly arranged. Furthermore, a plurality of selection control lines are provided in the horizontal direction for selecting connections from floating diffusion regions formed for each of the plurality of pixel units in the vertical direction to vertical signal lines.

[0017] According to one aspect of this disclosure, a solid-state camera device or electronic device may be a standalone device or an internal block constituting a device. Attached Figure Description

[0018] Figure 1This is a block diagram illustrating an example of the construction of a solid-state camera device applicable to the technology of this disclosure.

[0019] Figure 2 It is a diagram showing the planar layout of multiple pixels arranged in a pixel array unit.

[0020] Figure 3 This is a diagram illustrating an example of read-drive in a triple-exposure HDR / H2V2 additive mode.

[0021] Figure 4 It is a diagram that schematically represents the state when H2V2 is added uniformly throughout the entire region.

[0022] Figure 5 This is a schematic diagram illustrating the state where phase difference pixels are sparse during H2V2 addition.

[0023] Figure 6 This is a diagram illustrating an example of a construction that makes the selection control lines into multiple lines.

[0024] Figure 7 This is a diagram illustrating an example of drive control using two selection control lines in a triple-exposure HDR / H2V2 additive mode.

[0025] Figure 8 This is a diagram illustrating an example of drive control using two selection control lines in a triple-exposure HDR / H2V2 additive mode.

[0026] Figure 9 This is a diagram illustrating an example of a read drive in a nearby 4-pixel addition / H2V2 addition mode.

[0027] Figure 10 This diagram illustrates an example of drive control using two selection control lines in a near-adjacent 4-pixel addition / H2V2 addition mode.

[0028] Figure 11 This diagram illustrates the pixel driving operation when using the method of adding 4 adjacent pixels / H2V2 addition.

[0029] Figure 12 This diagram illustrates the pixel driving operation when using the method of adding 4 adjacent pixels / H2V2 addition.

[0030] Figure 13 This is another example of a construction that makes the selection control lines into multiple lines.

[0031] Figure 14 This is a diagram showing the correspondence between the selection control lines and the read drive.

[0032] Figure 15This is a diagram illustrating an example of a read driver.

[0033] Figure 16 This is a diagram illustrating an example of the construction of a light-blocking pixel.

[0034] Figure 17 This is a diagram illustrating another example of a readout drive for pixel cells arranged in a pixel array.

[0035] Figure 18 This is an enlarged view showing another example of the readout drive for pixel cells arranged in a pixel array.

[0036] Figure 19 This is a block diagram illustrating an example of the construction of an electronic device equipped with a solid-state camera device applicable to the technology of this disclosure.

[0037] Figure 20 This is a diagram illustrating an example of the use of a solid-state camera device applicable to the technology of this disclosure.

[0038] Figure 21 This is a block diagram illustrating an example of a schematic construction of a vehicle control system.

[0039] Figure 22 This is an explanatory diagram showing an example of the installation location of the vehicle exterior information detection unit and the camera unit.

[0040] Figure 23 This is a diagram illustrating an example of a schematic construction of an endoscopic surgical system.

[0041] Figure 24 This is a block diagram illustrating an example of the functional structure of a camera and camera control unit (CCU). Detailed Implementation

[0042] In the following description, embodiments of the technology (the present technology) according to the accompanying drawings will be illustrated. Note that the description will proceed in the following order.

[0043] 1. First Embodiment

[0044] 2. Second Embodiment

[0045] 3. Variations

[0046] 4. Structure of electronic devices

[0047] 5. Examples of the use of solid-state camera devices

[0048] 6. Examples of applications of moving bodies

[0049] 7. Examples of the application of endoscopic surgical systems

[0050] <1. First Embodiment>

[0051] (Example of solid-state camera device construction)

[0052] Figure 1 This is a block diagram illustrating a construction example of an embodiment of a solid-state camera device to which the technology applicable according to this disclosure is applicable.

[0053] Figure 1 The solid-state imaging device 10 includes an image sensor such as a CMOS (complementary metal oxide semiconductor) image sensor. The solid-state imaging device 10 acquires incident light (image light) from the subject via an optical lens system (not shown), converts the amount of incident light that forms an image on the imaging surface into an electrical signal on a pixel-by-pixel basis, and outputs it as a pixel signal.

[0054] exist Figure 1 In the solid-state camera device 10, there are pixel array units 11, vertical driving circuit 12, column signal processing circuit 13, horizontal driving circuit 14, output circuit 15, control circuit 16 and input / output terminals 17.

[0055] In pixel array unit 11, multiple pixel units 200 are arranged in a two-dimensional manner (in a matrix). Pixel unit 200 includes four pixels 100 of the same color (four pixels in a 2×2 pattern).

[0056] Each pixel unit 200 includes a red (R) pixel, a green (G) pixel, or a blue (B) pixel, which are four pixels 100 of the same color and correspond to a color filter that transmits light of the wavelengths of the red (R), green (G), or blue (B) components. Furthermore, the pixel unit 200 is configured as a shared pixel, wherein the pixel circuitry is shared by the four pixels 100 of the same color.

[0057] In pixel unit 200, each pixel 100 of the same color includes a photodiode (PD) as a photoelectric conversion element and a transmission transistor TR-Tr. Furthermore, in pixel unit 200, the reset transistor RST-Tr, the amplification transistor AMP-Tr, and the selection transistor SEL-Tr, which serve as pixel circuits, are shared by the four pixels 100 of the same color.

[0058] Note that, although detailed later, the pixels used for phase difference detection (hereinafter referred to as phase difference pixels) are distributed (in a repeating pattern) in the pixel array unit 11 as pixels 100 included in pixel unit 200.

[0059] The vertical drive circuit 12 includes a shift register, etc., selects a predetermined pixel drive line 121 to provide a drive signal (pulse) for driving the pixel 100 or pixel unit 200 to the selected pixel drive line 121, and drives the pixel 100 or pixel unit 200 in rows.

[0060] That is, the vertical drive circuit 12 sequentially and selectively scans each pixel 100 or each pixel unit 200 of the pixel array unit 11 in the vertical direction in rows, and provides the pixel signal based on the charge (signal charge) generated corresponding to the amount of light received in the photodiode of each pixel 100 to the column signal processing circuit 13 through the vertical signal line 131.

[0061] The column signal processing circuit 13 is arranged for each column of the pixel unit 200, and for each pixel column, the column signal processing circuit 13 performs signal processing such as noise removal on the signal output from the pixel unit 200 of a row. For example, the column signal processing circuit 13 performs signal processing such as correlated double sampling (CDS) for removing pixel-specific fixed-pattern noise and analog-to-digital (AD) conversion.

[0062] The horizontal drive circuit 14 includes a shift register, etc., and sequentially selects each column signal processing circuit 13 by outputting horizontal scanning pulses in sequence, and causes each column signal processing circuit 13 to output pixel signals to the horizontal signal line 141.

[0063] The output circuit 15 processes the signals sequentially provided from each column signal processing circuit 13 via the horizontal signal line 141 and outputs a signal. Note that the output circuit 15 may, for example, only perform buffering, or it may perform black level adjustment, column change correction, and various digital signal processing.

[0064] The control circuit 16 controls the operation of each unit of the solid-state camera device 10.

[0065] Furthermore, the control circuit 16 generates clock signals and control signals based on the vertical synchronization signal, the horizontal synchronization signal, and the master clock signal, which serve as operating references for the vertical drive circuit 12, the column signal processing circuit 13, and the horizontal drive circuit 14, etc. The control circuit 16 outputs the generated clock signals and control signals to the vertical drive circuit 12, the column signal processing circuit 13, and the horizontal drive circuit 14, etc.

[0066] Input / output terminal 17 exchanges signals with external devices.

[0067] Constructed as described above Figure 1The solid-state imaging device 10 is a CMOS image sensor known as a column AD system, in which column signal processing circuitry 13 is arranged for each pixel column to perform CDS processing and AD conversion processing. Furthermore, Figure 1 The solid-state camera device 10 in the image can be a back-illuminated CMOS image sensor or the like.

[0068] (Plane layout of pixels)

[0069] Figure 2 Examples illustrate in Figure 1 The pixel array unit 11 contains a plurality of pixels 100 arranged in a two-dimensional manner, which are arranged in 32 rows and 32 columns of pixels 100 when viewed from the light incident side.

[0070] In the following description, R pixel 100 refers to a pixel that is configured as a color filter that transmits light with a wavelength having a red (R) component, and obtains an electric charge corresponding to the red (R) component from the light transmitted through the R color filter. R pixel 100 has a diagonal pattern pointing to the upper right, and four adjacent pixels constitute R pixel unit 200A.

[0071] Furthermore, as shown in the lower right box of the figure, the exposure time is adjusted for every four R pixels 100 in R pixel unit 200A. Here, the exposure time is adjusted in three stages: T1, T2, and T3 (T1>T2>T3), where T1 is the long cumulative exposure time, T2 is the medium cumulative exposure time, and T3 is the short cumulative exposure time.

[0072] In R pixel unit 200A, among the four pixels, the upper left is the long-cumulative R. L Pixel 100, with the accumulated R values ​​in the upper right and lower left corners. M Pixel 100, with a short accumulated R in the lower right corner. S 100 pixels.

[0073] Furthermore, G pixel 100 refers to a pixel that receives a charge corresponding to the green (G) component of light from a color filter that transmits light of the wavelength of the green (G) component. G pixel 100 has a dot pattern, and four adjacent pixels constitute a G pixel unit 200A.

[0074] In the G pixel unit 200A, similar to the R pixel unit 200A, the exposure time is adjusted in three stages. Among the four pixels, the upper left is the long-cumulative G pixel. L Pixel 100, the upper right and lower left are the accumulated G. M 100 pixels, with a short accumulated G in the lower right corner. S 100 pixels.

[0075] Furthermore, B-pixel 100 refers to a pixel that receives a charge corresponding to the blue (B) component of light from a color filter that transmits light of the wavelength of the blue (B) component. B-pixel 100 has a diagonal pattern pointing to the lower right, and four adjacent pixels constitute B-pixel unit 200A.

[0076] In the B pixel unit 200A, similar to the R pixel unit 200A, the exposure time is adjusted in three stages. Among the four pixels, the upper left is the long-accumulated B pixel. L Pixel 100, the upper right and lower left are the accumulated B. M Pixel 100, the bottom right corner shows a short cumulative B. S 100 pixels.

[0077] Here, in Figure 2 In the pixel array unit 11, when focusing on the R pixel unit 200A, G pixel unit 200A and B pixel unit 200A, these pixel units 200A are regularly arranged to form a Bayer array.

[0078] In other words, it can also be said that each pixel in the Bayer array is divided into 2×2 pixels and includes four pixels of the same color. Note that the Bayer array is an array pattern in which G pixels are arranged in a grid pattern, and R pixels and B pixels are arranged alternately in each row of the remaining parts.

[0079] In addition, Figure 2 In the pixel array unit 11, a set of phase difference pixels 100L and phase difference pixels 100R are arranged according to a predetermined rule.

[0080] Here, a structure (2×1 OCL structure) is obtained, in which an on-chip lens (OCL) is provided for the two horizontal pixels (2×1 pixels) that form a set of phase difference pixels 100L and 100R. That is, a structure is obtained in which multiple photoelectric conversion elements such as photodiodes are embedded in an on-chip lens. In the following text, a pixel with a 2×1 OCL structure is also referred to as a 2×1 OCL pixel.

[0081] At this time, since the exposure times of the two pixels arranged horizontally in the four pixels constituting the R, G and B pixel units 200A are different, it is difficult to set an on-chip lens for these two pixels (2×1 pixels) due to the characteristics of phase difference pixels.

[0082] Therefore, here, an on-chip lens is arranged across adjacent left and right pixel units 200A, and adjacent left and right pixels 100 including either of the two photoelectric conversion elements formed for the on-chip lens have the same color filter and the same exposure time. By using this structure, an on-chip lens can be set for two pixels (2×1 pixels) in the horizontal direction.

[0083] For example, in adjacent B pixel units 200A and G pixel units 200A, the B pixel unit to the lower right of the B pixel unit 200A... S The photoelectric conversion element of pixel 100 and the G pixel unit 200A at the lower left corner. M When the photoelectric conversion element of pixel 100 is equipped with an on-chip lens, the following structure is obtained.

[0084] That is, during manufacturing, the B pixel in the lower right corner of the B pixel unit 200A... S The color filter for pixel 100 is configured as a G color filter for G pixel 100 instead of a B color filter. Furthermore, BS pixel 100 is a short-accumulation pixel (S), but the exposure time is changed to a medium-accumulation exposure time to obtain a medium-accumulation pixel (M).

[0085] As a result, for the G pixel to the lower right of the adjacent B pixel unit 200A... M The G pixel in the lower left corner of pixel 100 and G pixel unit 200A M Pixel 100 is set with an on-chip lens, and these horizontally adjacent Gs M Pixel 100 has the same G color filter and the same medium cumulative exposure time.

[0086] Therefore, the G pixel in the lower right corner of pixel B 200A M Pixel 100 is constructed as phase difference pixel 100L, and the G pixel unit 200A is located to the lower left of the G pixel. M Pixel 100 is constructed as a phase difference pixel 100R, and the phase difference between two images can be detected based on pixel signals obtained from a set of phase difference pixels 100L and 100R.

[0087] Note that in the B pixel unit 200A, which includes the phase difference pixel 100L, when the B pixel in the lower right corner... S 100 pixels becomes G M At pixel 100, there will be no short accumulated pixels (S). However, by using the B in the lower left corner... M Pixel 100 becomes B S Pixel 100, the four pixels will include short cumulative pixels (S) as well as long cumulative pixels (L) and medium cumulative pixels (M).

[0088] In this way, Figure 2 In the pixel array unit 11, multiple sets of phase difference pixels 100L and phase difference pixels 100R included in adjacent B pixel units 200A and G pixel units 200A are regularly arranged in the horizontal and vertical directions with a predetermined pixel interval.

[0089] (Triple exposure HDR / H2V2 summation method)

[0090] Figure 3 Schematic illustration of as Figure 2 The case where the three-exposure HDR / H2V2 summation method is performed by reading out the pixel unit 200A arranged in a Bayer array in the pixel array unit 11.

[0091] When using the triple-exposure HDR / H2V2 addition method, H2V2 addition is performed first. In H2V2 addition, the pixel signals from pixels 100 of the same color and exposure are added between two pixels horizontally and two pixels vertically (H2V2 addition). Figure 3 (A).

[0092] Specifically, by using the R pixel unit 200A at its upper left corner... L The pixel signals of the four pixels of pixel 100 are added together and then compared with... Figure 3 The black circles at the vertices of each of the four rectangles in A correspond to the four R's. L The pixel signals of pixel 100 are added together to obtain the summed signal. Figure 3 (B).

[0093] Similarly, by performing H2V2 summation on the remaining three pixels of R pixel unit 200A, the four pixels of G pixel unit 200A, and the four pixels of B pixel unit 200A, a summed signal of pixel 100 with the same color and exposure was obtained. Figure 3 (B).

[0094] Furthermore, after H2V2 addition, triple-exposure HDR is performed. That is, since the pixel signal obtained by H2V2 addition (analog addition) includes pixel signals from long-accumulating pixels (L), medium-accumulating pixels (M), and short-accumulating pixels (S) for each color of R, G, and B, in triple-exposure HDR, a high dynamic range (HDR) pixel signal is obtained by selecting or combining these pixel signals with different exposure times. Figure 3 (C).

[0095] Triple exposure HDR is a technique that extends the dynamic range by arranging three pixels of the same color with three different exposure times and selecting or combining signals from these three pixels with appropriate exposure.

[0096] In addition, Figure 2 In the pixel array, an on-chip lens is formed by crossing B pixel unit 200A and G pixel unit 200A, and phase difference pixels 100L and 100R are arranged to perform phase difference detection.

[0097] Here, for this pixel array, when performing H2V2 summation on the nearest neighbor pixels 100 with the same color and exposure, assuming all regions are summed uniformly, many ordinary pixels are summed with phase difference pixels. Figure 4 (A).

[0098] Therefore, a schematic representation was obtained. Figure 4 The pixel array of the state after adding H2V2 in B, and schematically represented. Figure 4 Compared to the pixel array in state A before H2V2 addition, the density of phase difference pixels 100L and 100R in the pixel array (the entire area of ​​the pixel array) increases. However, due to the mixing of signals from ordinary pixels and phase difference pixels, it is difficult to detect the phase difference.

[0099] Therefore, in the technology according to this disclosure, when performing horizontal and vertical addition (H2V2 addition), by making the phase difference pixels 100L and 100R sparse, and setting the density of the phase difference pixels after horizontal / vertical addition to be the same as, nearly the same as, or smaller than, the density of the phase difference pixels before horizontal / vertical addition, thereby preventing the density of the phase difference pixels 100L and 100R from becoming too high. Figure 5 (B).

[0100] That is, when executing Figure 3 When adding H2V2 as shown in A, if at least one of the four selected pixels includes a phase difference pixel, in some pixels only the phase difference pixel is selected and added, or in other pixels only ordinary pixels other than the phase difference pixel are selected and added.

[0101] This configuration allows for adjustment of the density of phase difference pixels and prevention of phase difference signal degradation. Furthermore, by controlling the density of phase difference pixels before and after horizontal / vertical addition to make the density the same or nearly the same, it enables the same signal processing pipeline to be shared.

[0102] Here, examples will be given to illustrate the addition methods for the three modes.

[0103] The first mode is the mode that leaves the phase difference pixels (2×10CL pixels) when H2V2 is added. Here, in order to leave the phase difference pixels, the pixel signals of other ordinary pixels (12 ordinary pixels) are excluded, so that these signals are not transmitted to the vertical signal line 131 (no VSL transmission is performed).

[0104] In the following text, under the control of the addition method in the first mode, among the four pixel units 200A of the same color that are added in H2V2, the three pixel units 200A (ordinary pixels of 12 pixels) other than the pixel unit 200A including the phase difference pixel do not perform VSL transmission and are not added, so they are referred to as "non-additive pixel units".

[0105] The second mode is to discard phase difference pixels (2×1 OCL pixels) when H2V2 is added. That is, due to the limitation of the phase difference pixel correction circuit relative to RAW pixels, the density of phase difference pixels is appropriately excluded.

[0106] In this case, pixel units 200A that include phase difference pixels and diagonally positioned pixel units 200A are excluded by not performing VSL transmission. Here, since the former type of pixel unit 200A includes phase difference pixels, it is called a "pixel unit with phase difference pixel discarded", while the diagonal pixel unit 200A is called a "partner pixel unit".

[0107] Note that the reason for excluding the diagonal pixel units 200A that exist in the pixel unit 200A that includes the phase difference pixel, i.e., excluding the diagonal "partner pixel units" of the "pixel units discarded by phase difference pixels", is to align the center of gravity of the pixels after H2V2 addition.

[0108] The third mode is one in which four pixel units 200A (16 ordinary pixels) are added together. That is, here, since the usual H2V2 is performed, the 16 ordinary pixels are transferred via VSL.

[0109] In order to control the on / off of VSL transmission in the above three modes, the selection control line (SEL line) for controlling the connection between the vertical signal line 131 and the floating diffusion area (FD) is made into multiple lines and drive control is performed.

[0110] Specifically, such as Figure 6 As shown, a selection control line (SEL) 122 is used to control the connection between the vertical signal line (VSL) 131 and the floating diffuser (FD) 111. Figure 6 The A) is configured as two selection control lines (SEL) 122-1 and 122-2 for multiple lines. Figure 6(B) is used to control the opening / closing of VSL transmission corresponding to the three modes mentioned above.

[0111] Note that selection control lines 122-1 and 122-2 are included in pixel drive line 121 ( Figure 1 The drive signal is applied from the vertical drive circuit 12 or the control circuit 16, or from an external control device (not shown). Furthermore, the selection transistor SEL-Tr connected to the selection control lines 122-1 and 122-2 operates according to the drive signal, thereby controlling the on / off state of the VSL transmission.

[0112] Specifically, when performing H2V2 addition, such as Figure 7 As shown, assume the following situation: among the phase difference pixels (2×1OCL pixels) represented by white squares in the figure, keep the group of phase difference pixels (100L, 100R) surrounded by box 210, and discard the other phase difference pixels (2×1OCL pixels).

[0113] In this case, by controlling the drive of the selection control lines 122-1 and 122-2, the VSL transmission is turned off at the position indicated by the symbol in the figure (the symbol with a diagonal line in the circle), so that only the phase difference pixels in box 210 can be retained.

[0114] That is, such as Figure 8 As shown, "1" in the figure indicates that the control of the addition method of the first mode is executed, and the phase difference pixels in box 210 are left, but the three pixel units 200A (12 ordinary pixels) other than the pixel unit 200A that includes the phase difference pixels are not transmitted by VSL and are not added. That is, "C" in the figure is "non-additive pixel unit".

[0115] Note that here, under the drive control of selection control lines 112-1 and 112-2, phase difference pixels are selected on a pixel-by-pixel basis. Furthermore, in Figure 7 and Figure 8 In this process, a drive signal is applied to the transmission control line (TRG line) and selects pixel 100 in pixel unit 200A.

[0116] In addition, Figure 8 In the attached figure, "2" indicates that the addition method of the second mode is executed, and pixel units 200A including phase difference pixels are not read. In addition, pixel units 200A located diagonally are also not read. That is, "A" in the figure is "pixel unit discarded by phase difference pixels", "B" in the figure is "partner pixel unit", and these pixel units are not added.

[0117] In addition, Figure 8In the attached diagram, "3" indicates that the control of the addition method of the third mode is performed, and the ordinary H2V2 addition is performed on four pixel units 200A (16 ordinary pixels).

[0118] As described above, by making the selection control line (SEL line) into multiple lines to control the on / off state of VSL transmission and to control the addition method of the first to third modes, it is possible to prevent phase difference pixels and ordinary pixels from being added together, while reducing the density of phase difference pixels.

[0119] Furthermore, in the technology according to this disclosure, by preparing two selection control lines (SEL lines), changing the connection of the selection control lines of each floating diffuser (FD) 111, and controlling the drive signals applied to the selection control lines 122-1 and 122-2, the bisecting of the signal level on the vertical signal line (VSL) 131 is avoided. Note that since the number of selection control lines is typically one, and the two floating diffuser (FD) 111 are connected to the same vertical signal line (VSL) 131 in the vertical direction, the signal level in the vertical signal line (VSL) 131 is bisected.

[0120] Note that, although the example here illustrates a construction with a two-row AD converter including comparator 151, even with other constructions, by applying a similar concept, high phase difference detection performance and image quality can be achieved during H2V2 addition using a three-exposure HDR / H2V2 addition method.

[0121] Here, when implementing low-power drives, it is expected that three requirements will be met in terms of implementation.

[0122] The first requirement is that the existing defective pixel circuitry is suitable. That is, if the first requirement cannot be met, countermeasure circuitry needs to be installed on the existing defective pixel circuitry, which will lead to an increase in circuit size or a decrease in yield.

[0123] In this respect, in the technology according to this disclosure, the density of phase difference pixels 100L and 100R (density of 2×10CL pixels) is reduced by controlling the drive signals applied to the two selection control lines during H2V2 addition. Therefore, existing defective pixel circuits are applicable, satisfying the first requirement.

[0124] The second requirement is that phase detection autofocus (PDAF) can be performed while leaving only the necessary phase difference pixels 100L and 100R (2×1OCL pixels).

[0125] In this respect, in the technology according to this disclosure, by controlling the drive signals to be applied to the two selection control lines and selecting the pixels to be added, the phase difference pixels 100L and 100R (2×10CL pixels) used in phase detection AF are not added but left as individual pixels, while the other surrounding pixels can be added, which satisfies the second requirement.

[0126] The third requirement is that the center of gravity positions must match after the simulation is added.

[0127] In this respect, the technology according to this disclosure achieves a lower power readout drive by using H2V2 pixel addition, and by controlling the drive signals applied to the two selection control lines and selecting the added pixels (such as "partner pixel units" relative to "pixel units discarded by phase difference pixels") to align the centroids after addition, the centroid deviation correction in the added pixels can be incorporated into the drive, which satisfies the third requirement.

[0128] As described above, although there is usually only one selection control line, in the technology according to this disclosure, phase difference pixel correction, phase difference detection and centroid correction are solved simultaneously by preparing two selection control lines and introducing a controllable physical specification (conversion of contact with the selection control line) for each floating diffusion region (FD).

[0129] <2. Second Embodiment>

[0130] (Add the 4 nearest neighbors / add H2V2)

[0131] Figure 9 The illustration shows the case where the near-adjacent 4-pixel addition / H2V2 addition method is performed as a readout driver for pixel units 200B arranged in a Bayer array within pixel array unit 11. However, in each of the four pixels in each pixel unit 200B within pixel array unit 11, that is, in all pixels 100 of the same color, the exposure time is the same (cumulative exposure time, etc.).

[0132] When using the 4-pixel addition / H2V2 addition method, the 4-pixel addition of neighboring pixels is performed first. In the 4-pixel addition of neighboring pixels, the four pixels (shared pixels) constituting the R, G, and B pixel units 200B are defined as the four pixels of neighboring pixels, and the pixel signals from the four pixels are added in the floating diffusion region 111 (FD addition) to generate the added signals of the R component, G component, and B component, respectively.

[0133] Furthermore, after adding the four adjacent pixels, H2V2 addition is performed. In H2V2 addition, the addition signals from pixel units 200B of the same color are further added in two pixels in the horizontal direction and two pixels in the vertical direction.

[0134] Specifically, in the R pixel unit 200B, with Figure 9 In grid A, the four white circles at the four corners of cell 311R correspond to the four R pixels 100, which are set to the four nearest neighboring pixels, and pixel addition (simulated addition) is performed. Furthermore, through H2V2 addition, in both the horizontal and vertical directions, the pixels in the... Figure 9 The summation signals obtained from the four adjacent pixels near the four corners of grid 311R in A are further added (simulated addition). The result is the summation signal of the R component. Figure 9 (The summed signal to the lower left of B).

[0135] Similarly, in the G pixel unit 200B, according to Figure 9 The white circle of grid 311G in A is subjected to the summation of the 4 adjacent pixels and the summation of H2V2, thereby obtaining the summed signal of the G component. Figure 9 (The summed signal of the upper left or lower right of B). Furthermore, in pixel unit 200B of pixel B, according to... Figure 9 The white circle of grid 311B in A is subjected to the summation of the 4 adjacent pixels and the H2V2 summation to obtain the summation signal of component B. Figure 9 (The summed signal above and to the right of B).

[0136] Here, also in Figure 9 In the pixel array, phase difference detection is performed by forming an on-chip lens across B pixel unit 200B and G pixel unit 200B and arranging phase difference pixels 100L and 100R groups.

[0137] When performing H2V2 addition on the nearest pixel units 200B of the same color in this pixel array, assuming that all areas are added uniformly, many ordinary pixels are added to the phase difference pixels. Therefore, after H2V2 addition, the density of phase difference pixels 100L and 100R among the pixels included in the pixel array (the entire area of ​​the pixel array) increases. However, due to the mixing of signals from ordinary pixels and phase difference pixels, it is difficult to detect the phase difference.

[0138] Therefore, in the 4-pixel addition / H2V2 addition method in the vicinity, similar to the three-exposure HDR / H2V2 addition method mentioned above, when performing H2V2 addition, the phase difference pixels 100L and 100R are sparsed, so that the density of the phase difference pixels 100L and 100R does not become too high.

[0139] That is, in execution Figure 9 When performing H2V2 addition as shown in Figure A, if at least one of the selected four pixels includes a phase difference pixel, some pixels may only select the phase difference pixel and add it, while others may only select ordinary pixels other than the phase difference pixel and add them. This construction can adjust the density of the phase difference pixels and prevent the degradation of the phase difference signal.

[0140] Furthermore, in the adjacent 4-pixel addition / H2V2 addition method, similar to the three-exposure HDR / H2V2 addition method described above, the selection control line 122 used to control the connection between the vertical signal line (VSL) 131 and the floating diffusion area (FD) 111 is made into multiple lines to control the on / off of VSL transmission in the three modes from the first mode to the third mode.

[0141] Specifically, such as Figure 10 As shown, suppose the phase difference pixels (2×1OCL pixels) enclosed by box 210 are kept, while other phase difference pixels (2×1OCL pixels) are discarded.

[0142] In this case, "1" in the figure indicates that the control of the addition method of the first mode is performed, and the phase difference pixels in box 210 are left, but the three pixel units 200B (12 ordinary pixels) other than the pixel unit 200B containing the phase difference pixels are not transmitted by VSL and are not added. That is, "C" in the figure is "non-additive pixel unit".

[0143] In addition, Figure 10 In the attached figure, "2" indicates that the addition method of the second mode is controlled and pixel unit 200B including phase difference pixels is not read. In addition, pixel unit 200B located diagonally opposite to pixel unit 200B including phase difference pixels is also not read. That is, "A" in the figure is "pixel unit discarded by phase difference pixels" and "B" in the figure is "partner pixel unit".

[0144] In addition, Figure 10 In the attached diagram, "3" indicates control over the addition method in the third mode, and a standard H2V2 addition is performed on four pixel units 200B (16 ordinary pixels). Note that in Figure 10 In this process, a drive signal is applied to the transmission control line (TRG line) and selects pixel 100 in pixel unit 200B.

[0145] As described above, by making the selection control line into multiple lines to control the on / off state of VSL transmission and to control the addition method of the first to third modes, it is possible to prevent phase difference pixels and ordinary pixels from being added together, while reducing the density of phase difference pixels.

[0146] Here, in the method of adding 4 adjacent pixels / H2V2 addition, when performing the addition of 4 adjacent pixels, in order to handle the reading of phase difference pixels, the following steps are executed: Figure 11 and Figure 12 The driver shown.

[0147] exist Figure 11 and Figure 12 In the four pixel units 200B arranged horizontally, the two central B and G pixel units 200B, in addition to ordinary pixels, also include phase difference pixels 100L and 100R, respectively. However, in the G and B pixel units 200B on the left and right, the four pixels are ordinary pixels.

[0148] When performing the addition of 4 adjacent pixels, such as Figure 11 As shown, the charge accumulated in the photoelectric conversion element of one of the four pixels constituting each pixel unit 200B is transferred to a different floating diffusion region 111 for each pixel unit 200B. Figure 11 (S1 in the middle).

[0149] At this time, the pixel signals obtained from the phase difference pixels 100L and 100R of the central B and G pixel units 200B are maintained. Figure 11 (S2 in the middle).

[0150] After that, as Figure 12 As shown, the charge accumulated in the photoelectric conversion elements of the four pixels constituting each pixel unit 200B is transferred to the floating diffusion region 111. Figure 12 (S4-1 to S4-3 in the text).

[0151] Note that in Figure 12 In the middle, as Figure 11 In the transmission, a pixel of the object is represented by a black square, and these pixels are also transmitted again.

[0152] At this point, among the four pixel units 200B arranged horizontally, since the G and B pixel units 200B on the left and right only include ordinary pixels, the four pixels are added together respectively. Figure 12 S5 in the process, and performs subsequent processing on the summed signal obtained as a result ( Figure 12 (S6 in the middle).

[0153] Furthermore, since the central B and G pixel units 200B, in addition to being ordinary pixels of B and G pixels, also include phase difference pixels 100L and 100R respectively, therefore, without using the sum of the four pixels ( Figure 12 In the case of the summed signal obtained by S5), subsequent processing is performed on the pixel signals from the maintained phase difference pixels 100L and 100R. Figure 11 and Figure 12 (S3 in the middle).

[0154] As mentioned above, when performing the addition of 4 adjacent pixels, the same applies to reading phase difference pixels, followed by performing H2V2 addition.

[0155] <3. Variations>

[0156] (Select control line extension)

[0157] The above description has explained the construction of making multiple selection control lines to increase the number of selection control lines to two to control the on / off state of VSL transmission. However, the number of selection control lines is not limited to two, but can be more, such as three.

[0158] Specifically, such as Figure 13 As shown, it can select a control line (SEL)122 ( Figure 13 The A) multiline is converted into three selection control lines (SEL) 122-1, 122-2 and 122-3. Figure 13 (B), and can control the connection between the vertical signal line (VSL) 131 and the floating diffusion area (FD) 111.

[0159] Here, the selection control line 122 is converted into three lines, such as... Figure 14 As shown, it can perform seven reads corresponding to Case 0 to Case 6.

[0160] That is, in Figure 14 In the diagram, the three selection control lines are associated with Case 0 to Case 6 respectively. "○" indicates that the target selection control line 122 is driven, and "×" indicates that the target selection control line 122 is not driven.

[0161] Specifically, when all three selection control lines 122-1 to 122-3 are driven, Figure 15In box 401 of A, four pixel units 200 of the same color can be read as Case 0, and their pixel signals can be added together. Similarly, in Cases 1 to 6, as in Case 0, pixel units 200 of the same color are read according to the drive of the three selection control lines 122-1 to 122-3, and the pixel units 200 of the same color are added together in the floating diffusion area 111 (FD addition). Figure 15 (B to G in the middle).

[0162] As described above, in the current technology, since the connection from the floating diffusion region (FD) 111 to the vertical signal line (VSL) 131 is determined by the physical structure, the source follower (SF) addition method can only be performed in one way. In the technology according to this disclosure, the variation of SF addition can be controlled by making multiple selection control lines and controlling the driving of these lines.

[0163] (Example of constructing a light-blocking pixel)

[0164] Furthermore, the above description has already explained the case of using 2×1OCL pixels as phase difference pixels, but other phase difference pixels such as light-blocking pixels can also be used.

[0165] Here, as Figure 16 As shown, light-blocking pixels 100L and 100R, which are symmetrically arranged in the light-blocking area, can be used. That is, light-blocking pixels 100L and 100R are grouped together as phase difference pixels, and the phase difference is detected based on the light-blocking pixel signals obtained from light-blocking pixels 100L and 100R.

[0166] (Example of a left-right staggered structure)

[0167] Meanwhile, to achieve a high frame rate, a horizontally interleaved structure is used as a method to simultaneously read twice the number of rows as vertical signal lines (VSL). Furthermore, the above description illustrates a method in which, when phase difference pixels 100L and 100R are arranged in pixel array unit 11 according to a predetermined rule, the density of phase difference pixels 100L and 100R is adjusted during drive control via the H2V2 addition method, and pixel units 200 with four pixels of the same color are regularly arranged in pixel array unit 11.

[0168] The following section will describe a construction that satisfies both of these methods and is capable of reading pixel signals in units of predetermined pixels, such as 2×2 pixels.

[0169] Figure 17 Another example of a readout drive for pixel cells 200 arranged in a Bayer array within pixel array cell 11 is shown. Furthermore, for ease of understanding, Figure 18 The arrangement is shown Figure 17 A magnified view of a portion of the region of pixel unit 200 shown.

[0170] In pixel array unit 11, the four pixels constituting pixel unit 200 are all pixels 100 of the same color. Furthermore, in pixel array unit 11, phase difference pixels 100L and 100R are arranged periodically in a predetermined pixel unit (e.g., 16×16 pixel unit).

[0171] The pixel array unit 11 employs a left-right interleaved structure. This left-right interleaved structure is one that divides the left and right sides and reads pixel signals, and it is a structure in which the vertical signal lines (VSLs) 131 connected to two specific floating diffusion regions (FDs) 111 are common. Since the vertical signal lines (VSLs) 131 are common, the column to be read is selected by switching control based on the drive signal (SEL signal) applied to the selection control line (SEL line).

[0172] For example, in Figure 18 In the enlarged image, for multiple vertical signal lines (VSL) 131, there are floating diffusion areas (FD) 111 shared by the upper and lower pixel units 200 on the left and the upper and lower pixel units 200 on the right, and the vertical signal lines (VSL) 131 connected to the two floating diffusion areas (FD) 111 in different columns on the left and right are common. However, in the floating diffusion areas (FD) 111 in the left and right columns connected to the common vertical signal lines (VSL) 131, the common upper pixel unit 200 and lower pixel unit 200 have the same color and are arranged in the same row.

[0173] As described above, in the pixel array unit 11, since the two columns (the left column and the right column) of floating diffusion areas (FD) 111 share a vertical signal line (VSL) 131, in the above drive control, the operation of reading the left floating diffusion area (FD) 111 and the right floating diffusion area (FD) 111 is performed separately in a manner (mode) without performing SF addition.

[0174] Here, we assume that the aforementioned drive control is performed in a scenario where there are two vertical signal lines (VSL) 131 in the column of the floating diffusion region (FD) 111. In this case, when using a non-left-right interleaving structure, four rows (FD rows) can be read simultaneously, and when using a left-right interleaving structure, eight rows (FD rows) can be read simultaneously, which is twice the number of rows that can be read in the non-left-right interleaving structure. That is, the left-right interleaving structure is designed for reading at a high frame rate.

[0175] In the pixel array unit 11, among the pixel units 200 that share the same floating diffusion area (FD) 111, the pixel unit 200 that includes four R, G, or B pixels 100 corresponding to the upper R, G, or B color filter is called the top pixel unit 200, and the pixel unit 200 that includes four R, G, or B pixels 100 corresponding to the lower R, G, or B color filter is called the bottom pixel unit 200.

[0176] When reading the top pixel unit 200, the floating diffuser (FD) 111, which is a non-additive object, is connected to the selection control line as SELQBINSF, and other floating diffuser (FD) 111 are connected to the selection control line as SEL. In the case of using a left-right staggered structure, since the left and right columns need to be controlled independently in addition to the above, selection control lines as SEL and SELQBINSF are prepared separately for the left and right columns.

[0177] exist Figure 17 and Figure 18 In the diagram, the selection control lines used in the left column are labeled "*_EVN", and the selection control lines used in the right column are labeled "*_ODD". That is, four selection control lines (SEL lines) are prepared: SEL_EVN, SELQBINSF_EVN, SEL_ODD, and SELQBINSF_ODD.

[0178] Furthermore, for the transmission control line (TRG line) used to select the pixel 100 in the pixel unit 200 of the shared floating diffusion area (FD) 111, since the left and right columns need to be controlled independently, the transmission control line is prepared for the left and right columns in the same way as the selection control line (SEL line).

[0179] exist Figure 17 and Figure 18 In the diagram, "*_T" is described in the transfer control line used for the top-side pixel unit, "*_B" is described in the transfer control line used for the bottom-side pixel unit, and "*_ZAF" is described in the transfer control line used for pixels adjacent to phase difference pixels 100L and 100R. That is, six transfer control lines (TRG lines) are prepared: TRG_EVN_T, TRG_EVN_B, TRG_EVN_ZAF, TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF.

[0180] These transmission control lines and selection control lines are summarized below.

[0181] (Transmission control line)

[0182] The drive signal (TRG signal) for the transfer control (TRG control) of pixel 100 in the top pixel unit 200 of the left column is applied (output) to TRG_EVN_T. The drive signal for the transfer control of pixel 100 in the bottom pixel unit 200 of the left column is applied to TRG_EVN_B.

[0183] A drive signal for controlling the transfer of pixels 100 in the top pixel unit 200 of the right column is applied to TRG_ODD_T. A drive signal for controlling the transfer of pixels 100 in the bottom pixel unit 200 of the right column is applied to TRG_ODD_B.

[0184] Drive signals for transmission control of pixels 100 adjacent to phase difference pixels 100L and 100R in the left column are applied to TRG_EVN_ZAF. Drive signals for transmission control of pixels 100 adjacent to phase difference pixels 100L and 100R in the right column are applied to TRG_ODD_ZAF.

[0185] (Select control lines)

[0186] The signal levels of the transmission control lines TRG_EVN_T, TRG_EVN_B, and TRG_EVN_ZAF are applied (output to) SEL_EVN via a logic-established drive signal (SEL signal). That is, in SEL_EVN, when "TRG_EVN_*" is in the Read state, the output is based on the input signal as RSEL; otherwise, the output is fixed at a low level.

[0187] The drive signal established by logic is applied to SEL_ODD based on the signal levels of the transmission control lines TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF. That is, in SEL_ODD, when "TRG_ODD_*" is in the Read state, the output is performed according to the input signal RSEL, while when "TRG_ODD_*" is in a state other than Read, the output is fixed at a low level.

[0188] The drive signal established by logic is applied to SELQBINSF_EVN based on the signal levels of the transmission control lines TRG_EVN_T, TRG_EVN_B, and TRG_EVN_ZAF. That is, in SELQBINSF_EVN, when "TRG_EVN_*" is in the Read state, the output is based on the input signal for RSELQBINSF; when "TRG_EVN_*" is in a state other than Read, the output is fixed at a low level. Note, for example, when the floating diffusion region (FD) 111 is read as a non-additive object in the above drive control, the input signal for RSELQBINSF needs to be fixed at a low level.

[0189] The drive signal established by logic is applied to SELQBINSF_ODD based on the signal levels of the transmission control lines TRG_ODD_T, TRG_ODD_B, and TRG_ODD_ZAF. That is, in SELQBINSF_ODD, when "TRG_ODD_*" is in the Read state, the output is based on the input signal as RSELQBINSF, while when "TRG_ODD_*" is in a state other than Read, the output is fixed at a low level. Note, for example, when reading the floating diffusion region (FD) 111 as a non-additive object in the above drive control, it is necessary to fix the input signal as RSELQBINSF at a low level.

[0190] In this way, when performing the above-mentioned drive control, by controlling the drive signals (TRG signals and SEL signals) applied to the transmission control lines (TRG_EVN_B, TRG_EVN_T, TRG_EVN_ZAF, TRG_ODD_B, TRG_ODD_T and TRG_ODD_ZAF) and the selection control lines (SEL_EVN, SELQBINSF_EVN, SEL_ODD and SELQBINSF_ODD), the phase difference pixels 100L and 100R are sparsed and read at a certain period to prevent the density of the phase difference pixels 100L and 100R, which would cause defects in the captured image, from becoming too high.

[0191] At this point, by controlling the drive signal applied to the selection control line and sparsely positioning the pixels 100 diagonally with the sparsed phase difference pixels 100L and 100R in a similar manner to align the centroids of the pixels after H2V2 addition, it is possible to suppress defect density while keeping the color centroid centered. This sparse method is as described in the drive control above.

[0192] As mentioned above, in Figure 17 and Figure 18In the illustrated configuration, the aforementioned drive control can be achieved at a high frame rate with fewer vertical signal lines (VSLs). Furthermore, the point defect density originating from phase difference pixels in the captured image can be controlled. Additionally, pixel signals can be read in predetermined pixel units, such as 2×2 pixels.

[0193] (Examples of other methods)

[0194] Furthermore, although the above explanation used three-exposure HDR / H2V2 addition and adjacent 4-pixel addition / H2V2 addition as examples, other methods using H2V2 addition can also be used. Moreover, H2V2 addition is an example of horizontal / vertical addition, and is not limited to the addition of two pixels in the horizontal direction and two pixels in the vertical direction. Any method can be used as long as the pixel signals of predetermined pixels in the horizontal direction and predetermined pixels in the vertical direction are added horizontally / vertically.

[0195] (Achieving both high-speed frame rate driving and PDAF)

[0196] Here, to achieve a high frame rate, it is necessary to reduce the total number of pixels by pixel summation and increase the number of simultaneous AD conversions in the horizontal direction. However, to achieve both high frame rate (multi-pixel summation) driving and phase detection AF (PDAF), it is necessary to perform driving that leaves the phase difference pixels (2×1OCL pixels) during pixel summation. Therefore, as the current implementation method, by using a selection control line for each FD during SF summation, only the phase difference pixels (2×1OCL pixels) are selected by Mux control before AD conversion.

[0197] In this implementation method, since there are limitations to the simultaneous increase of AD conversion in the horizontal direction, this problem must be addressed. However, in the technology according to this disclosure, this problem can be solved by making the selection control line into multiple lines.

[0198] As described above, according to the technology of this disclosure, when using a configuration in which phase difference pixels 100L and 100R are included in a plurality of pixels 100 arranged in two dimensions in a pixel array unit 11 (including pixel unit 200 of the plurality of pixels 100), when reading the plurality of pixels 100, when performing horizontal / vertical addition on the pixel signals of predetermined pixels 100 in the horizontal direction and the pixel signals of predetermined pixels 100 in the vertical direction, the degradation of phase difference information can be suppressed by partially not adding phase difference pixels 100L and 100R.

[0199] In particular, in recent years, the pixel count of image sensors used in mobile devices has been continuously increasing, necessitating high-speed and low-power image downsizing techniques for motion image formats. Currently, when performing near-adjacent 4-pixel addition and H2V2 addition as image downsizing techniques, phase difference information is lost and negatively impacts image quality because phase difference pixels are also added to regular pixels. Furthermore, currently, after using triple-exposure HDR, further downsizing results in pixel sparsity (H2V2 sparsity), leading to image quality issues such as signal-to-noise ratio (SN ratio) degradation and aliasing.

[0200] However, in the technology according to this disclosure, when performing H2V2 addition using a triple-exposure HDR / H2V2 addition method or a nearby adjacent 4-pixel addition / H2V2 addition method, by mixing regions where only phase difference pixels are added or selected and regions where ordinary pixels are added or selected, the density of phase difference pixels does not increase excessively even when the image is reduced. Therefore, with this construction, high-speed and low-power operation can be achieved while suppressing image quality degradation as much as possible, and phase difference information is not lost even when the image is reduced.

[0201] Furthermore, in the technology according to this disclosure, when using the triple-exposure HDR / H2V2 addition method, the same exposure signals are simulatedly added while maintaining the same shutter control as the current triple-exposure HDR. Therefore, this configuration can improve readout speed while increasing the SN ratio, and further suppress power consumption.

[0202] However, when the H2V2 addition is performed uniformly on all pixels, in the case of embedded phase difference pixels, the phase difference information deteriorates due to the addition of phase difference pixels and ordinary pixels, and the density of pixels affected by the phase difference pixels increases. This raises concerns about the adverse effects on image quality caused by the correction. To avoid this adverse effect, the degradation of phase difference information and image quality is suppressed by performing the drive control of the selection control lines, which are made into multiple lines.

[0203] Note that Patent Document 1 disclosed above discloses a method for adding phase difference detection and HDR in full phase difference detection, but certainly does not disclose or imply a method for selectively adding embedded phase difference pixels.

[0204] Furthermore, Patent Document 2 discloses a method for arranging phase difference pixels according to an addition method, but it neither discloses nor implies a construction for selecting the addition pixels by means of wiring of a selection control line and drive control, as is the case with the technology according to this disclosure.

[0205] <4. Structure of Electronic Devices>

[0206] Figure 19 This is a block diagram illustrating an example of the construction of an electronic device having a solid-state camera device applicable to the technology according to this disclosure.

[0207] Electronic device 1000 is, for example, an electronic device with a camera function, such as a camera accessory like a digital camera or camcorder, or a portable terminal device like a smartphone or tablet.

[0208] The electronic device 1000 includes a lens unit 1011, a camera unit 1012, a signal processing unit 1013, a control unit 1014, a display unit 1015, a recording unit 1016, an operation unit 1017, a communication unit 1018, a power supply unit 1019, and a drive unit 1020.

[0209] Furthermore, in the electronic device 1000, the signal processing unit 1013, the control unit 1014, the display unit 1015, the recording unit 1016, the operation unit 1017, the communication unit 1018, and the power supply unit 1019 are connected to each other via the bus 1021.

[0210] The lens unit 1011 includes a zoom lens and a focusing lens, and collects light from the subject. The light collected by the lens unit 1011 (subject light) is incident on the imaging unit 1012.

[0211] The camera unit 1012 includes a solid-state camera device (e.g., a camera that is suitable for the technology of this disclosure) Figure 1 (Solid-state camera device 10). In the camera unit 1012, the light (subject light) received by the solid-state camera device via the lens unit 1011 is photoelectrically converted into an electrical signal, and the resulting signal is provided to the signal processing unit 1013.

[0212] Note that, as a plurality of pixels arranged in a predetermined array pattern, the pixel array unit of the solid-state camera device includes pixels (ordinary pixels) that generate signals for generating captured images corresponding to the subject light and pixels (phase difference pixels) that generate signals for performing phase difference detection.

[0213] For example, in the aforementioned solid-state camera device 10 ( Figure 1 In this context, ordinary pixels correspond to R pixel 100 (R pixel unit 200), G pixel 100 (G pixel unit 200) and B pixel 100 (B pixel unit 200), while phase difference pixels correspond to phase difference pixels 100L and 100R and light-blocking pixels 100L and 100R.

[0214] The signal processing unit 1013 is a signal processing circuit for processing signals provided from the camera unit 1012. For example, the signal processing unit 1013 may be configured as a digital signal processor (DSP) circuit, etc.

[0215] The signal processing unit 1013 processes the signal from the camera unit 1012, generates image data of still or moving images, and provides it to the display unit 1015 or the recording unit 1016. Furthermore, based on the signal from the camera unit 1012 (the phase difference pixels of the image sensor), the signal processing unit 1013 generates data for detecting the phase difference (phase difference detection data) and provides it to the control unit 1014.

[0216] For example, the control unit 1014 is configured as a central processing unit (CPU) or a microprocessor. The control unit 1014 controls the operation of each unit of the electronic device 1000.

[0217] For example, the display unit 1015 is configured as a display device such as a liquid crystal panel or an organic electroluminescent (EL) panel. The display unit 1015 processes image data provided from the signal processing unit 1013 and displays still images or moving images captured by the camera unit 1012.

[0218] For example, the recording unit 1016 is configured as a recording medium such as a semiconductor memory or a hard disk. The recording unit 1016 records image data provided by the signal processing unit 1013. Furthermore, the recording unit 1016 provides the recorded image data under the control of the control unit 1014.

[0219] For example, in addition to physical buttons, the operation unit 1017 is also configured as a touch panel combined with the display unit 1015. The operation unit 1017 outputs operation commands for various functions of the electronic device 1000 based on the user's operation. The control unit 1014 controls the operation of each unit based on the operation commands provided from the operation unit 1017.

[0220] For example, the communication unit 1018 is configured as a communication interface circuit, etc. The communication unit 1018 exchanges data with external devices via wireless or wired communication according to a predetermined communication standard.

[0221] The power supply unit 1019 appropriately supplies various power sources, which serve as the operating power sources for the signal processing unit 1013, the control unit 1014, the display unit 1015, the recording unit 1016, the operation unit 1017, and the communication unit 1018, to these power supply targets.

[0222] Furthermore, the control unit 1014 detects the phase difference between two images based on the phase difference detection data provided by the signal processing unit 1013. Based on the phase difference detection result, the control unit 1014 determines whether the target object to be focused (the object to be focused on) is in focus. If the object to be focused is not in focus, the control unit 1014 calculates the focus offset (defocusing amount) and provides it to the drive unit 1020.

[0223] For example, the drive unit 1020 is composed of a motor and drives the lens unit 1011, which includes a zoom lens or a focusing lens.

[0224] The drive unit 1020 calculates the driving amount of the focusing lens of the lens unit 1011 based on the defocusing amount provided from the control unit 1014, and moves the focusing lens according to the driving amount. Note that when the object being focused is in focus, the drive unit 1020 maintains the current position of the focusing lens.

[0225] The electronic device 1000 is constructed as described above.

[0226] <5. Examples of the use of solid-state camera devices>

[0227] Figure 20 This is a diagram illustrating an example of the use of a solid-state camera device applicable to the technology of this disclosure.

[0228] For example, as described below, the solid-state camera device 10 can be used to sense various types of light, such as visible light, infrared light, ultraviolet light, and X-rays. That is, as... Figure 20 As shown, the solid-state camera device 10 can be used not only in the field of capturing and appreciating images, but also in fields such as transportation, home appliances, healthcare, security, beauty, sports, and agriculture.

[0229] Specifically, in the field of appreciation, the solid-state camera device 10 can be used, for example, as a device for capturing images for appreciation (e.g., Figure 19 Electronic devices (1000), such as digital cameras, smartphones, or mobile phones with camera functions.

[0230] In the field of transportation, for example, for safe driving such as automatic stopping and for recognizing the driver's state, solid-state camera device 10 can be used in transportation equipment, such as vehicle-mounted sensors that capture images of the front, rear, surroundings and interior of a car; surveillance cameras that monitor moving vehicles and roads; and distance measuring sensors that measure the distance between vehicles.

[0231] In the field of home appliances, for example, solid-state camera device 10 can be used in devices such as TV receivers, refrigerators, and air conditioners to capture images of user gestures and operate devices based on those gestures. Furthermore, in the field of healthcare, for example, solid-state camera device 10 can be used in healthcare devices such as endoscopes and devices that perform angiography by receiving infrared light.

[0232] In the security field, for example, the solid-state camera device 10 can be used in security equipment, such as surveillance cameras for crime prevention or cameras for personal authentication. Furthermore, in the beauty field, for example, the solid-state camera device 10 can be used in beauty equipment, such as skin measuring instruments for imaging the skin or microscopes for imaging the scalp.

[0233] In the field of sports, for example, the solid-state camera device 10 can be used in sports equipment, such as action cameras and wearable cameras for sports applications. Furthermore, in the field of agriculture, for example, the solid-state camera device 10 can be used in agricultural equipment, such as cameras for monitoring the condition of fields and crops.

[0234] <6. Examples of applications of moving objects>

[0235] The technology disclosed herein (the Technology) is applicable to a wide variety of products. For example, the Technology disclosed herein can be implemented as a device mounted on any type of mobile body, such as a car, electric car, hybrid electric car, motorcycle, bicycle, personal mobile device, airplane, drone, ship, and robot.

[0236] Figure 21 This is a block diagram illustrating a schematic construction example of a vehicle control system, which is an example of a mobile body control system to which the technology according to this disclosure can be applied.

[0237] The vehicle control system 12000 includes multiple electronic control units connected via a communication network 12001. Figure 21 In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and a comprehensive control unit 12050. Furthermore, as part of the functional structure of the comprehensive control unit 12050, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown.

[0238] The drive system control unit 12010 controls the operation of equipment related to the vehicle's drive system according to various programs. For example, the drive system control unit 12010 serves as: a drive force generating device for generating the vehicle's drive force, such as an internal combustion engine or drive motor; a drive force transmission mechanism for transmitting the drive force to the wheels; a steering mechanism for adjusting the vehicle's steering angle; and control devices such as braking devices for generating the vehicle's braking force.

[0239] The body system control unit 12020 controls the operation of various devices installed on the vehicle body according to various programs. For example, the body system control unit 12020 acts as a control device for devices such as: keyless entry systems; smart key systems; power windows; or various lights such as headlights, taillights, brake lights, turn signals, or fog lights. In this case, radio waves or signals from various switches sent from a portable device that replaces the key can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signal inputs and controls the vehicle's door locks, power windows, lights, etc.

[0240] The exterior information detection unit 12030 detects information about the exterior of the vehicle on which the vehicle control system 12000 is installed. For example, the exterior information detection unit 12030 is connected to a camera unit 12031. The exterior information detection unit 12030 causes the camera unit 12031 to capture images of the exterior of the vehicle and receives the captured images. Based on the received images, the exterior information detection unit 12030 can perform object detection processing or distance detection processing for pedestrians, vehicles, obstacles, signs, or letters on the road surface.

[0241] The camera unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The camera unit 12031 can output this electrical signal as an image or as ranging information. Furthermore, the light received by the camera unit 12031 can be visible light or non-visible light such as infrared light.

[0242] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, the in-vehicle information detection unit 12040 is connected to a driver state detection unit 12041 for detecting the driver's state. The driver state detection unit 12041 may include, for example, a camera for capturing images of the driver, and based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's level of fatigue or concentration, or determine whether the driver is asleep.

[0243] Based on information acquired by the external information detection unit 12030 or the internal information detection unit 12040, the microcomputer 12051 can calculate the control target values ​​for the drive force generating device, steering mechanism, or braking device, and can output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control for the purpose of realizing the functions of an advanced driver assistance system (ADAS), including: collision avoidance or impact mitigation, following based on inter-vehicle distance, speed maintenance, collision warning, and lane departure warning.

[0244] Furthermore, by controlling the drive force generating device, steering mechanism, or braking device based on information about the vehicle's surroundings obtained by the external information detection unit 12030 or the internal information detection unit 12040, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving that does not depend on the driver's operation.

[0245] Furthermore, based on the information about the exterior of the vehicle acquired by the external information detection unit 12030, the microcomputer 12051 can output control commands to the body system control unit 12020. For example, the microcomputer 12051 can control the headlights according to the position of the vehicle in front or oncoming vehicle detected by the external information detection unit 12030, thereby performing cooperative control for the purpose of anti-glare, such as switching the high beams to the low beams.

[0246] The sound / image output unit 12052 sends an output signal of at least one of sound or image to an output device capable of visually or audibly notifying passengers in the vehicle or outside the vehicle of information. Figure 21 In the example, audio speaker 12061, display unit 12062, and instrument panel 12063 are illustrated as output devices. Display unit 12062 may include at least one of an on-board display or a head-up display, for example.

[0247] Figure 22 This is a diagram showing an example of the mounting location of the camera unit 12031.

[0248] exist Figure 22 In the middle, as a camera unit 12031, camera units 12101, 12102, 12103, 12104 and 12105 are provided.

[0249] Camera units 12101, 12102, 12103, 12104, and 12105 are, for example, installed on the front nose, rearview mirrors, rear bumper, rear door, or upper part of the windshield inside the vehicle 12100. Camera unit 12101 installed on the front nose and camera unit 12105 installed on the upper part of the windshield inside the vehicle primarily acquire images of the front of the vehicle 12100. Camera units 12102 and 12103 installed on the rearview mirrors primarily acquire images of the sides of the vehicle 12100. Camera unit 12104 installed on the rear bumper or rear door primarily acquires images of the rear of the vehicle 12100. Camera unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used to detect vehicles ahead, pedestrians, obstacles, traffic lights, traffic signs, or lanes, etc.

[0250] Notice, Figure 22 An example of the camera range of camera units 12101 to 12104 is shown. Camera range 12111 represents the camera range of camera unit 12101 located at the front nose; camera ranges 12112 and 12113 represent the camera ranges of camera units 12102 and 12103 located at the rearview mirrors, respectively; and camera range 12114 represents the camera range of camera unit 12104 located at the rear bumper or rear door. For example, by superimposing the image data captured by camera units 12101 to 12104, a bird's-eye view of the vehicle 12100 as seen from above can be obtained.

[0251] At least one of the camera units 12101 to 12104 may have the function of acquiring distance information. For example, at least one of the camera units 12101 to 12104 may be a stereo camera including multiple camera elements, or may be a camera element having pixels for detecting phase difference.

[0252] For example, based on distance information obtained from camera units 12101 to 12104, by obtaining the distance to each three-dimensional object within the camera range 12111 to 12114 and the change of that distance over time (relative speed to vehicle 12100), microcomputer 12051 can identify the three-dimensional object that is closest to vehicle 12100 on the driving road and is traveling in almost the same direction as vehicle 12100 at a predetermined speed (e.g., greater than or equal to 0 km / h) as the preceding vehicle. Furthermore, microcomputer 12051 can preset the vehicle-to-the-leader distance and can perform automatic braking control (including follow-stop control) and automatic acceleration control (including follow-start control), etc. In this way, cooperative control for the purpose of autonomous driving, such as autonomous driving without driver intervention, can be performed.

[0253] For example, based on distance information obtained from camera units 12101-12104, microcomputer 12051 can classify three-dimensional object data into three-dimensional object data such as two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, and utility poles, and extract the classified three-dimensional object data. It can then use the extracted three-dimensional object data to automatically avoid obstacles. For example, microcomputer 12051 distinguishes obstacles around vehicle 12100 into obstacles that are visible to the driver of vehicle 12100 and obstacles that are difficult to see. Then, microcomputer 12051 can determine the collision risk, representing the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, microcomputer 12051 can provide assisted driving to avoid collisions by outputting a warning to the driver via audio speaker 12061 or display unit 12062, or by performing forced deceleration and evasive steering via drive system control unit 12010.

[0254] At least one of the camera units 12101 to 12104 can be an infrared camera for detecting infrared light. For example, the microcomputer 12051 can identify a pedestrian by determining whether a pedestrian exists in the images captured by the camera units 12101 to 12104. This pedestrian identification is performed, for example, by extracting feature points from the images captured by the camera units 12101 to 12104 (which are infrared cameras); and determining whether the object is a pedestrian by performing pattern matching processing on a series of feature points representing the outline of the object. When the microcomputer 12051 determines that a pedestrian exists in the images captured by the camera units 12101 to 12104 and identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 to overlay and display a rectangular outline for emphasis on the identified pedestrian. Furthermore, the sound / image output unit 12052 can also control the display unit 12062 to display an icon or similar symbol representing the pedestrian at a desired location.

[0255] The above describes an example of a vehicle control system to which the technology according to this disclosure is applicable. The technology according to this disclosure is applicable to the camera unit 12031 in the above configuration. Specifically, the solid-state camera device 10 can be applied to camera units 12101 to 12105. By applying the technology according to this disclosure to camera unit 12031, for example, degradation of phase difference information and image quality can be suppressed. Therefore, by improving visibility, objects such as pedestrians, vehicles, obstacles, signs, or letters on the road surface can be identified more accurately.

[0256] <7. Examples of the Application of Endoscopic Surgical Systems>

[0257] Figure 23This is a diagram illustrating an example of a schematic construction of an endoscopic surgical system capable of applying the technology (the present technology) according to this disclosure.

[0258] Figure 23 The illustration shows a surgeon (physician) 11131 performing surgery on a patient 11132 on a bed 11133 using an endoscopic surgical system 11000. As shown, the endoscopic surgical system 11000 includes: an endoscope 11100; other surgical instruments 11110 such as an insufflation tube 11111 and an energy delivery device 11112; a support arm assembly 11120 supporting the endoscope 11100; and a trolley 11200 equipped with various devices for endoscopic surgery.

[0259] Endoscope 11100 includes: a tube 11101, a region having a predetermined length extending from the distal end of the tube 11101 being inserted into the body cavity of a patient 11132; and a camera 11102, the camera 11102 being connected to the proximal end of the tube 11101. In the example shown, an endoscope 11100 is illustrated as a so-called rigid endoscope having a rigid tube 11101, but endoscope 11100 can also be configured as a so-called flexible endoscope having a flexible tube.

[0260] The distal end of the endoscope tube 11101 has an opening for mounting an objective lens. A light source device 11203 is connected to the endoscope 11100 such that light generated by the light source device 11203 is guided to the distal end of the endoscope tube 11101 via a light guide extending within the tube, and this light is then projected onto the object of observation within the body cavity of the patient 11132 via the aforementioned objective lens. Note that the endoscope 11100 may be a forward-viewing endoscope, or an oblique-viewing endoscope, or a side-viewing endoscope.

[0261] An optical system and an imaging element are provided within the camera 11102, such that reflected light (observation light) from the observed target is converged onto the imaging element by the optical system. The observation light undergoes photoelectric conversion by the imaging element, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. This image signal is sent as raw data to the CCU (Camera Control Unit) 11201.

[0262] The CCU 11201 consists of a central processing unit (CPU) and a graphics processing unit (GPU), and the CCU 11201 controls the operation of the endoscope 11100 and the display device 11202 as a whole. Furthermore, the CCU 11201 receives image signals from the camera 11102 and performs various image processing operations on the image signals, such as image processing (de-mosaicing), for displaying images based on the image signals.

[0263] Under the control of CCU 11201, display device 11202 displays an image based on an image signal (image processed by CCU 11201).

[0264] For example, the light source device 11203 is composed of a light source such as a light emitting diode (LED), and provides illumination light for taking images of surgical sites, etc., to the endoscope 11100.

[0265] Input device 11204 is the input interface of endoscopic surgery system 11000. Users can input various information and commands into endoscopic surgery system 11000 through input device 11204. For example, users can input commands to change the imaging conditions of endoscope 11100 (type of illumination light, magnification, focal length, etc.).

[0266] The treatment tool control device 11205 controls the actuation of the energy treatment tool 11112 used for tissue cauterization, cutting, or vascular sealing. The pneumoperitoneum device 11206 delivers gas into the patient's body cavity 11132 via the pneumoperitoneum tube 11111 to inflate the cavity, thereby ensuring the field of vision of the endoscope 11100 and ensuring the surgeon's working space. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various formats such as text, images, and charts.

[0267] Note that the light source device 11203, which provides illumination light to the endoscope 11100 for capturing images of the surgical site, may include, for example, a white light source, which is composed of an LED, a laser light source, or a combination of LEDs and laser light sources. When the white light source is composed of a combination of red, green, and blue (RGB) laser light sources, the light source device 11203 can adjust the white balance of the captured image because the output intensity and timing of each color (wavelength) can be controlled with high precision. Furthermore, in this case, by illuminating the object of observation with lasers from each of the RGB laser light sources in a time-division manner and controlling the driving of the imaging element of the camera 11102 in sync with the illumination timing, images corresponding to each of the R, G, and B colors can also be captured in a time-division manner. According to this method, color images can be obtained even if a color filter is not provided in the imaging element.

[0268] Furthermore, the drive of the light source device 11203 can be controlled to change the light intensity to be output at predetermined time intervals. By controlling the drive of the imaging element of the camera 11102 in sync with the moment of light intensity change to acquire images in a time-division manner, and then synthesizing these images, high dynamic range images without so-called black defects and whiteouts can be produced.

[0269] Furthermore, the light source device 11203 can be configured to provide light in a predetermined wavelength band corresponding to special light observation. In special light observation, for example, so-called narrowband imaging is performed, where a predetermined tissue, such as blood vessels in the mucosal surface, is imaged with high contrast by irradiating the predetermined tissue with narrowband light (i.e., white light) that has a narrower bandwidth than the irradiation light used in ordinary observation, utilizing the wavelength dependence of light absorption in body tissue. Alternatively, in special light observation, fluorescence observation can be performed to obtain an image based on the fluorescence generated by irradiation with excitation light. In fluorescence observation, excitation light can be irradiated onto body tissue and fluorescence from the body tissue can be observed (autofluorescence observation), or a fluorescence image can be obtained by locally injecting a reagent such as indocyanine green (ICG) into body tissue and irradiating the body tissue with excitation light corresponding to the fluorescence wavelength of the reagent. The light source device 11203 can be configured to provide narrowband light and / or excitation light corresponding to such special light observation.

[0270] Figure 24 It shows Figure 23 A block diagram illustrating an example of the functional configuration of the camera 11102 and CCU 11201.

[0271] Camera 11102 includes a lens unit 11401, an image capture unit 11402, a drive unit 11403, a communication unit 11404, and a camera control unit 11405. CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. Camera 11102 and CCU 11201 are communicatively connected to each other via a transmission cable 11400.

[0272] Lens unit 11401 is an optical system disposed at the connection point with lens barrel 11101. Observation light obtained from the distal end of lens barrel 11101 is guided to camera 11102 and incident on lens unit 11401. Lens unit 11401 is composed of multiple lenses, including zoom lenses and focusing lenses.

[0273] The camera unit 11402 includes camera elements. The number of camera elements included in the camera unit 11402 can be one (so-called single-plate type) or multiple (so-called multi-plate type). For example, when the camera unit 11402 is configured as a multi-plate type camera unit, individual camera elements can generate image signals corresponding to R, G, and B respectively, and a color image can be obtained by synthesizing these image signals. Alternatively, the camera unit 11402 may also have a pair of camera elements to acquire right-eye image signals and left-eye image signals corresponding to three-dimensional (3D) display respectively. By performing 3D display, the surgeon 11131 can more accurately determine the depth of living tissue in the surgical site. Note that when the camera unit 11402 is configured as a multi-plate type camera element, multiple lens unit 11401 systems can also be provided corresponding to each camera element.

[0274] Furthermore, the camera unit 11402 does not necessarily need to be mounted on the camera 11102. For example, the camera unit 11402 can be mounted inside the lens barrel 11101 immediately behind the objective lens.

[0275] The drive unit 11403 is composed of an actuator, and under the control of the camera control unit 11405, the drive unit 11403 moves the zoom lens and focusing lens of the lens unit 11401 a predetermined distance along the optical axis. With this configuration, the magnification and focus of the image captured by the camera unit 11402 can be appropriately adjusted.

[0276] The communication unit 11404 is composed of a communication device used to exchange various types of information between CCUs 11201. The communication unit 11404 transmits the image signal acquired from the camera unit 11402 as RAW data to the CCU 11201 via the transmission cable 11400.

[0277] Additionally, the communication unit 11404 receives control signals from the CCU 11201 for controlling the camera 11102 and provides these control signals to the camera control unit 11405. For example, the control signals include information related to shooting conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value during shooting, and / or information specifying the magnification and focus of the captured image.

[0278] Note that imaging conditions such as frame rate, exposure value, magnification, and focus can be appropriately specified by the user, or can be automatically set by the control unit 11413 of CCU 11201 based on the acquired image signal. In the latter case, so-called automatic exposure (AE), automatic focus (AF), and automatic white balance (AWB) functions are installed in endoscope 11100.

[0279] The camera control unit 11405 controls the driving of the camera 11102 based on the control signals received from the CCU 11201 via the communication unit 11404.

[0280] The communication unit 11411 is composed of a communication device and is used to exchange various types of information with the camera 11102. The communication unit 11411 receives image signals transmitted from the camera 11102 through the transmission cable 11400.

[0281] In addition, the communication unit 11411 transmits control signals for controlling the camera 11102 to the camera 11102. The image signals and control signals can be transmitted via electrical communication or optical communication, etc.

[0282] The image processing unit 11412 performs various image processing on the image signal transmitted from the camera 11102 as RAW data.

[0283] The control unit 11413 performs various controls related to imaging the surgical site, etc., through the endoscope 11100 and displaying the images obtained by imaging the surgical site, etc. For example, the control unit 11413 generates control signals for controlling the drive of the camera 11102.

[0284] Furthermore, the control unit 11413 uses the image signal that has already been processed by the image processing unit 11412 to display the captured image of the surgical site on the display device 11202. At this time, the control unit 11413 uses various image recognition technologies to identify various objects in the captured image. For example, by detecting the shape and color of the edges of objects included in the captured image, the control unit 11413 can identify surgical instruments such as forceps, specific living tissue sites, bleeding, and fogging when using the energy treatment tool 11112. When the display device 11202 displays the captured image, the control unit 11413 can use the recognition results to overlay and display various surgical support information on the image of the surgical site. By overlaying and displaying surgical support information and presenting it to the surgeon 11131, the workload of the surgeon 11131 can be reduced, and the surgeon 11131 can perform the surgery reliably.

[0285] The transmission cable 11400 connecting the camera 11102 and the CCU 11201 is an electrical signal cable for electrical signal communication, an optical fiber for optical communication, or a composite cable for both electrical and optical communication.

[0286] Here, in the example shown, although communication is conducted via wired communication using transmission cable 11400, communication between camera 11102 and CCU 11201 can also be conducted wirelessly.

[0287] The above describes an example of an endoscopic surgical system to which the technology according to this disclosure is applicable. The technology according to this disclosure is applicable to the endoscope 11100 in the above configuration. Specifically, the solid-state imaging device 10 is applicable to the imaging unit 11402 of the camera 11102. By applying the technology according to this disclosure to the imaging unit 11402, for example, degradation of phase difference information and image quality can be suppressed to obtain a clearer image of the surgical site, thereby enabling the surgeon to reliably examine the surgical site.

[0288] Note that although an endoscopic surgical system has been used as an example here, the technology disclosed herein can also be applied to other microsurgical systems, for example.

[0289] Note that the embodiments based on the technology disclosed herein are not limited to the above embodiments, and various modifications can be made without departing from the scope of the technology disclosed herein.

[0290] Furthermore, the technology according to this disclosure can have the following structure. (1)

[0292] A solid-state camera device, comprising:

[0293] A pixel array unit, wherein multiple pixels are arranged in a two-dimensional manner, wherein...

[0294] The plurality of pixels includes phase difference pixels for phase difference detection.

[0295] The pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are arranged regularly, and

[0296] When reading the plurality of pixels, when the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are added horizontally / vertically, the phase difference pixels are partially not added. (2)

[0298] According to the solid-state camera device described in (1), wherein,

[0299] The density of the phase difference pixels after horizontal / vertical addition is the same as, almost the same as, or lower than the density of the phase difference pixels before horizontal / vertical addition. (3)

[0301] According to the solid-state camera device described in (1) or (2), wherein,

[0302] Multiple selection control lines are set in the horizontal direction to select the connection from the floating diffusion region formed for each of the plurality of pixel units in the vertical direction to the vertical signal line, and

[0303] During the horizontal / vertical addition, the drive of the multiple selection control lines is controlled. (4)

[0305] The solid-state camera device according to any one of (1) to (3), wherein,

[0306] For each pixel included in the pixel unit, the exposure time is adjusted, and

[0307] When reading the plurality of pixels, a high dynamic range (HDR) signal is generated by synthesizing pixel signals with the same color and different exposure times obtained by horizontal / vertical addition. (5)

[0309] According to the solid-state camera device described in (4), wherein,

[0310] Among the plurality of pixels, the phase difference pixel, which includes any one of the plurality of photoelectric conversion elements formed for a lens on a chip, is an adjacent pixel with the same color and is included in adjacent different pixel units. (6)

[0312] According to the solid-state camera device described in (5), wherein,

[0313] The pixel unit comprises four pixels in a 2×2 grid.

[0314] The phase difference pixel includes a first phase difference pixel and a second phase difference pixel, wherein the first phase difference pixel and the second phase difference pixel are respectively two 2×1 pixels included in the left and right adjacent pixel units, and

[0315] The first phase difference pixel and the second phase difference pixel are adjusted to have the same exposure time. (7)

[0317] According to the solid-state camera device described in (4), wherein,

[0318] The phase difference pixels are each constructed as light-shielding pixels. (8)

[0320] The solid-state camera device according to any one of (1) to (3), wherein,

[0321] In the pixel unit, all pixels of the same color have the same exposure time, and

[0322] When reading the plurality of pixels, the horizontal / vertical addition is performed after the pixel signals of pixels of the same color included in the pixel unit are added together. (9)

[0324] According to the solid-state camera device described in (8), wherein,

[0325] Among the plurality of pixels, the phase difference pixel, which includes any one of the plurality of photoelectric conversion elements formed for a lens on a chip, is an adjacent pixel with the same color and is included in adjacent different pixel units. (10)

[0327] According to the solid-state camera device described in (9), wherein,

[0328] The pixel unit comprises four pixels in a 2×2 grid, and

[0329] The phase difference pixel includes a first phase difference pixel and a second phase difference pixel, wherein the first phase difference pixel and the second phase difference pixel are respectively two pixels of 2×1 included in the left and right adjacent pixel units. (11)

[0331] According to the solid-state camera device described in (8), wherein,

[0332] The phase difference pixels are each constructed as light-shielding pixels. (12)

[0334] According to the solid-state camera device described in (3), wherein,

[0335] In the horizontal / vertical addition, the pixel signals of two predetermined pixels in the horizontal direction and the pixel signals of two pixels in the vertical direction are added together, and

[0336] In the horizontal / vertical addition, the driving of multiple selection control lines is controlled to select the pixel signal of the phase difference pixel included in a specific pixel unit, and the pixel signals of ordinary pixels included in other pixel units are not added. (13)

[0338] According to the solid-state camera device described in (3) or (12), wherein,

[0339] In the horizontal / vertical addition of pixel signals of two predetermined pixels in the horizontal direction and pixel signals of two pixels in the vertical direction, the driving of multiple selection control lines is controlled so that the pixel signals of pixels included in a pixel unit including pixels with a specific phase difference and the pixel signals of ordinary pixels included in a pixel unit diagonally opposite the pixel unit are not added. (14)

[0341] According to the solid-state camera device described in (3), (12) or (13), wherein,

[0342] In the horizontal / vertical addition of the pixel signals of two predetermined pixels in the horizontal direction and the pixel signals of two pixels in the vertical direction, the driving of multiple selection control lines is controlled so that the pixel signals of pixels including only ordinary pixel units are added. (15)

[0344] According to the solid-state camera device described in (3), (12) or (13), wherein,

[0345] A specific floating diffusion zone is connected to a common vertical signal line, and

[0346] In the horizontal / vertical addition, the driving of multiple selection control lines is controlled to read pixel signals from the specific floating diffusion region. (16)

[0348] According to the solid-state camera device described in (15), wherein,

[0349] Specific floating diffusion zones connected to a common vertical signal line are formed in different columns in the vertical direction, and

[0350] Each pixel unit sharing a specific floating diffusion region is arranged in the same row in the horizontal direction and includes pixels of the same color. (17)

[0352] A solid-state camera device, comprising:

[0353] A pixel array unit, wherein multiple pixels are arranged in a two-dimensional manner, wherein...

[0354] The plurality of pixels includes phase difference pixels for phase difference detection.

[0355] The pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are arranged regularly, and

[0356] Multiple selection control lines are set in the horizontal direction to select the connection from the floating diffusion area formed for each of the plurality of pixel units in the vertical direction to the vertical signal line. (18)

[0358] According to the solid-state camera device described in (17), wherein,

[0359] For each pixel included in the pixel unit, the exposure time is adjusted, and

[0360] When reading the plurality of pixels, an HDR signal is generated by combining pixel signals with the same color but different exposure times obtained by horizontally / vertically adding the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction. (19)

[0362] According to the solid-state camera device described in (17), wherein,

[0363] In the pixel unit, all pixels of the same color have the same exposure time, and

[0364] When reading the plurality of pixels, after adding the pixel signals of pixels of the same color included in the pixel unit, the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are added horizontally / vertically. (20)

[0366] An electronic device is equipped with a solid-state camera, the solid-state camera comprising:

[0367] A pixel array unit, wherein multiple pixels are arranged in a two-dimensional manner, wherein...

[0368] The plurality of pixels includes phase difference pixels for phase difference detection.

[0369] The pixel array unit has an array pattern in which pixel units including adjacent pixels of the same color are arranged regularly, and

[0370] When reading the plurality of pixels, when the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are added horizontally / vertically, the phase difference pixels are partially not added.

[0371] List of reference numerals

[0372] 10 Solid-state camera devices

[0373] 11-pixel array unit

[0374] 12 Vertical drive circuit

[0375] 111 Floating Diffusion Region (FD)

[0376] 121 pixel drive line

[0377] 122, 122-1, 122-2, 122-3 Select control lines

[0378] 131 Vertical Signal Line (VSL)

[0379] 100 pixels

[0380] 200, 200A, 200B pixel units

[0381] 1000 electronic devices

[0382] 1012 camera units

[0383] 1014 Control Unit

[0384] 1020 drive unit

[0385] 12031 Camera Unit

[0386] 11402 Camera Unit.

Claims

1. A solid-state camera device, comprising: A pixel array unit, wherein multiple pixels are arranged in a two-dimensional manner, wherein... The plurality of pixels includes multiple sets of phase difference pixels for phase difference detection, each set of phase difference pixels including its own first phase difference pixel and its own second phase difference pixel that are adjacent to each other. The pixel array unit has an array pattern in which multiple pixel units are regularly arranged, and each pixel unit includes adjacent pixels of the same color. Specifically, when reading the plurality of pixels, in the first mode of horizontal and vertical addition, the pixel signals of the phase difference pixels of the first subgroup are added together, and in the second mode of horizontal and vertical addition, the phase difference pixels of the second subgroup are discarded.

2. The solid-state camera device according to claim 1, wherein, The density of the phase difference pixels after the horizontal and vertical addition is the same as, almost the same as, or lower than the density of the phase difference pixels before the horizontal and vertical addition.

3. The solid-state camera device according to claim 1, wherein, Multiple selection control lines are provided in the horizontal direction. These lines are used to select the connection from the floating diffusion region formed for each of the multiple pixel units in the vertical direction to the vertical signal line. When the horizontal and vertical additions are performed, the drive of the multiple selection control lines is controlled.

4. The solid-state camera device according to any one of claims 1 to 3, wherein, For each pixel included in the pixel unit, the exposure time is adjusted, and When reading the plurality of pixels, a high dynamic range signal is generated by synthesizing pixel signals with the same color but different exposure times obtained by adding the horizontal and vertical signals.

5. The solid-state camera device according to claim 4, wherein, Among the plurality of pixels, the respective first phase difference pixel and the respective second phase difference pixel are adjacent pixels with the same color included in adjacent different pixel units.

6. The solid-state camera device according to claim 5, wherein, The pixel unit comprises four pixels in a 2×2 grid. The respective first phase difference pixels and the respective second phase difference pixels are adjusted to have the same exposure time.

7. The solid-state camera device according to claim 4, wherein, The phase difference pixels are configured as light-shielding pixels.

8. The solid-state camera device according to any one of claims 1 to 3, wherein, In the pixel unit, all pixels of the same color have the same exposure time, and When reading the plurality of pixels, the horizontal and vertical addition is performed after the pixel signals of pixels of the same color included in the pixel unit are added together.

9. The solid-state camera device according to claim 8, wherein, Among the plurality of pixels, the respective first phase difference pixel and the respective second phase difference pixel are adjacent pixels with the same color included in adjacent different pixel units.

10. The solid-state camera device according to claim 9, wherein, The pixel unit comprises four pixels in a 2×2 pattern.

11. The solid-state camera device according to claim 8, wherein, The phase difference pixels are configured as light-shielding pixels.

12. The solid-state camera device according to claim 3, wherein, In the horizontal and vertical addition, the pixel signals of two predetermined pixels in the horizontal direction and the pixel signals of two pixels in the vertical direction are added together, and In the horizontal and vertical addition of the first mode, the driving of multiple selection control lines is controlled so that the pixel signals of phase difference pixels included in a specific pixel unit are selected, and the pixel signals of ordinary pixels included in pixel units other than the specific pixel unit are not added.

13. The solid-state camera device according to claim 12, wherein, In the second mode of horizontal and vertical addition, the driving of multiple selection control lines is controlled so that the pixel signals of pixels included in a pixel unit including a pixel with a specific phase difference and the pixel signals of ordinary pixels included in a pixel unit diagonally opposite to the pixel unit including the pixel with the specific phase difference are not added.

14. The solid-state camera device according to claim 13, wherein, In the third mode of horizontal and vertical addition, the driving of multiple selection control lines is controlled so that the pixel signals of pixels that only include ordinary pixel units are added together.

15. The solid-state camera device according to claim 3, wherein, A specific floating diffusion zone is connected to a common vertical signal line, and In the horizontal and vertical addition, the driving of multiple selection control lines is controlled to read pixel signals from the specific floating diffusion region.

16. The solid-state camera device according to claim 15, wherein, Specific floating diffusion regions connected to the common vertical signal line are formed in different columns in the vertical direction, and Each pixel unit sharing the specific floating diffusion region is arranged in the same row in the horizontal direction and is configured to include pixels of the same color.

17. A solid-state camera device, comprising: A pixel array unit, wherein multiple pixels are arranged in a two-dimensional manner, wherein... The plurality of pixels includes multiple sets of phase difference pixels for phase difference detection, each set of phase difference pixels including its own first phase difference pixel and its own second phase difference pixel that are adjacent to each other. The pixel array unit has an array pattern in which multiple pixel units are arranged regularly, and each pixel unit includes adjacent pixels of the same color. Multiple selection control lines are provided in the horizontal direction. These lines are used to select the connection from the floating diffusion region formed for each group of pixel units in the vertical direction to the vertical signal line. Specifically, the driving of the multiple selection control lines is controlled to enable at least a first mode of horizontal and vertical addition and different second modes of horizontal and vertical addition when reading the multiple pixels.

18. The solid-state camera device according to claim 17, wherein, For each pixel included in the pixel unit, the exposure time is adjusted, and When reading the plurality of pixels, an HDR signal is generated by combining pixel signals with the same color but different exposure times obtained by horizontally and vertically adding the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction.

19. The solid-state camera device according to claim 17, wherein, In the pixel unit, all pixels of the same color have the same exposure time, and When reading the plurality of pixels, after adding the pixel signals of pixels with the same color included in the pixel unit, the pixel signals of predetermined pixels in the horizontal direction and the pixel signals of predetermined pixels in the vertical direction are added horizontally and vertically.

20. An electronic device having a solid-state camera device according to any one of claims 1 to 16.

Images