Imaging element, imaging device, control method for imaging element, and control program for imaging element

Abstract

The present invention provides an imaging element, an imaging device, a control method for an imaging element, and a control program for an imaging element.　An imaging element (5) comprises a plurality of pixels (61), wherein: the plurality of pixels (61) include phase-contrast pixels P1 and normal pixels P2 differing from the phase-contrast pixels P1; the normal pixels P2 include normal pixels (61R), normal pixels (61G), and normal pixels (61B) which are a plurality of types corresponding to a plurality of wavelength bands; and the phase-contrast pixels P1 include phase-contrast pixels (61FAg) and phase-contrast pixels (61FBg), which correspond to one of the plurality of wavelength bands, and phase-contrast pixels (61FAw) and phase-contrast pixels (61FBw), which differ from the phase-contrast pixels (61FAg) and the phase-contrast pixels (61FBg) and which correspond to a plurality of the plurality of wavelength bands.

Description

Image pickup element, image pickup device, image pickup element control method, and image pickup element control program

[0001] The technology of the present disclosure relates to an imaging element, an imaging apparatus, a control method for an imaging element, and a control program for an imaging element.

[0002] Patent Document 1 describes an imaging element having a configuration in which no color filter is provided in the phase difference pixels.

[0003] Patent Document 2 describes an imaging element in which a transparent filter is provided in a phase difference pixel.

[0004] Patent Document 3 describes an imaging element including phase-detection pixels and image-generating pixels. The image-generating pixels in this imaging element are composed of red, blue, and green pixels, and the phase-detection pixels are covered with a white filter or a transparent layer that transmits light in the visible light range.

[0005] Patent Document 4 describes an imaging element having a configuration in which no color filter is provided in the phase difference pixels.

[0006] International Publication No. 2013 / 031537 Japanese Patent Application Publication No. 2016-57546 Japanese Patent Application Publication No. 2012-134581 Japanese Patent Application Publication No. 2022-103180

[0007] An image sensor, an image sensor apparatus, a method for controlling an image sensor, and a control program for an image sensor according to one embodiment of the technology disclosed herein are shown below. Note that, although the parentheses indicate corresponding components in the following embodiments, the present invention is not limited to these.

[0008] (1) An imaging element (imaging element 5) includes a plurality of pixels (pixels 61), wherein the plurality of pixels include first pixels (phase difference pixels P1) for detecting a phase difference and second pixels (normal pixels P2) different from the first pixels, wherein the second pixels include a plurality of types of pixels (normal pixels 61R, 61G, and 61B) corresponding to a plurality of wavelength bands, and wherein the first pixels include first-type pixels (phase difference pixels 61FAg and 61FBg, or phase difference pixels 61FAc and 61FBc) corresponding to at least one of the plurality of wavelength bands, and second-type pixels (phase difference pixels 61FAw and 61FBw, or phase difference pixels 61FAy and 61FBy) different from the first-type pixels and corresponding to a plurality of the plurality of wavelength bands.

[0009] (2) The image sensor according to (1), wherein the first-type pixels correspond to any one of the plurality of wavelength bands, and the second-type pixels correspond to the plurality of wavelength bands.

[0010] (3) The imaging element according to (2), wherein the plurality of wavelength bands include a first wavelength band, a second wavelength band, and a third wavelength band.

[0011] (4) The imaging element according to (3), wherein the first wavelength band is a red wavelength band, the second wavelength band is a green wavelength band, the third wavelength band is a blue wavelength band, and the first type pixels correspond to the second wavelength band.

[0012] (5) The imaging element according to (1), wherein the first type pixels correspond to a plurality of the plurality of wavelength bands.

[0013] (6) The imaging element according to (5), wherein the plurality of wavelength bands include a first wavelength band, a second wavelength band, and a third wavelength band, the first type of pixels correspond to the second wavelength band and the third wavelength band, and the second type of pixels correspond to the first wavelength band and the second wavelength band.

[0014] (7) The imaging element according to (6), wherein the second pixels are arranged in a predetermined pattern, and the first type pixels (phase difference pixels 61FAc and phase difference pixels 61FBc) are provided at some of the positions of the second pixels (normal pixels 61G) corresponding to the second wavelength band based on the pattern and at some of the positions of the second pixels (normal pixels 61B) corresponding to the third wavelength band based on the pattern.

[0015] (8) The imaging element according to (7), wherein the second type pixels (phase difference pixels 61FAy and phase difference pixels 61FBy) are provided at a part of the arrangement positions of the second pixels (normal pixels 61R) corresponding to the first wavelength band based on the pattern and at a part of the arrangement positions of the second pixels (normal pixels 61G) corresponding to the second wavelength band based on the pattern.

[0016] (9) An imaging element according to any one of (1) to (8), comprising an imaging surface (imaging surface 60) in which pixel groups each consisting of a plurality of the pixels aligned in a first direction are arranged in a second direction intersecting the first direction, and the first type pixels and the second type pixels are included in different pixel groups.

[0017] (10) The imaging element according to (9), wherein the first pixel is capable of detecting a phase difference in the first direction.

[0018] (11) The imaging element according to any one of (2) to (4), wherein the second pixels are arranged in a predetermined pattern, the first pixels are provided at a portion of the arrangement positions of the second pixels based on the pattern, and filters (color filters CFa and CFb) that transmit the corresponding wavelength bands of the second pixels that correspond to the arrangement positions of the second type pixels based on the pattern are provided at the periphery of the second type pixels (phase difference pixels 61FAw and phase difference pixels 61FBw).

[0019] (12) The imaging element according to (11), comprising an imaging surface (imaging surface 60) in which pixel groups each consisting of a plurality of the pixels aligned in a first direction are aligned in a second direction intersecting the first direction, and the width of the filter in the first direction at the end of the second type pixel in the first direction and the width of the filter in the second direction at the end of the second type pixel in the second direction differ depending on the position of the second type pixel on the imaging surface.

[0020] (13) The imaging element according to any one of (5) to (9), wherein the second pixels are arranged in a predetermined pattern, the first pixels are provided at a portion of the arrangement positions of the second pixels based on the pattern, and filters (color filters CFa and CFb) that transmit the corresponding wavelength bands of the second pixels that correspond to the arrangement positions of the first pixels based on the pattern are provided around the periphery of the first pixels (phase difference pixels 61FAc, 61FBc, 61FAy, and 61FBy).

[0021] (14) An imaging element according to (13), comprising an imaging surface in which pixel groups each consisting of a plurality of the pixels arranged in a first direction are arranged in a second direction intersecting the first direction, and the width of the filter in the first direction at the end of the first pixel in the first direction and the width of the filter in the second direction at the end of the first pixel in the second direction differ depending on the position of the first pixel on the imaging surface.

[0022] (15) The imaging device according to any one of (1) to (14), wherein the plurality of wavelength bands are included in a wavelength band of visible light.

[0023] (16) An imaging apparatus including: the imaging element according to any one of (1) to (15); and a processor (system control unit 11) that performs focus detection processing based on the pixel value of the first pixel.

[0024] (17) The imaging device according to (16), wherein the processor performs control to make the exposure times of the first type pixels and the second type pixels different from each other.

[0025] (18) A control method for an image sensor, wherein the image sensor includes a plurality of pixels, the plurality of pixels including a first pixel for detecting a phase difference and a second pixel different from the first pixel, the second pixel including a plurality of types of pixels corresponding to a plurality of wavelength bands, the first pixel including a first type pixel corresponding to at least one of the plurality of wavelength bands and a second type pixel different from the first type pixel and corresponding to a plurality of the plurality of wavelength bands, the control method including a step of performing control to make the exposure times of the first type pixels and the second type pixels different.

[0026] (19) A control program for an image sensor, wherein the image sensor includes a plurality of pixels, the plurality of pixels including a first pixel for phase difference detection and a second pixel different from the first pixel, the second pixel including a plurality of types of pixels corresponding to a plurality of wavelength bands, the first pixel including a first type pixel corresponding to at least one of the plurality of wavelength bands and a second type pixel different from the first type pixel and corresponding to a plurality of the plurality of wavelength bands, the control program causing a processor to execute a control step of differentiating exposure times between the first type pixels and the second type pixels.

[0027] FIG. 1 is a diagram showing a schematic configuration of a digital camera 100 that is an embodiment of an imaging device relating to the technique of the present disclosure. FIG. 2 is a plan view schematic diagram showing a schematic configuration of the image sensor 5 shown in FIG. 1. FIG. 3 is a partially enlarged schematic diagram showing an imaging surface 60 of the image sensor 5 shown in FIG. 2. FIG. 4 is a schematic diagram showing an example of a cross section of a range A1 shown in FIG. 3. FIG. 5 is a schematic diagram showing an example of a cross section of a range A2 shown in FIG. 4. FIG. 6 is a schematic diagram showing an example of a cross section of a range A3 shown in FIG. 4. FIG. 7 is a schematic diagram showing a modified example of the cross-sectional structure of a phase difference detection pair. FIG. 8 is a schematic diagram showing a first modified example of the image sensor 5 and corresponds to FIG. 3. FIG. 9 is a diagram showing a second modified example of the image sensor 5 and corresponds to FIG. 3. FIG. 10 is a diagram showing a third modified example of the image sensor 5 and corresponds to FIG. 3. FIG. 11 is a schematic diagram showing an example of a cross section of a range A4 shown in FIG. 10. FIG. 12 is a schematic diagram showing an example of a cross section of a range A5 shown in FIG. 10. FIG. 13 is a schematic diagram illustrating a fourth modified example of the image sensor 5. FIG. 14 is a diagram illustrating a preferred example of color filters CFa and CFb shown in FIG. 13. FIG. 15 is a diagram illustrating a preferred example of color filters CFa and CFb shown in FIG. 13. FIG. 16 is a diagram illustrating a preferred example of color filters CFa and CFb shown in FIG. 13. FIG. 17 is a diagram illustrating a preferred example of color filters CFa and CFb shown in FIG. 13. FIG. 18 is a diagram illustrating a configuration example of color filters mounted on the phase difference pixels 61FAy and 61FBy of the image sensor 5 shown in FIG. 10. FIG. 19 is a diagram illustrating a configuration example of color filters mounted on the phase difference pixels 61FAc and 61FBc of the image sensor 5 shown in FIG. 10. FIG. 20 shows the external appearance of a smartphone 200. FIG. 21 is a block diagram illustrating the configuration of the smartphone 200 shown in FIG. 20.

[0028] Fig. 1 is a diagram showing a schematic configuration of a digital camera 100, which is an embodiment of an imaging device relating to the technique of the present disclosure. The digital camera 100 shown in Fig. 1 includes a lens device 40 having an imaging lens 1, an aperture 2, a lens driver 8 that drives the imaging lens 1, an aperture driver 9 that drives the aperture 2, and a lens controller 4 that controls the lens driver 8 and the aperture driver 9, and a main body 100A.

[0029] The main body 100A includes an image sensor 5, a system control unit 11 that controls the overall electrical control system of the digital camera 100, an operation unit 14, a display device 22, a memory 16 that includes RAM (Random Access Memory) and ROM (Read Only Memory), a memory control unit 15 that controls the storage of data in the memory 16 and the reading of data from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 that controls the storage of data in a storage medium 21 and the reading of data from the storage medium 21.

[0030] The lens device 40 may be detachable from the main body 100 A, or may be integrated with the main body 100 A. The imaging lens 1 includes a focus lens.

[0031] The focus lens is a lens for adjusting the focus of an imaging optical system including the imaging lens 1 and the aperture 2, and is composed of a single lens or multiple lenses. When the focus lens moves in the optical axis direction, the position of the principal point of the focus lens (hereinafter also referred to as the focus lens position) changes along the optical axis, thereby changing the focal position on the subject side. Note that a liquid lens whose principal point position in the optical axis direction can be changed by electrical control may also be used as the focus lens.

[0032] The lens control unit 4 of the lens device 40 controls the lens drive unit 8 based on the lens drive signal transmitted from the system control unit 11 to change the focus lens position. The lens control unit 4 of the lens device 40 controls the diaphragm drive unit 9 based on the drive control signal transmitted from the system control unit 11 to change the opening amount of the diaphragm 2.

[0033] The image sensor 5 captures an image of the subject through the imaging optical system provided between the image sensor 5 and the subject. The image sensor 5 has an imaging surface 60 (see FIG. 2 ) on which a plurality of pixels are arranged two-dimensionally, and converts an image of the subject formed on the imaging surface 60 by the imaging optical system into an image signal using the plurality of pixels and outputs the image signal. The output of the pixels included in the image sensor 5 is referred to as a pixel signal or a pixel value, and a set of pixel signals or pixel values ​​is referred to as an image signal.

[0034] For example, a complementary metal-oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor is used as the imaging element 5. In the following, an example in which the imaging element 5 is a CMOS image sensor will be described.

[0035] The system control unit 11 controls the entire digital camera 100, and its hardware structure consists of various processors that execute programs and perform processing. The programs executed by the system control unit 11 (including a control program for the image sensor 5) are stored in ROM (a non-transitory storage medium) of the memory 16. The memory 16 and the system control unit 11 constitute a control device for the image sensor 5.

[0036] The various types of processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes programs to perform various processes, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), or a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically to perform specific processes. More specifically, the structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0037] The system control unit 11 may be configured with one of various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA).

[0038] The system control unit 11 drives the image sensor 5 and the lens device 40 to output, as an image signal, an image of a subject captured through the imaging optical system of the lens device 40. The image signal output from the image sensor 5 is processed by the digital signal processing unit 17 to generate captured image data that is data suitable for display on the display device 22 or data suitable for storage in the storage medium 21.

[0039] An instruction signal from the user is input to the system control unit 11 through the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b, various buttons, and the like.

[0040] The display device 22 includes a display surface 22b configured by an organic EL (electroluminescence) panel, a liquid crystal panel, or the like, and a display controller 22a that controls the display on the display surface 22b.

[0041] The memory control unit 15 , digital signal processing unit 17 , external memory control unit 20 , and display controller 22 a are interconnected by a control bus 24 and a data bus 25 , and are controlled by commands from the system control unit 11 .

[0042] Fig. 2 is a schematic plan view showing the general configuration of the image sensor 5 shown in Fig. 1. The image sensor 5 includes an image sensor surface 60 on which pixel groups 62, each composed of a plurality of pixels 61 arranged in a row direction X, are arranged in a column direction Y intersecting the row direction X, a drive circuit 63 that drives the pixels 61 arranged on the image sensor surface 60, and a signal processing circuit 64 that processes pixel signals read out to signal lines from each pixel 61 of the pixel groups 62 arranged on the image sensor surface 60.

[0043] In the example of Fig. 2, the row direction X and the column direction Y are orthogonal to each other. One of the row directions X is referred to as a right direction XR, and the other of the row directions X is referred to as a left direction XL. One of the column directions Y is referred to as an upward direction YU, and the other of the column directions Y is referred to as a downward direction YD.

[0044] It can also be said that on the imaging surface 60, a plurality of pixel groups each consisting of a plurality of pixels 61 aligned in a column direction Y are arranged in a row direction X. One of the row direction X and the column direction Y constitutes a first direction, and the other of the row direction X and the column direction Y constitutes a second direction.

[0045] 3 is a schematic diagram showing a partially enlarged image pickup surface 60 of the image pickup element 5 shown in FIG. 2. The plurality of pixels 61 arranged on the image pickup surface 60 include phase difference pixels P1 for detecting a phase difference (blocks marked with the letters "FA" or "FB" in the figure) and normal pixels P2 that are different from the phase difference pixels P1. The normal pixels P2 have optical characteristics (changes in output relative to the angle of incidence of light) that are different from those of the phase difference pixels P1. The phase difference pixels P1 constitute first pixels, and the normal pixels P2 constitute second pixels.

[0046] The normal pixels P2 include a plurality of types of pixels corresponding to a plurality of wavelength bands, which are included in the wavelength band of visible light (for example, a wavelength band of 350 nm or more and 800 nm or less).

[0047] The multiple wavelength bands include, for example, a red wavelength band (e.g., a wavelength band of 600 nm or more and 800 nm or less, hereinafter also referred to as R light), a green wavelength band (e.g., a wavelength band of 500 nm or more and less than 600 nm, hereinafter also referred to as G light), and a blue wavelength band (e.g., a wavelength band of 400 nm or more and less than 500 nm, hereinafter also referred to as B light). The R light constitutes the first wavelength band, the G light constitutes the second wavelength band, and the B light constitutes the third wavelength band.

[0048] 3, the plurality of pixels 61 include, as normal pixels P2, normal pixels 61R (blocks marked with the letter "R" in the figure) corresponding to R light, normal pixels 61G (blocks marked with the letter "G" in the figure) corresponding to G light, and normal pixels 61B (blocks marked with the letter "B" in the figure) corresponding to B light. Each of the normal pixels 61R, 61G, and 61B receives at least a portion of the light in the corresponding wavelength band, and outputs a pixel signal according to the amount of light.

[0049] The phase difference pixel P1 includes first-type pixels (phase difference pixels 61FAg and 61FBg labeled with the letters "FA(G)" in the figure) corresponding to G light among the plurality of wavelength bands (R light, G light, and B light), and second-type pixels (phase difference pixels 61FAw and 61FBw labeled with the letters "FA(W)" in the figure) corresponding to the plurality of wavelength bands (R light, G light, and B light) that differ from the first-type pixels. The first-type pixels and the second-type pixels have different spectral characteristics.

[0050] The phase difference pixel 61FAg and the phase difference pixel 61FBg each receive at least a portion of the corresponding G light and output a pixel signal according to the amount of the light.

[0051] The phase difference pixels 61FAw and 61FBw each receive at least a portion of the corresponding wavelength bands (R light, G light, and B light) and output a pixel signal corresponding to the amount of light. That is, the phase difference pixels 61FAw and 61FBw each receive at least a portion of R light, at least a portion of G light, and at least a portion of B light and output a pixel signal corresponding to the amount of light. The phase difference pixels 61FAw and 61FBw can receive light of more wavelengths than the phase difference pixels 61FAg and 61FBg, and therefore can increase sensitivity.

[0052] Normal pixels 61R, 61G, and 61B are arranged based on a predetermined pattern (Bayer pattern in the illustrated example) on the imaging surface 60. That is, on the imaging surface 60, a GR pixel group in which normal pixels 61G and normal pixels 61R are arranged alternately in the row direction X, and a BG pixel group in which normal pixels 61B and normal pixels 61G are arranged alternately in the row direction X are arranged alternately in the column direction Y. The arrangement pattern of the pixels 61 arranged on the imaging surface 60 is not limited to the Bayer pattern, and various other patterns can be adopted.

[0053] The phase difference pixels P1 are provided at some of the positions where the normal pixels P2 are arranged based on this Bayer pattern.

[0054] Specifically, in some BG pixel groups among the multiple pixel groups provided on the imaging surface 60, phase difference pixels 61FAg are arranged in some of the positions of normal pixels 61B based on the Bayer pattern, and phase difference pixels 61FBg are arranged in some of the positions of normal pixels 61G based on the Bayer pattern. The phase difference pixels 61FAg and 61FBg are adjacent to each other. In this part of the BG pixel group, multiple first-type phase difference detection pairs, each consisting of a phase difference pixel 61FAg and its adjacent phase difference pixel 61FBg, are arranged side by side with a gap in the row direction X. This part of the BG pixel group may be composed only of first-type phase difference detection pairs. On the imaging surface 60, multiple pixel groups G1 including this first-type phase difference detection pair are discretely arranged in the column direction Y.

[0055] Furthermore, in some BG pixel groups among the multiple pixel groups provided on the imaging surface 60 that do not include the above-described first-type phase difference detection pairs, phase difference pixels 61FAw are arranged in some of the positions of normal pixels 61B based on the Bayer pattern, and phase difference pixels 61FBw are arranged in some of the positions of normal pixels 61G based on the Bayer pattern. The phase difference pixels 61FAw and 61FBw are adjacent to each other. In these BG pixel groups, multiple second-type phase difference detection pairs, each consisting of a phase difference pixel 61FAw and its adjacent phase difference pixel 61FBw, are arranged side by side with a gap in the row direction X. These BG pixel groups may be composed only of second-type phase difference detection pairs. Multiple pixel groups G2 including these second-type phase difference detection pairs are arranged discretely in the column direction Y on the imaging surface 60. As shown in FIG. 3 , the pixel groups G1 and G2 are arranged alternately in the column direction Y.

[0056] Fig. 4 is a schematic diagram showing an example of a cross section of a range A1 shown in Fig. 3. Fig. 5 is a schematic diagram showing an example of a cross section of a range A2 shown in Fig. 4. Fig. 6 is a schematic diagram showing an example of a cross section of a range A3 shown in Fig. 4.

[0057] 4 to 6 , the normal pixel P2 provided on the imaging surface 60 includes a microlens ML that collects light from a subject, a photoelectric conversion unit PD that converts the light collected by the microlens ML into an electric charge, a color filter CF that transmits light in a specific wavelength band and is provided between the photoelectric conversion unit PD and the microlens ML, and a light-shielding film LS that is provided between the color filter CF and the photoelectric conversion unit PD. The photoelectric conversion unit PD is a photodiode formed in a semiconductor substrate such as silicon, but may also be composed of an organic material film or the like that is arranged above the semiconductor substrate.

[0058] The color filter CF (referred to as R filter in FIG. 4) included in normal pixel 61R transmits at least a portion of the R light, the color filter CF (referred to as G filter in FIGS. 4 to 6) included in normal pixel 61G transmits at least a portion of the G light, and the color filter CF (referred to as B filter in FIGS. 5 and 6) included in normal pixel 61B transmits at least a portion of the B light.

[0059] 5 , the phase difference pixel 61FAg and the phase difference pixel 61FBg each have a configuration in which the size of the opening of the light-shielding film LS is changed compared to the configuration of the normal pixel 61G. The opening of the light-shielding film LS of the phase difference pixel 61FAg is configured such that, for example, the left half of the opening of the light-shielding film LS of the normal pixel 61G is closed. The opening of the light-shielding film LS of the phase difference pixel 61FBg is configured such that, for example, the right half of the opening of the light-shielding film LS of the normal pixel 61G is closed.

[0060] As shown in FIG. 6, the phase difference pixel 61FAw and the phase difference pixel 61FBw each have a configuration in which the size of the opening of the light-shielding film LS is changed and the color filter CF is changed to a luminance filter LF, compared to the configuration of the normal pixel P2.

[0061] The luminance filter LF has spectral characteristics correlated with the luminance component of light, and may be an ND (Neutral Density) filter, a transparent filter, a white filter, or a gray filter. A configuration in which no material is provided between the microlens ML and the light-shielding film LS to prevent light from passing through and light is directly incident on the photoelectric conversion unit PD can also be said to have a luminance filter LF. The luminance filter LF can transmit light with more wavelength components than the color filter CF. The luminance filter LF transmits at least a portion of R light, at least a portion of G light, and at least a portion of B light.

[0062] The opening of the light-shielding film LS of the phase difference pixel 61FAw is configured such that, for example, the left half of the opening of the light-shielding film LS of the normal pixel P2 is closed. The opening of the light-shielding film LS of the phase difference pixel 61FBw is configured such that, for example, the right half of the opening of the light-shielding film LS of the normal pixel P2 is closed.

[0063] With such a configuration of the light-shielding film LS, the phase difference pixel 61FAg and the phase difference pixel 61FAw each receive one of a pair of light beams that have passed through two different portions aligned in the row direction X of the pupil region of the imaging optical system of the lens device 40. Furthermore, the phase difference pixel 61FBg and the phase difference pixel 61FBw each receive the other of the pair of light beams. Furthermore, the normal pixel P2 receives both of the pair of light beams.

[0064] In the examples of FIGS. 4 to 6, the optical characteristics of the phase difference pixel P1 and the normal pixel P2 are changed depending on the shape of the light-shielding film LS, but this can also be achieved by the shape of the microlens ML.

[0065] 7 , the aperture sizes of the light-shielding films LS of all the pixels 61 may be the same, and instead, in the second-type phase difference detection pair, the microlens ML may be shared between the phase difference pixel 61FAw and the phase difference pixel 61FBw. Although not shown, in the first-type phase difference detection pair, the microlens ML may be shared between the phase difference pixel 61FAg and the phase difference pixel 61FBg.

[0066] The system control unit 11 performs focus detection processing based on the pixel values ​​of the phase difference pixels P1. Specifically, the system control unit 11 detects a first phase difference in the row direction X by performing a correlation calculation between a pixel signal group output from the phase difference pixels 61FAg and a pixel signal group output from the phase difference pixels 61FBg included in the same pixel group G1. The system control unit 11 also detects a second phase difference in the row direction X by performing a correlation calculation between a pixel signal group output from the phase difference pixels 61FAw and a pixel signal group output from the phase difference pixels 61FBw included in the same pixel group G2. The system control unit 11 performs focus detection processing to derive a focus lens position required to focus on the target subject based on at least one of the first phase difference and the second phase difference.

[0067] The second type phase difference detection pair can receive light in more wavelength bands than the first type phase difference detection pair, thereby increasing sensitivity. This increases the detection accuracy of the second phase difference derived based on the pixel values ​​of the second type phase difference detection pair. However, since the second type phase difference detection pair can receive light in multiple wavelength bands, there are subjects for which phase difference detection is difficult.

[0068] For example, it may be difficult to detect the second phase difference for an object that includes an RB stripe pattern in which red subject areas extending linearly in the column direction Y and blue subject areas extending linearly in the column direction Y are alternately arranged in the row direction X, a BG stripe pattern in which blue subject areas extending linearly in the column direction Y and green subject areas extending linearly in the column direction Y are alternately arranged in the row direction X, or an RG stripe pattern in which red subject areas extending linearly in the column direction Y and green subject areas extending linearly in the column direction Y are alternately arranged in the row direction X.

[0069] However, for a subject including a BG striped pattern or an RG striped pattern, the first phase difference can be detected with high accuracy by using the pixel values ​​of the first type phase difference detection pair. For example, if the second phase difference cannot be detected, or if the second phase difference is detected but its reliability is below a threshold, the system control unit 11 determines the focus lens position based on the first phase difference. In this way, the phase difference detection accuracy is improved, making it possible to perform focus control with high accuracy.

[0070] The system control unit 11 may perform control to set different exposure times for the pixel group G1 including the first type phase difference detection pairs and the pixel group G2 including the second type phase difference detection pairs. For example, the system control unit 11 controls the exposure time of the pixel group G1 to a first time, and controls the exposure time of the pixel group G2 to a second time that is shorter than the first time.

[0071] Since the second type phase difference pairs included in the pixel group G2 have high sensitivity, it is possible to prevent saturation of pixel values ​​by shortening the exposure time. On the other hand, for the first type phase difference pairs included in the pixel group G1, it is possible to improve the signal-to-noise ratio by lengthening the exposure time. These effects make it possible to further improve the detection accuracy of the phase difference.

[0072] Fig. 8 is a schematic diagram showing a first modified example of the image sensor 5, and corresponds to Fig. 3. The image sensor 5 of the modified example shown in Fig. 8 has a configuration in which, compared to the image sensor 5 shown in Fig. 3, some of the first type phase difference detection pairs arranged in the pixel group G1 are changed to second type phase difference detection pairs, and some of the second type phase difference detection pairs arranged in the pixel group G2 are changed to first type phase difference detection pairs. In the example of Fig. 8, the first type phase difference detection pairs and the second type phase difference detection pairs are arranged alternately in the row direction X in each of the pixel group G1 and the pixel group G2.

[0073] The accuracy of phase difference detection can also be improved with the configuration shown in Fig. 8. In the configuration shown in Fig. 8, the system control unit 11 can detect a phase difference based on pixel values ​​of a first type phase difference detection pair included in the pixel group G1, and can detect a phase difference based on pixel values ​​of a second type phase difference detection pair included in the pixel group G1.

[0074] In the configuration shown in Fig. 3, the arrangement intervals of the first type phase difference detection pairs in the row direction X and the arrangement intervals of the second type phase difference detection pairs in the row direction X can be narrowed compared to the configuration shown in Fig. 8. Therefore, the configuration shown in Fig. 3 makes it possible to further improve the phase difference detection accuracy. Furthermore, in the configuration shown in Fig. 3, the first type phase difference detection pairs and the second type phase difference detection pairs are included in different pixel groups, so that even when the exposure times are different between the first type phase difference detection pairs and the second type phase difference detection pairs, the control thereof is easy, and the manufacturing cost of the digital camera 100 can be reduced.

[0075] 3 and 8, the two pixels 61 constituting the first type phase difference detection pair correspond to G light, but this is not limiting. For example, the two pixels 61 constituting the first type phase difference detection pair may correspond to R light or B light.

[0076] Furthermore, the imaging surface 60 may be configured to include two or more of the first type phase difference detection pairs, one corresponding to R light, one corresponding to G light, and one corresponding to B light.

[0077] In the description up to this point, it has been assumed that a phase difference detection pair for detecting a phase difference in the row direction X is arranged on the imaging surface 60. The technology of the present disclosure is similarly applicable to an image sensor 5 in which a phase difference detection pair for detecting a phase difference in the column direction Y is arranged on the imaging surface 60. Furthermore, it is similarly applicable to an image sensor 5 in which a phase difference detection pair for detecting a phase difference in the row direction X and a phase difference detection pair for detecting a phase difference in the column direction Y are arranged on the imaging surface 60.

[0078] Fig. 9 is a diagram showing a second modified example of the image sensor 5, and corresponds to Fig. 3. The image sensor 5 shown in Fig. 9 has a configuration in which the position of the phase difference pixel 61FBg that constitutes the first type phase difference detection pair in the image sensor 5 shown in Fig. 8 is changed to be adjacent to the phase difference pixel 61FAg that constitutes the first type phase difference detection pair in the downward direction YD, and a normal pixel 61G is arranged in the position where the phase difference pixel 61FBg was originally located.

[0079] 9 is configured such that, in the image sensor 5 shown in FIG. 8 , the position of the phase difference pixel 61FBw that constitutes the second type phase difference detection pair is changed to be adjacent to the phase difference pixel 61FAw that constitutes the second type phase difference detection pair in the downward direction YD, and a normal pixel 61G is arranged in the position where the phase difference pixel 61FBw was originally located.

[0080] 9 , the openings of the light-shielding film LS of the phase difference pixels 61FAg and 61FAw are configured to close, for example, the upper half of the opening of the light-shielding film LS of the normal pixel 61G. The openings of the light-shielding film LS of the phase difference pixels 61FBg and 61FBw are configured to close, for example, the lower half of the opening of the light-shielding film LS of the normal pixel 61G.

[0081] In this way, the image sensor 5 shown in Figure 9 is configured such that first type phase difference detection pairs are arranged side by side in the column direction Y in a part of a pixel group made up of a plurality of pixels 61 arranged in the column direction Y, and second type phase difference detection pairs are arranged side by side in the column direction Y in a part of another pixel group that does not include this first type phase difference detection pair.

[0082] The system control unit 11 detects a third phase difference in the column direction Y by performing a correlation calculation between a pixel signal group output from the phase difference pixel 61FAg and a pixel signal group output from the phase difference pixel 61FBg included in the same pixel group. The system control unit 11 also detects a fourth phase difference in the column direction Y by performing a correlation calculation between a pixel signal group output from the phase difference pixel 61FAw and a pixel signal group output from the phase difference pixel 61FBw included in the same pixel group. The system control unit 11 performs focus detection processing to derive a focus lens position required to focus on a target subject based on at least one of the third phase difference and the fourth phase difference.

[0083] Fig. 10 is a diagram showing a third modified example of the image sensor 5, and corresponds to Fig. 3. Fig. 11 is a schematic diagram showing an example of a cross section of a range A4 shown in Fig. 10. Fig. 12 is a schematic diagram showing an example of a cross section of a range A5 shown in Fig. 10.

[0084] In the image sensor 5 shown in FIG. 10 , compared to the image sensor 5 shown in FIG. 3 , the phase difference pixels 61FAg that constitute the first type phase difference detection pair are changed to phase difference pixels 61FAc (the block marked with “FA(Cy)” in the figure), and the phase difference pixels 61FBg that constitute the first type phase difference detection pair are changed to phase difference pixels 61FBc (the block marked with “FB(Cy)” in the figure).

[0085] 10, compared to the image sensor 5 shown in FIG. 3, the phase difference pixels 61FAw constituting the second type phase difference detection pair are changed to phase difference pixels 61FAy (blocks marked with “FA(Y)” in the drawing), and the arrangement positions thereof are changed from the BG pixel group to the GR pixel group.

[0086] 10, compared to the image sensor 5 shown in FIG. 3, the phase difference pixel 61FBw constituting the second type phase difference detection pair is changed to a phase difference pixel 61FBy (the block marked with “FB(Y)” in the drawing), and the arrangement position thereof is changed from the BG pixel group to the GR pixel group.

[0087] The phase difference pixel 61FAc and the phase difference pixel 61FBc constituting the first type phase difference detection pair each correspond to multiple (specifically, G light and B light) of the multiple wavelength bands (R light, G light, and B light). The phase difference pixel 61FAc and the phase difference pixel 61FBc each receive at least a portion of the G light and at least a portion of the B light and output pixel signals according to the amount of light. The phase difference pixel 61FAc and the phase difference pixel 61FBc can receive light of more wavelengths than the phase difference pixel 61FAg and the phase difference pixel 61FBg in the image sensor 5 shown in FIG. 3, and therefore can increase sensitivity.

[0088] The phase difference pixel 61FAy and the phase difference pixel 61FBy constituting the second type phase difference detection pair each correspond to multiple (specifically, R light and G light) of the multiple wavelength bands (R light, G light, and B light). The phase difference pixel 61FAy and the phase difference pixel 61FBy each receive at least a portion of the R light and at least a portion of the G light and output a pixel signal according to the amount of the received light. The phase difference pixel 61FAy and the phase difference pixel 61FBy can receive light of more wavelengths than the phase difference pixel 61FAg and the phase difference pixel 61FBg in the image sensor 5 shown in FIG. 3, and therefore can increase sensitivity.

[0089] 10 , in some BG pixel groups among a plurality of pixel groups provided on the imaging surface 60, phase difference pixels 61FAc are arranged at some of the positions of normal pixels 61B based on the Bayer pattern, and phase difference pixels 61FBc are arranged at some of the positions of normal pixels 61G based on the Bayer pattern. The phase difference pixels 61FAc and 61FBc are adjacent to each other. In this portion of the BG pixel groups, multiple first-type phase difference detection pairs, each consisting of a phase difference pixel 61FAc and its adjacent phase difference pixel 61FBc, are arranged side by side at intervals in the row direction X. On the imaging surface 60, multiple pixel groups G3 including these first-type phase difference detection pairs are arranged discretely in the column direction Y.

[0090] Furthermore, in some GR pixel groups among the multiple pixel groups provided on the imaging surface 60, phase difference pixels 61FAy are arranged in some of the positions of normal pixels 61G based on the Bayer pattern, and phase difference pixels 61FBy are arranged in some of the positions of normal pixels 61R based on the Bayer pattern. The phase difference pixels 61FAy and 61FBy are adjacent to each other. In these GR pixel groups, multiple second-type phase difference detection pairs, each consisting of a phase difference pixel 61FAy and its adjacent phase difference pixel 61FBy, are arranged side by side at intervals in the row direction X. Multiple pixel groups G4, each including this second-type phase difference detection pair, are arranged discretely in the column direction Y on the imaging surface 60. As shown in FIG. 10 , the pixel groups G3 and G4 are arranged alternately in the column direction Y.

[0091] As shown in FIG. 11, each of the phase difference pixel 61FAy and the phase difference pixel 61FBy has a color filter CF (shown as a Y filter in the figure) between the microlens ML and the light-shielding film LS, which is capable of transmitting at least a portion of G light and at least a portion of R light.

[0092] As shown in FIG. 12, each of the phase difference pixel 61FAc and the phase difference pixel 61FBc has a color filter CF (shown as a Cy filter in the figure) between the microlens ML and the light-shielding film LS, which is capable of transmitting at least a portion of G light and at least a portion of B light.

[0093] In the digital camera 100 including the image sensor 5 configured as described above, the system control unit 11 performs focus detection processing based on the pixel values ​​of the phase difference pixels P1.

[0094] Specifically, the system control unit 11 detects a fifth phase difference in the row direction X by performing a correlation calculation between a pixel signal group output from the phase difference pixel 61FAc and a pixel signal group output from the phase difference pixel 61FBc included in the same pixel group G3. The system control unit 11 also detects a sixth phase difference in the row direction X by performing a correlation calculation between a pixel signal group output from the phase difference pixel 61FAy and a pixel signal group output from the phase difference pixel 61FBy included in the same pixel group G4. The system control unit 11 performs focus detection processing to derive a focus lens position required to focus on the target subject based on at least one of the fifth phase difference and the sixth phase difference.

[0095] 10 , the sensitivity of each of the first type phase difference detection pair and the second type phase difference detection pair can be increased. Therefore, the accuracy of phase difference detection can be improved. On the other hand, since each of the first type phase difference detection pair and the second type phase difference detection pair can receive light of multiple wavelength bands, there are subjects for which phase difference detection is difficult.

[0096] For example, for a subject with the above-mentioned BG striped pattern, the use of pixel values ​​of the second type phase difference detection pair allows for more accurate detection of the phase difference than when using pixel values ​​of the first type phase difference detection pair.Furthermore, for a subject with the above-mentioned RG striped pattern, the use of pixel values ​​of the first type phase difference detection pair allows for more accurate detection of the phase difference than when using pixel values ​​of the second type phase difference detection pair.

[0097] Note that such an effect of improving the phase difference detection accuracy can be similarly obtained even in a configuration in which the positions of the phase difference pixels 61FAy and 61FAc are reversed and the positions of the phase difference pixels 61FBy and 61FBc are reversed in Fig. 10. However, the configuration shown in Fig. 10 is advantageous in terms of color mixing.

[0098] 10 , a portion of the B light and G light traveling in the column direction Y that has passed through the color filter of the phase difference pixel 61FAc may be incident on the photoelectric conversion unit PD of the normal pixel 61G that is adjacent to the phase difference pixel 61FAc in the column direction Y, which may cause blue color mixing in the pixel signal of the normal pixel 61G. However, a normal pixel 61B is disposed adjacent to the normal pixel 61G in the column direction Y, and a portion of the B light that has passed through the color filter CF of the normal pixel 61B may also be incident on the photoelectric conversion unit PD of the normal pixel 61G, which may cause blue color mixing.

[0099] If a phase difference pixel 61FAy is provided at the position of the phase difference pixel 61FAc, part of the R light that has passed through the color filter CF of the phase difference pixel 61FAy is incident on the photoelectric conversion unit PD of the normal pixel 61G. Therefore, there is a possibility that color mixing of blue and red occurs in the normal pixel 61G. Thus, according to the configuration shown in FIG. 10 , color mixing that may occur in the normal pixel 61G adjacent to the phase difference pixel 61FAc in the column direction Y can be limited to one color, thereby improving the image quality.

[0100] 10 , a portion of the B light and G light traveling in the row direction X that has passed through the color filter of the phase difference pixel 61FAc may be incident on the photoelectric conversion unit PD of the normal pixel 61G that is adjacent to the phase difference pixel 61FAc in the column direction Y, which may result in blue color mixing in the pixel signal of the normal pixel 61G. However, a normal pixel 61B is disposed adjacent to the normal pixel 61G in the column direction Y, and a portion of the B light that has passed through the color filter CF of the normal pixel 61B may also be incident on the photoelectric conversion unit PD of the normal pixel 61G, which may result in blue color mixing.

[0101] If a phase difference pixel 61FAy is provided at the position of the phase difference pixel 61FAc, part of the R light that has passed through the color filter CF of the phase difference pixel 61FAy is incident on the photoelectric conversion unit PD of the normal pixel 61G. Therefore, there is a possibility that color mixing of blue and red occurs in the normal pixel 61G. Thus, according to the configuration shown in FIG. 10 , color mixing that may occur in the normal pixel 61G adjacent to the phase difference pixel 61FAc in the row direction X can be limited to one color, thereby improving the image quality.

[0102] Such an effect of suppressing color mixing is similarly obtained in the normal pixels 61G on both sides of the phase difference pixel 61FBy in the column direction Y and in the normal pixel 61G to the right of the phase difference pixel 61FBy in the row direction X. The effect is also similarly obtained in the normal pixel 61R to the left of the phase difference pixel 61FAy in the row direction X. The effect is also similarly obtained in the normal pixel 61B to the right of the phase difference pixel 61FBc in the row direction X.

[0103] 10, the system control unit 11 may perform control such that the exposure times of the pixel group G3 including the first type phase difference detection pairs are different from those of the pixel group G4 including the second type phase difference detection pairs. This control can further improve the accuracy of phase difference detection.

[0104] Fig. 13 is a schematic diagram illustrating a fourth modified example of the image sensor 5. The pixel array of the image sensor 5 of the fourth modified example is the same as that shown in Fig. 3, but the internal structures of the phase difference pixels 61FAw and the phase difference pixels 61FBw are different. Fig. 13 illustrates the layer between the light-shielding film LS and the microlens ML as viewed from the microlens ML side in six pixels 61 in the range A6 shown in Fig. 3 in the image sensor 5 of the fourth modified example.

[0105] 13 is the same as the arrangement position of the normal pixel 61B based on the Bayer pattern. A color filter CFa that transmits at least a part of B light, which is in the wavelength band corresponding to the normal pixel 61B, is provided on the periphery of the phase difference pixel 61FAw that is arranged at the position where the normal pixel 61B should be arranged based on the Bayer pattern.

[0106] The color filter CFa is provided in the same layer as the color filter CF. The color filter CFa is provided in a substantially C-shape around the luminance filter LF. The color filter CFa includes: a first portion CFau provided along an edge of the luminance filter LF on the upward YU side; a second portion CFad provided along an edge of the luminance filter LF on the downward YD side; and a third portion CFal provided along an edge of the luminance filter LF on the left XL side, connecting the edges of the first portion CFau and the second portion CFad together in the left XL direction.

[0107] 13 is the same as the arrangement position of the normal pixel 61G based on the Bayer pattern. A color filter CFb that transmits at least a part of G light, which is in the wavelength band corresponding to the normal pixel 61G, is provided on the periphery of the phase difference pixel 61FBw that is arranged at the position where the normal pixel 61G should be arranged based on the Bayer pattern.

[0108] The color filter CFb is provided in the same layer as the color filter CF. The color filter CFb is provided around the luminance filter LF in a generally inverted C-shape. The color filter CFb includes a first portion CFbu provided along an edge of the luminance filter LF on the upward YU side, a second portion CFbd provided along an edge of the luminance filter LF on the downward YD side, and a third portion CFbr provided along an edge of the luminance filter LF on the right direction XR side, connecting the edges of the first portion CFbu and the second portion CFbd in the right direction XR.

[0109] 13 , the R light and the G light among the R light, the G light, and the B light that have passed through the luminance filter LF of the phase detection pixel 61FAw can be attenuated by the color filter CFa. As a result, most of the light that enters the photoelectric conversion unit PD of the normal pixel P2 adjacent to the phase detection pixel 61FAw can be made blue, and it is possible to prevent color mixture of multiple colors from occurring in this normal pixel P2.

[0110] Furthermore, the R light and the B light among the R light, the G light, and the B light that have passed through the luminance filter LF of the phase difference pixel 61FBw can be attenuated by the color filter CFb. As a result, most of the light that enters the photoelectric conversion unit PD of the normal pixel P2 adjacent to the phase difference pixel 61FBw can be made green, and it is possible to prevent color mixture of multiple colors from occurring in this normal pixel P2.

[0111] In the phase difference pixel 61FAw, the width in the column direction Y of the first portion CFau of the color filter CFa, the width in the column direction Y of the second portion CFad of the color filter CFa, and the width in the row direction X of the third portion CFal of the color filter CFa may be configured to differ depending on the arrangement position of the phase difference pixel 61FAw on the imaging surface 60.

[0112] Furthermore, in the phase difference pixel 61FBw, the width in the column direction Y of the first portion CFbu of the color filter CFb, the width in the column direction Y of the second portion CFbd of the color filter CFb, and the width in the row direction X of the third portion CFbr of the color filter CFb may be configured to differ depending on the arrangement position of the phase difference pixel 61FBw on the imaging surface 60.

[0113] For example, for the phase difference pixel 61FAw and the phase difference pixel 61FBw located at the end on the upward YU side of the imaging surface 60, as shown in FIG. 14, the width in the column direction Y of the first portion CFau and the width in the column direction Y of the first portion CFbu are made larger than those of the phase difference pixel 61FAw and the phase difference pixel 61FBw located at the center of the imaging surface 60 in the column direction Y, respectively.

[0114] Furthermore, for the phase difference pixels 61FAw and 61FBw located at the end of the imaging surface 60 on the downward direction YD side, as shown in FIG. 15 , the width in the column direction Y of the second portion CFad and the width in the column direction Y of the second portion CFbd are made larger than those of the phase difference pixels 61FAw and 61FBw located at the center of the imaging surface 60 in the column direction Y.

[0115] Furthermore, for the phase difference pixel 61FAw and the phase difference pixel 61FBw located at the end on the left side in the direction XL on the imaging surface 60, the width of the third portion CFal in the row direction X is made larger than that of the phase difference pixel 61FAw located at the center in the row direction X on the imaging surface 60, as shown in FIG. 16 .

[0116] 17 , for the phase difference pixel 61FAw and the phase difference pixel 61FBw located at the end on the right side in the XR direction on the imaging surface 60, the width of the third portion CFbr in the row direction X is made larger than that of the phase difference pixel 61FBw located at the center of the imaging surface 60 in the row direction X.

[0117] In this way, by changing the widths of the color filters CFa and CFb depending on the arrangement positions of the phase difference pixels 61FAw and the phase difference pixels 61FBw, it is possible to prevent the occurrence of color mixing of multiple colors in the pixels 61 adjacent to these, even in the peripheral parts of the imaging surface 60 where the angle of incidence of light is sharp.

[0118] 13 to 17 can also be applied to the image sensor 5 having the configuration shown in Fig. 10. For example, as shown in Fig. 18, a color filter CFa that transmits at least a portion of G light is added to a phase difference pixel 61FAy in a manner that surrounds the color filter CF included in the phase difference pixel 61FAy. This makes it possible to prevent R light that has passed through the color filter CF of the phase difference pixel 61FAy from being incident on the photoelectric conversion unit PD of the adjacent normal pixel P2.

[0119] Furthermore, a color filter CFb that transmits at least a part of the R light is added to the phase difference pixel 61FBy so as to surround the color filter CF included in the phase difference pixel 61FBy. This makes it possible to prevent the G light that has passed through the color filter CF of the phase difference pixel 61FBy from being incident on the photoelectric conversion unit PD of the adjacent normal pixel P2.

[0120] 19 , a color filter CFa that transmits at least a part of the B light is added to the phase difference pixel 61FAc so as to surround the color filter CF included in the phase difference pixel 61FAc. This makes it possible to prevent the G light that has passed through the color filter CF of the phase difference pixel 61FAc from being incident on the photoelectric conversion unit PD of the adjacent normal pixel P2.

[0121] Furthermore, a color filter CFb that transmits at least a part of the G light is added to the phase difference pixel 61FBc so as to surround the color filter CF included in the phase difference pixel 61FBc. This makes it possible to prevent the B light that has passed through the color filter CF of the phase difference pixel 61FBc from being incident on the photoelectric conversion unit PD of the adjacent normal pixel P2.

[0122] Next, the configuration of a smartphone, which is another embodiment of an imaging device according to the technique of the present disclosure, will be described.

[0123] Fig. 20 shows the external appearance of a smartphone 200. The smartphone 200 shown in Fig. 20 has a flat housing 201, and is provided with a display input unit 204 on one surface of the housing 201, which is an integrated unit of a display panel 202 as a display unit and an operation panel 203 as an input unit.

[0124] Such a housing 201 also includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. Note that the configuration of the housing 201 is not limited to this, and for example, it is also possible to adopt a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure or a sliding mechanism.

[0125] FIG. 21 is a block diagram showing the configuration of the smartphone 200 shown in FIG.

[0126] As shown in FIG. 21 , the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a memory unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiving unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0127] The smartphone 200 also has, as its main function, a wireless communication function for performing mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0128] The wireless communication unit 210 performs wireless communication with a base station device BS accommodated in the mobile communication network NW in accordance with instructions from the main control unit 220. Using this wireless communication, various file data such as audio data and image data, e-mail data, etc. are sent and received, and web data, streaming data, etc. are received.

[0129] The display input unit 204 is a so-called touch panel that, under the control of the main control unit 220, displays images (still images and moving images) or text information, etc. to visually convey information to the user and detects user operations on the displayed information, and is equipped with a display panel 202 and an operation panel 203.

[0130] The display panel 202 uses a liquid crystal display (LCD), an organic electroluminescence display (OELD), or the like as a display device.

[0131] The operation panel 203 is a device placed so that an image displayed on the display surface of the display panel 202 can be seen, and detects one or more coordinates operated by a user's finger or a stylus. When this device is operated by the user's finger or a stylus, a detection signal generated by the operation is output to the main control unit 220. Next, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0132] As shown in Figure 21, the display panel 202 and operation panel 203 of a smartphone 200, which is an example of one embodiment of the imaging device of the present invention, are integrated to form a display input unit 204, and the operation panel 203 is positioned so that it completely covers the display panel 202.

[0133] When such an arrangement is adopted, operation panel 203 may also have a function for detecting user operations in areas outside display panel 202. In other words, operation panel 203 may have a detection area for the overlapping portion that overlaps display panel 202 (hereinafter referred to as a display area), and a detection area for the remaining outer edge portion that does not overlap display panel 202 (hereinafter referred to as a non-display area).

[0134] The size of the display area and the size of the display panel 202 may be exactly the same, but they do not necessarily have to be the same. The operation panel 203 may also have two sensitive areas: an outer edge portion and an inner portion. The width of the outer edge portion is designed appropriately depending on the size of the housing 201, etc.

[0135] Furthermore, the position detection method used in the operation panel 203 may be a matrix switch method, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, a capacitance method, or the like, and any method may be used.

[0136] The call unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes voice data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0137] Also, as shown in FIG. 20, for example, the speaker 205 can be mounted on the same surface as the display input unit 204, and the microphone 206 can be mounted on the side of the housing 201.

[0138] The operation unit 207 is a hardware key using a key switch or the like, and receives instructions from a user. For example, as shown in Fig. 20 , the operation unit 207 is a push-button switch mounted on the side surface of the housing 201 of the smartphone 200, which turns on when pressed with a finger or the like, and turns off when the finger is released due to the restoring force of a spring or the like.

[0139] The storage unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with names or telephone numbers of communication partners, data of sent and received e-mails, web data downloaded by web browsing, downloaded content data, and also temporarily stores streaming data, etc. The storage unit 212 is composed of an internal storage unit 217 built into the smartphone and an external storage unit 218 having a removable external memory slot.

[0140] The internal memory unit 217 and the external memory unit 218 constituting the memory unit 212 are realized using storage media such as a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., MicroSD (registered trademark) memory, etc.), a RAM (Random Access Memory), a ROM (Read Only Memory), etc.

[0141] The external input / output unit 213 serves as an interface with all external devices connected to the smartphone 200, and is used to directly or indirectly connect to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.) or a network (e.g., Ethernet (registered trademark), wireless LAN (Local Area Network), etc.).

[0142] Examples of external devices that can be connected to the smartphone 200 include wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card sockets, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video devices connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video devices, wired / wirelessly connected smartphones, wired / wirelessly connected personal computers, wired / wirelessly connected personal computers, earphones, etc.

[0143] The external input / output unit 213 can transmit data received from such external devices to each component inside the smartphone 200, or transmit data inside the smartphone 200 to external devices.

[0144] The GNSS receiving unit 214 receives GNSS signals transmitted from the GNSS satellites ST1 to STn in accordance with instructions from the main control unit 220, executes positioning calculation processing based on the received multiple GNSS signals, and detects a position consisting of the latitude, longitude, and altitude of the smartphone 200. When the GNSS receiving unit 214 can acquire position information from the wireless communication unit 210 or the external input / output unit 213 (for example, a wireless LAN), it can also detect the position using the position information.

[0145] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor, and detects the physical movement of the smartphone 200 in accordance with instructions from the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection result is output to the main control unit 220.

[0146] The power supply unit 216 supplies power stored in a battery (not shown) to each unit of the smartphone 200 in accordance with instructions from the main control unit 220 .

[0147] The main control unit 220 includes a microprocessor, operates according to the control program and control data stored in the storage unit 212, and controls all the units of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. The main control unit 220 also includes a mobile communication control function that controls all the units of the communication system to perform voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0148] The application processing function is realized by the main control unit 220 operating in accordance with the application software stored in the storage unit 212. Examples of the application processing function include an infrared communication function that controls the external input / output unit 213 to perform data communication with a connected device, an email function that sends and receives emails, and a web browsing function that views web pages.

[0149] The main control unit 220 also has an image processing function for displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0150] The image processing function refers to a function in which the main control unit 220 decodes the image data, performs image processing on the decoded result, and displays the image on the display input unit 204 .

[0151] Furthermore, the main control unit 220 executes display control for the display panel 202 and operation detection control for detecting user operations via the operation unit 207 and the operation panel 203 .

[0152] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for starting application software, or displays a window for creating an e-mail.

[0153] The scroll bar refers to a software key for receiving an instruction to move the displayed portion of an image, such as a large image that cannot fit in the display area of ​​the display panel 202 .

[0154] In addition, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the above icons and input of character strings into the input field of the above window through the operation panel 203, or accepts requests to scroll the displayed image through the scroll bar.

[0155] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is an overlapping portion (display area) that overlaps the display panel 202 or an outer edge portion (non-display area) that does not overlap the display panel 202, and is equipped with a touch panel control function that controls the sensitive area of ​​the operation panel 203 or the display position of the software key.

[0156] The main control unit 220 can also detect a gesture operation on the operation panel 203 and execute a preset function in response to the detected gesture operation.

[0157] Gesture operation is not a simple touch operation as in the past, but rather refers to an operation of drawing a path with a finger or the like, specifying multiple positions simultaneously, or combining these to draw a path for at least one of multiple positions.

[0158] The camera unit 208 includes the lens device 40, the image sensor 5, and the digital signal processing unit 17 shown in FIG.

[0159] The captured image data generated by the camera unit 208 can be stored in the storage unit 212 or output via the external input / output unit 213 or the wireless communication unit 210 .

[0160] In the smartphone 200 shown in FIG. 21 , the camera unit 208 is mounted on the same surface as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it may be mounted on the back surface of the display input unit 204.

[0161] The camera unit 208 can also be used for various functions of the smartphone 200. For example, an image acquired by the camera unit 208 can be displayed on the display panel 202, or an image from the camera unit 208 can be used as one of the operation inputs for the operation panel 203.

[0162] Furthermore, when the GNSS receiving unit 214 detects a position, it can also detect the position by referring to an image from the camera unit 208. Furthermore, it can also refer to an image from the camera unit 208 to determine the optical axis direction of the camera unit 208 of the smartphone 200 or determine the current usage environment without using a triaxial acceleration sensor or by using the image in combination with a triaxial acceleration sensor. Of course, the image from the camera unit 208 can also be used in application software.

[0163] In addition, image data of still images or videos can be added with location information acquired by the GNSS receiving unit 214, audio information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., and stored in the memory unit 212 or output via the external input / output unit 213 or wireless communication unit 210.

[0164] Although various embodiments have been described above, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above embodiments may be combined in any manner as long as they do not deviate from the spirit of the invention.

[0165] This application is based on a Japanese patent application (Patent Application No. 2023-169880) filed on September 29, 2023, the contents of which are incorporated herein by reference.

[0166] REFERENCE SIGNS LIST 1 Imaging lens A1, A2, A3, A4, A5, A6 Range 4 Lens control unit 5 Imaging element 8 Lens driving unit 9 Driving unit 11 System control unit 14, 207 Operation unit 15 Memory control unit 16 Memory 17 Digital signal processing unit 20 External memory control unit 21 Storage medium 22 Display device 22a Display controller 22b Display surface 24 Control bus 25 Data bus 40 Lens device 60 Imaging surface 61 Pixel 61B, 61G, 61R Normal pixel 61FAg, 61FBg, 61FAw, 61FBw Phase difference pixel 61FAc, 61FBc, 61FAy, 61FBy Phase difference pixel 62 Pixel group 63 Driving circuit 64 Signal processing circuit 100 Digital camera 100A Main body 200 Smartphone 201 Housing 202 Display panel 203 Operation panel 204 Display input unit 205 Speaker 206 Microphone 208 Camera unit 210 Wireless communication unit 211 Call unit 212 Memory unit 213 External input / output unit 214 GNSS receiving unit 215 Motion sensor unit 216 Power supply unit 217 Internal memory unit 218 External memory unit 220 Main control unit P1 Phase difference pixel P2 Normal pixel G1, G2, G3, G4 Pixel group

Claims

1. An imaging element comprising a plurality of pixels, the plurality of pixels including a first pixel for detecting a phase difference and a second pixel different from the first pixel, the second pixel including a plurality of types of pixels corresponding to a plurality of wavelength bands, the first pixels including a first type of pixel corresponding to at least one of the plurality of wavelength bands, and a second type of pixel different from the first type of pixel and corresponding to a plurality of the plurality of wavelength bands.

2. An image sensor according to claim 1, wherein the first type pixels correspond to any one of the plurality of wavelength bands, and the second type pixels correspond to the plurality of wavelength bands.

3. An imaging element according to claim 2, wherein the plurality of wavelength bands include a first wavelength band, a second wavelength band, and a third wavelength band.

4. An imaging element according to claim 3, wherein the first wavelength band is a red wavelength band, the second wavelength band is a green wavelength band, the third wavelength band is a blue wavelength band, and the first type pixel corresponds to the second wavelength band.

5. An imaging element according to claim 1, wherein the first type pixels correspond to multiple of the multiple wavelength bands.

6. An imaging element according to claim 5, wherein the plurality of wavelength bands include a first wavelength band, a second wavelength band, and a third wavelength band, the first type of pixels correspond to the second wavelength band and the third wavelength band, and the second type of pixels correspond to the first wavelength band and the second wavelength band.

7. An imaging element as described in claim 6, wherein the second pixels are arranged in a predetermined pattern, and the first type pixels are provided at some of the positions of the second pixels corresponding to the second wavelength band based on the pattern, and at some of the positions of the second pixels corresponding to the third wavelength band based on the pattern.

8. An imaging element as described in claim 7, wherein the second type pixels are provided at some of the positions of the second pixels corresponding to the first wavelength band based on the pattern, and at some of the positions of the second pixels corresponding to the second wavelength band based on the pattern.

9. An imaging element as described in claim 1, comprising an imaging surface in which pixel groups each composed of a plurality of the pixels aligned in a first direction are arranged in a second direction intersecting the first direction, and the first type pixels and the second type pixels are included in different pixel groups.

10. An imaging element according to claim 9, wherein the first pixel is capable of detecting a phase difference in the first direction.

11. An imaging element as described in claim 2, wherein the second pixels are arranged in a predetermined pattern, the first pixels are provided at a portion of the arrangement positions of the second pixels based on the pattern, and a filter that transmits the corresponding wavelength band of the second pixels that corresponds to the arrangement positions of the second type pixels based on the pattern is provided at the periphery of the second type pixels.

12. An imaging element as described in claim 11, comprising an imaging surface in which pixel groups each composed of a plurality of the pixels aligned in a first direction are arranged in a row in a second direction intersecting the first direction, and the width of the filter in the first direction at the end of the second type pixel in the first direction and the width of the filter in the second direction at the end of the second type pixel in the second direction differ depending on the position of the second type pixel on the imaging surface.

13. An imaging element as described in claim 5, wherein the second pixels are arranged in a predetermined pattern, the first pixels are provided at a portion of the arrangement positions of the second pixels based on the pattern, and a filter is provided at the periphery of the first pixels, which transmits the corresponding wavelength band of the second pixels that corresponds to the arrangement positions of the first pixels based on the pattern.

14. An imaging element as described in claim 13, comprising an imaging surface in which pixel groups each composed of a plurality of the pixels aligned in a first direction are arranged in a line in a second direction intersecting the first direction, and the width of the filter in the first direction at an end of the first pixel in the first direction and the width of the filter in the second direction at an end of the first pixel in the second direction differ depending on the position of the first pixel on the imaging surface.

15. An imaging element according to claim 1, wherein the plurality of wavelength bands are included in the wavelength band of visible light.

16. An imaging device comprising: an imaging element according to any one of claims 1 to 15; and a processor that performs focus detection processing based on the pixel value of the first pixel.

17. An imaging device according to claim 16, wherein the processor performs control to make the exposure times of the first type pixels and the second type pixels different.

18. A control method for an image sensor, wherein the image sensor comprises a plurality of pixels, the plurality of pixels including a first pixel for detecting a phase difference and a second pixel different from the first pixel, the second pixel including a plurality of types of pixels corresponding to a plurality of wavelength bands, the first pixels including a first type pixel corresponding to at least one of the plurality of wavelength bands, and a second type pixel different from the first type pixel and corresponding to a plurality of the plurality of wavelength bands, the control method comprising a step of performing control to make the exposure times of the first type pixels and the second type pixels different.

19. A control program for an image sensor, wherein the image sensor comprises a plurality of pixels, the plurality of pixels including a first pixel for detecting a phase difference and a second pixel different from the first pixel, the second pixel including a plurality of types of pixels corresponding to a plurality of wavelength bands, the first pixels including a first type pixel corresponding to at least one of the plurality of wavelength bands, and a second type pixel different from the first type pixel and corresponding to a plurality of the plurality of wavelength bands, the control program causing a processor to execute a control step of making the exposure times of the first type pixels and the second type pixels different.

Images