Imaging element, imaging device, method for controlling imaging element, and program for controlling imaging element

Abstract

The invention provides an imaging element, an imaging device, a method for controlling the imaging element, and a program for controlling the imaging element. An imaging element (5) is provided with a plurality of pixels (61), the plurality of pixels (61) including phase difference pixels (P1) and common pixels (P2) different from the phase difference pixels (P1), and the common pixels (P2) include a plurality of types of common pixels (61R), common pixels (61G), and common pixels (61B) corresponding to a plurality of wavelength bands. The phase difference pixels (P1) include a phase difference pixel (61FAg) and a phase difference pixel (61FBg) corresponding to one of the plurality of wavelength bands, and a phase difference pixel (61FAw) and a phase difference pixel (61FBw) different from the phase difference pixel (61FAg) and the phase difference pixel (61FBg) and corresponding to a plurality of the wavelength bands.

Description

Technical Field

[0001] The present invention relates to an imaging element, a camera device, a control method for the imaging element, and a control program for the imaging element. Background Technology

[0002] Patent document 1 describes an imaging element with a structure in which a color filter is not provided on the phase difference pixel.

[0003] Patent document 2 describes an imaging element with a structure in which a transparent filter is provided on the phase difference pixel.

[0004] Patent document 3 describes an imaging element having phase difference pixels and image generation pixels. The image generation pixels in this imaging element are composed of red pixels, blue pixels, and green pixels, and the phase difference pixels are covered by a white filter or transparent layer that allows light from the visible light region to pass through.

[0005] Patent document 4 describes an imaging element with a structure in which a color filter is not provided on the phase difference pixel.

[0006] Previous technical documents Patent documents Patent Document 1: International Publication No. 2013 / 031537 Patent Document 2: Japanese Patent Application Publication No. 2016-57546 Patent Document 3: Japanese Patent Application Publication No. 2012-134581 Patent Document 4: Japanese Patent Application Publication No. 2022-103180 Summary of the Invention

[0007] means for solving technical problems The imaging element, camera device, control method for the imaging element, and control program for the imaging element according to one embodiment of the present invention are shown below. Furthermore, the components corresponding to those in the embodiments described later are shown in parentheses, but the invention is not limited thereto. (1) An imaging element (imaging element 5) has a plurality of pixels (pixels 61). The aforementioned pixels include a first pixel (phase difference pixel P1) used for phase difference detection and a second pixel (ordinary pixel P2) that is different from the first pixel. The aforementioned second pixel includes multiple pixels corresponding to multiple wavelength bands (ordinary pixel 61R, ordinary pixel 61G, ordinary pixel 61B). The first pixel mentioned above includes a first type of pixel (phase difference pixel 61FAg and phase difference pixel 61FBg, or phase difference pixel 61FAc and phase difference pixel 61FBc) that corresponds to at least one of the plurality of wavelength bands, and a second type of pixel (phase difference pixel 61FAw and phase difference pixel 61FBw, or phase difference pixel 61FAy and phase difference pixel 61FBy) that is different from the first type of pixel and corresponds to the plurality of wavelength bands. (2) According to the imaging element described in (1), wherein, The first type of pixel mentioned above corresponds to any one of the aforementioned wavelength bands. The second type of pixel mentioned above corresponds to the multiple wavelength bands mentioned above. (3) According to the imaging element described in (2), wherein, The aforementioned wavelength bands include the first wavelength band, the second wavelength band, and the third wavelength band. (4) According to the imaging element described in (3), wherein, The first wavelength band mentioned above is the red wavelength band. The second wavelength band mentioned above is the green wavelength band. The third wavelength band mentioned above is the blue wavelength band. The first type of pixel mentioned above corresponds to the second wavelength band mentioned above. (5) According to the imaging element described in (1), wherein, The first type of pixel mentioned above corresponds to multiple wavelength bands mentioned above. (6) According to the imaging element described in (5), wherein, The aforementioned wavelength bands include the first wavelength band, the second wavelength band, and the third wavelength band. The first type of pixel mentioned above corresponds to the second and third wavelength bands mentioned above. The second type of pixel mentioned above corresponds to the first wavelength band and the second wavelength band mentioned above. (7) According to the imaging element described in (6), wherein, The second pixel mentioned above is arranged according to a predetermined pattern. The first type of pixel (phase difference pixel 61FAc and phase difference pixel 61FBc) is disposed in a portion of the configuration position of the second pixel (ordinary pixel 61G) corresponding to the second wavelength band based on the above pattern and a portion of the configuration position of the second pixel (ordinary pixel 61B) corresponding to the third wavelength band based on the above pattern. (8) According to the imaging element described in (7), wherein, The second type of pixel (phase difference pixel 61FAy and phase difference pixel 61FBy) is disposed in a portion of the configuration position of the second pixel (ordinary pixel 61R) corresponding to the first wavelength band based on the above pattern and a portion of the configuration position of the second pixel (ordinary pixel 61G) corresponding to the second wavelength band based on the above pattern. (9) The imaging element according to any one of (1) to (8), wherein, An imaging surface (imaging surface 60) is provided in a second direction intersecting the first direction, in which a plurality of pixel groups consisting of the plurality of pixels arranged in the first direction are arranged. The first type of pixel and the second type of pixel mentioned above are contained in different groups of pixels mentioned above. (10) According to the imaging element described in (9), wherein, The first pixel mentioned above is a pixel capable of detecting the phase difference in the first direction. (11) The imaging element according to any one of (2) to (4), wherein, The second pixel mentioned above is arranged according to a predetermined pattern. The first pixel is set at a portion of the configuration position of the second pixel based on the pattern described above. A filter (color filter CFa and color filter CFb) is provided at the periphery of the second type of pixel (phase difference pixel 61FAw and phase difference pixel 61FBw) to transmit the corresponding wavelength band of the second type of pixel according to the pattern and the arrangement position of the second type of pixel. (12) According to the imaging element described in (11), wherein, An imaging surface (imaging surface 60) is provided in a second direction intersecting the first direction, in which a plurality of pixel groups consisting of the plurality of pixels arranged in the first direction are arranged. The width of the filter in the first direction at the end of the second type of pixel in the first direction and the width of the filter in the second direction at the end of the second type of pixel in the second direction are different depending on the position of the imaging surface of the second type of pixel. (13) The imaging element according to any one of (5) to (9), wherein, The second pixel mentioned above is arranged according to a predetermined pattern. The first pixel is set at a portion of the configuration position of the second pixel based on the pattern described above. A filter (color filter CFa and color filter CFb) is provided at the periphery of the first pixel (phase difference pixel 61FAc, phase difference pixel 61FBc, phase difference pixel 61FAy and phase difference pixel 61FBy) that transmits the corresponding wavelength band of the second pixel according to the pattern and the arrangement position of the first pixel. (14) According to the imaging element described in (13), wherein, An imaging surface is provided in which a plurality of pixel groups, each consisting of a plurality of pixels arranged in the first direction, are arranged in a second direction intersecting the first direction. The width of the filter in the first direction at the end of the first pixel in the first direction and the width of the filter in the second direction at the end of the first pixel in the second direction are different depending on the position of the imaging surface of the first pixel. (15) The imaging element according to any one of (1) to (14), wherein, The aforementioned wavelength bands are included in the visible light wavelength band. (16) A camera device comprising: The imaging element as described in any one of (1) to (15); and The processor (system control unit 11) performs focus detection processing based on the pixel value of the first pixel. (17) According to the camera device described in (16), wherein, The processor described above controls the exposure time to be different for the first type of pixel and the second type of pixel described above. (18) A method for controlling an imaging element, wherein, The imaging element described above includes multiple pixels, including a first pixel for phase difference detection and a second pixel different from the first pixel. The second pixel includes multiple pixels corresponding to multiple wavelength bands. The first pixel includes a first type of pixel corresponding to at least one of the multiple wavelength bands and a second type of pixel different from the first type of pixel and corresponding to multiple of the multiple wavelength bands. The above control method includes the following steps: controlling the exposure time to be different for the first type of pixel and the second type of pixel. (19) A control program for an imaging element, wherein, The imaging element described above includes multiple pixels, including a first pixel for phase difference detection and a second pixel different from the first pixel. The second pixel includes multiple pixels corresponding to multiple wavelength bands. The first pixel includes a first type of pixel corresponding to at least one of the multiple wavelength bands and a second type of pixel different from the first type of pixel and corresponding to multiple of the multiple wavelength bands. The above control program causes the processor to perform the following steps: control the exposure time to be different for the first type of pixel and the second type of pixel. Attached Figure Description

[0027] Figure 1 This is a diagram showing a schematic structure of a digital camera 100, which is an embodiment of the imaging device according to the technology of the present invention.

[0028] Figure 2 It means Figure 1 A schematic plan view of the outline structure of the imaging element 5 shown.

[0029] Figure 3 It is a local magnification representation Figure 2 A schematic diagram of the imaging surface 60 of the imaging element 5 shown.

[0030] Figure 4 It means Figure 3 A schematic diagram of an example of a cross-section of range A1 shown.

[0031] Figure 5 It means Figure 4 A schematic diagram of an example of a cross-section of range A2 shown.

[0032] Figure 6 It means Figure 4 A schematic diagram of an example of a cross-section of range A3 shown.

[0033] Figure 7 This is a schematic diagram showing a modified example of the cross-sectional structure of a phase difference detection pair.

[0034] Figure 8 This is a schematic diagram showing a first modified example of imaging element 5, and is related to... Figure 3 The corresponding diagram.

[0035] Figure 9 This is a diagram showing the second modified example of imaging element 5, and it is related to... Figure 3 The corresponding diagram.

[0036] Figure 10 This is a diagram showing the third modified example of imaging element 5, and it is related to... Figure 3 The corresponding diagram.

[0037] Figure 11 It means Figure 10 A schematic diagram of an example of a cross-section of range A4 shown.

[0038] Figure 12 It means Figure 10 A schematic diagram of an example of a cross-section of range A5 shown.

[0039] Figure 13 This is a schematic diagram showing the fourth modified example of imaging element 5.

[0040] Figure 14 It means Figure 13 A diagram showing preferred examples of color filters CFa and CFb.

[0041] Figure 15 It means Figure 13 A diagram showing preferred examples of color filters CFa and CFb.

[0042] Figure 16 It means Figure 13 A diagram showing preferred examples of color filters CFa and CFb.

[0043] Figure 17 It means Figure 13 A diagram showing preferred examples of color filters CFa and CFb.

[0044] Figure 18 It means that it is carried on Figure 10 The diagram shows an example of the structure of the color filter for the phase difference pixel 61FAy and the phase difference pixel 61FBy of the imaging element 5.

[0045] Figure 19 It means that it is carried on Figure 10 The diagram shows an example of the structure of the color filter for the phase difference pixel 61FAc and phase difference pixel 61FBc of the imaging element 5.

[0046] Figure 20 This is a diagram showing the appearance of the smartphone 200.

[0047] Figure 21 It means Figure 20 The diagram shows the structure of the smartphone 200. Detailed Implementation

[0048] Figure 1 This is a diagram showing a schematic structure of a digital camera 100, which is an embodiment of the imaging device according to the technology of the present invention. Figure 1 The digital camera 100 shown includes: a lens assembly 40, which has an imaging lens 1, an aperture 2, a lens driving unit 8 for driving the imaging lens 1, an aperture driving unit 9 for driving the aperture 2, and a lens control unit 4 for controlling the lens driving unit 8 and the aperture driving unit 9; and a main body 100A.

[0049] The main body 100A includes: an imaging element 5; a system control unit 11, which centrally controls the entire electrical control system of the digital camera 100; an operation unit 14; a display device 22; a memory 16, including RAM (Random Access Memory) and ROM (Read Only Memory); a memory control unit 15, which controls the data storage to and from the memory 16; a digital signal processing unit 17; and an external memory control unit 20, which controls the data storage to and from the storage medium 21.

[0050] The lens assembly 40 can be a device that can be attached to and detached from the main body 100A, or it can be an integrated device with the main body 100A. The imaging lens 1 includes a focusing lens.

[0051] A focusing lens is a lens used to adjust the focal point of a camera optical system, including an imaging lens 1 and an aperture 2. It consists of a single lens or multiple lenses. By moving the focusing lens along the optical axis, the position of the principal point of the focusing lens (hereinafter also referred to as the focusing lens position) changes along the optical axis, thereby changing the focal point position on the subject side. Alternatively, a liquid lens that allows the position of the principal point in the optical axis direction to be changed electrically can be used as the focusing lens.

[0052] The lens control unit 4 of the lens assembly 40 controls the lens drive unit 8 according to the lens drive signal sent from the system control unit 11, thereby changing the position of the focusing lens. The lens control unit 4 of the lens assembly 40 controls the aperture drive unit 9 according to the drive control signal sent from the system control unit 11, thereby changing the aperture 2 opening size.

[0053] Imaging element 5 captures an image of the subject using the aforementioned imaging optical system positioned between the imaging element 5 and the subject. Imaging element 5 has a two-dimensional imaging surface 60 composed of multiple pixels (reference). Figure 2 The imaging element 5 converts the image of the subject, which is imaged onto the imaging surface 60 by the imaging optical system, into an image signal through the multiple pixels and outputs it. The output of the pixels included in the imaging element 5 is recorded as a pixel signal or pixel value, and the set of pixel signals or pixel values ​​is recorded as an image signal.

[0054] Imaging element 5 may be, for example, a CMOS (complementary metal-oxide semiconductor) image sensor or a CCD (charge-coupled device) image sensor. The following describes an example where imaging element 5 is a CMOS image sensor.

[0055] The system control unit 11 centrally controls the entire digital camera 100, and its hardware structure consists of various processors that execute programs for processing. The programs executed by the system control unit 11 (including the control program for the imaging element 5) are stored in the ROM (non-temporary storage medium) of the memory 16. The memory 16 and the system control unit 11 constitute the control device for the imaging element 5.

[0056] Various processors include general-purpose processors that execute programs for various processes, such as CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), and programmable logic devices (PLDs) whose circuit structures can be modified after manufacturing. Application-Specific Integrated Circuits (ASICs) are processors with circuit structures specifically designed to perform specific processes, i.e., dedicated circuits. More specifically, the structure of these various processors is a circuit composed of circuit elements such as semiconductor components.

[0057] The system control unit 11 can be composed of one of various processors, or it can be composed of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of CPU and FPGA).

[0058] The system control unit 11 drives the imaging element 5 and the lens device 40, and outputs the image of the subject captured by the camera optical system through the lens device 40 as an image signal. The image signal output from the imaging element 5 is processed by the digital signal processing unit 17 to generate data suitable for display on the display device 22 or data suitable for storage in the storage medium 21, i.e., camera image data.

[0059] User-instructed signals are input to the system control unit 11 via the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b and various buttons.

[0060] The display device 22 includes: a display surface 22b, which is composed of an organic EL (electroluminescence) panel or a liquid crystal panel, etc.; and a display controller 22a, which controls the display on the display surface 22b.

[0061] The memory control unit 15, digital signal processing unit 17, external memory control unit 20 and display controller 22a are connected to each other via control bus 24 and data bus 25 and are controlled by instructions from system control unit 11.

[0062] Figure 2 It means Figure 1 The diagram shows a schematic plan view of the imaging element 5. The imaging element 5 includes: an imaging surface 60, in which a plurality of pixel groups 62, each consisting of a plurality of pixels 61 arranged in the row direction X, are arranged in the column direction Y, which intersects the row direction X; a driving circuit 63, which drives the pixels 61 arranged in the imaging surface 60; and a signal processing circuit 64, which processes the pixel signals read from each pixel 61 of the pixel group 62 arranged in the imaging surface 60 to a signal line.

[0063] exist Figure 2 In this example, the row direction X is orthogonal to the column direction Y. One direction of the row direction X is denoted as the right direction XR, and the other direction is denoted as the left direction XL. One direction of the column direction Y is denoted as the up direction YU, and the other direction is denoted as the down direction YD.

[0064] Alternatively, it can be said that multiple pixel groups, each consisting of multiple pixels 61 arranged along the column direction Y, are arranged along the row direction X on the imaging surface 60. One of the row direction X and the column direction Y constitutes the first direction, and the other constitutes the second direction.

[0065] Figure 3 It is a local magnification representation Figure 2 The diagram shows a schematic of the imaging surface 60 of the imaging element 5. Multiple pixels 61 disposed on the imaging surface 60 include a phase difference pixel P1 (blocks labeled "FA" or "FB" in the diagram) for phase difference detection and a regular pixel P2 that is different from the phase difference pixel P1. The optical characteristics (changes in output relative to the angle of incidence of light) of the regular pixel P2 are different from those of the phase difference pixel P1. The phase difference pixel P1 constitutes the first pixel, and the regular pixel P2 constitutes the second pixel.

[0066] The ordinary pixel P2 includes multiple pixels corresponding to multiple wavelength bands. These multiple wavelength bands are included in the wavelength bands of visible light (for example, the wavelength band above 350nm and below 800nm).

[0067] These multiple wavelength bands include, for example, a red wavelength band (e.g., a wavelength band above 600 nm and below 800 nm, hereinafter also referred to as R light), a green wavelength band (e.g., a wavelength band above 500 nm and below 600 nm, hereinafter also referred to as G light), and a blue wavelength band (e.g., a wavelength band above 400 nm and below 500 nm, hereinafter also referred to as B light). R light constitutes the first wavelength band, G light constitutes the second wavelength band, and B light constitutes the third wavelength band.

[0068] exist Figure 3In the example, among the multiple pixels 61, ordinary pixels P2 include ordinary pixel 61R (the block marked with the character "R" in the figure) corresponding to R light, ordinary pixel 61G (the block marked with the character "G" in the figure) corresponding to G light, and ordinary pixel 61B (the block marked with the character "B" in the figure) corresponding to B light. Ordinary pixels 61R, 61G, and 61B each receive at least a portion of the light in their corresponding wavelength band and output a pixel signal corresponding to that light quantity.

[0069] Phase difference pixel P1 includes a first type of pixel corresponding to G light in the aforementioned wavelength bands (R light, G light, and B light) (phase difference pixel 61FAg and phase difference pixel 61FBg marked with the character "FA(G)" in the figure) and a second type of pixel that is different from the first type of pixel and corresponds to the aforementioned wavelength bands (R light, G light, and B light) (phase difference pixel 61FAw and phase difference pixel 61FBw marked with the character "FA(W)" in the figure). The spectral characteristics of the first type of pixel and the second type of pixel are different.

[0070] Phase difference pixel 61FAg and phase difference pixel 61FBg each receive at least a portion of the corresponding G light and output a pixel signal corresponding to the amount of light.

[0071] Phase difference pixels 61FAw and 61FBw respectively receive at least a portion of light from each of the aforementioned multiple wavelength bands (R light, G light, and B light) and output a pixel signal corresponding to the amount of light. That is, phase difference pixels 61FAw and 61FBw respectively receive at least a portion of R light, at least a portion of G light, and at least a portion of B light, and output a pixel signal corresponding to the amount of light. Phase difference pixels 61FAw and 61FBw can receive light of more wavelengths than phase difference pixels 61FAg and 61FBg, thus improving sensitivity.

[0072] On the imaging surface 60, ordinary pixels 61R, 61G, and 61B are arranged according to a predetermined pattern (a Bayer pattern in the example shown in the figure). That is, on the imaging surface 60, ordinary pixels 61G and 61R are alternately arranged along the column direction Y, forming a GR pixel group, and ordinary pixels 61B and 61G are alternately arranged along the row direction X, forming a BG pixel group. The arrangement pattern of the pixels 61 on the imaging surface 60 is not limited to the Bayer pattern, and various patterns can be used.

[0073] The phase difference pixel P1 is set in part of the configuration position of the ordinary pixel P2 based on the Bayer pattern.

[0074] Specifically, in a portion of the BG pixel groups among the multiple pixel groups disposed on the imaging surface 60, a phase difference pixel 61FAg is disposed in a portion of the configuration position of the ordinary pixel 61B based on the Bayer pattern, and a phase difference pixel 61FBg is disposed in a portion of the configuration position of the ordinary pixel 61G based on the Bayer pattern. The phase difference pixel 61FAg and the phase difference pixel 61FBg are adjacent. In this portion of the BG pixel group, a plurality of first-type phase difference detection pairs, consisting of the phase difference pixel 61FAg and its adjacent phase difference pixel 61FBg, are arranged at intervals in the row direction X. This portion of the BG pixel group may consist only of the first-type phase difference detection pairs. On the imaging surface 60, a plurality of pixel groups G1 containing the first-type phase difference detection pairs are dispersedly disposed along the column direction Y.

[0075] Furthermore, in a portion of the BG pixel group, which does not include the first type of phase difference detection pair, among the multiple pixel groups disposed on the imaging surface 60, a phase difference pixel 61FAw is disposed in a portion of the configuration position of the ordinary pixel 61B based on the Bayer pattern, and a phase difference pixel 61FBw is disposed in a portion of the configuration position of the ordinary pixel 61G based on the Bayer pattern. The phase difference pixel 61FAw and the phase difference pixel 61FBw are adjacent. In this portion of the BG pixel group, multiple second type of phase difference detection pairs, consisting of the phase difference pixel 61FAw and its adjacent phase difference pixel 61FBw, are arranged at intervals in the row direction X. This portion of the BG pixel group may consist only of the second type of phase difference detection pairs. On the imaging surface 60, multiple pixel groups G2 containing the second type of phase difference detection pairs are dispersed along the column direction Y. Figure 3 As shown, pixel group G1 and pixel group G2 are arranged alternately in the column direction Y.

[0076] Figure 4 It means Figure 3 A schematic diagram of an example of a cross-section of range A1 shown. Figure 5 It means Figure 4 A schematic diagram of an example of a cross-section of range A2 shown. Figure 6 It means Figure 4 A schematic diagram of an example of a cross-section of range A3 shown.

[0077] like Figures 4 to 6 As shown, the ordinary pixel P2 disposed on the imaging surface 60 includes: a microlens ML that gathers light from the subject; a photoelectric conversion unit PD that converts the light gathered by the microlens ML into electrical charge; a color filter CF disposed between the photoelectric conversion unit PD and the microlens ML, and transmits light of a specific wavelength band; and a light-shielding film LS disposed between the color filter CF and the photoelectric conversion unit PD. The photoelectric conversion unit PD is a photodiode formed in a semiconductor substrate such as silicon, but it may also be composed of an organic material film disposed on the top of the semiconductor substrate.

[0078] The color filter CF included in the ordinary pixel 61R (in Figure 4 The color filter (referred to as R filter in the text) transmits at least a portion of the R light, and the color filter CF included in the ordinary pixel 61G (in... Figures 4 to 6 The color filter CF (referred to as G filter in the text) transmits at least a portion of the G light, and the ordinary pixel 61B contains a color filter CF (in Figure 5 and Figure 6 (Recorded as a B filter) transmits at least a portion of the B light.

[0079] like Figure 5 As shown, phase difference pixels 61FAg and 61FBg are structures in which the size of the opening of the light-shielding film LS is changed relative to the structure of the ordinary pixel 61G. The opening of the light-shielding film LS of the phase difference pixel 61FAg is such that it closes, for example, the left half of the opening of the light-shielding film LS of the ordinary pixel 61G. The opening of the light-shielding film LS of the phase difference pixel 61FBg is such that it closes, for example, the right half of the opening of the light-shielding film LS of the ordinary pixel 61G.

[0080] like Figure 6 As shown, phase difference pixels 61FAw and 61FBw have respectively changed the size of the opening of the light-shielding film LS and the color filter CF has been changed to the structure of the brightness filter LF, which is relative to the ordinary pixel P2.

[0081] A luminance filter (LF) possesses spectral characteristics related to the luminance component of light and is equivalent to an ND (Neutral Density) filter, a transparent filter, a white filter, or a gray filter. A structure in which light directly enters the photoelectric conversion unit (PD) without any components obstructing light transmission between the microlens (ML) and the light-shielding film (LS) can also be considered to have an luminance filter (LF). Compared to a color filter (CF), the luminance filter (LF) can transmit more wavelength components of light. The luminance filter (LF) transmits at least a portion of the R-wave light, at least a portion of the G-wave light, and at least a portion of the B-wave light.

[0082] The opening of the light-shielding film LS of phase difference pixel 61FAw is structured to close, for example, the left half of the opening of the light-shielding film LS of ordinary pixel P2. The opening of the light-shielding film LS of phase difference pixel 61FBw is structured to close, for example, the right half of the opening of the light-shielding film LS of ordinary pixel P2.

[0083] Through the structure of this light-shielding film LS, phase difference pixels 61FAg and 61FAw each receive one of a pair of light beams consisting of two different portions arranged in the row direction X of the pupil region of the imaging optical system of the lens device 40. Furthermore, phase difference pixels 61FBg and 61FBw each receive the other portion of the aforementioned pair of light beams. And the ordinary pixel P2 receives both of the aforementioned pair of light beams.

[0084] In addition, Figures 4 to 6 In the example, the optical characteristics of phase difference pixel P1 and ordinary pixel P2 are changed according to the shape of the light-shielding film LS, but the same optical characteristics can also be achieved according to the shape of the microlens ML.

[0085] For example, such as Figure 7 As shown, the opening size of the light-shielding film LS of all pixels 61 is made the same. Instead, in the second type of phase difference detection pair, the microlens ML can be configured to be shared by phase difference pixels 61FAw and 61FBw. Although the illustration is omitted, in the first type of phase difference detection pair, the microlens ML can also be configured to be shared by phase difference pixels 61FAg and 61FBg.

[0086] The system control unit 11 performs focus detection processing based on the pixel value of the phase difference pixel P1. Specifically, the system control unit 11 performs correlation operations between the pixel signal group output from the phase difference pixel 61FAg included in the same pixel group G1 and the pixel signal group output from the phase difference pixel 61FBg to detect a first phase difference in the line direction X. Furthermore, the system control unit 11 performs correlation operations between the pixel signal group output from the phase difference pixel 61FAw included in the same pixel group G2 and the pixel signal group output from the phase difference pixel 61FBw to detect a second phase difference in the line direction X. Based on at least one of the first and second phase differences, the system control unit 11 performs focus detection processing to derive the focusing lens position required for focusing on the subject.

[0087] The second phase difference detection pair can receive light across more wavelength bands than the first phase difference detection pair and improves sensitivity. Therefore, the detection accuracy of the second phase difference derived from the pixel values ​​of the second phase difference detection pair can be improved. On the other hand, because the second phase difference detection pair can receive light across multiple wavelength bands, it may not be well-suited for detecting phase differences in certain subjects.

[0088] For example, for subjects containing alternating RB stripe patterns of red and blue subject regions extending in a straight line in the column direction Y and arranged in a straight line in the column direction Y, BG stripe patterns of alternating blue and green subject regions extending in a straight line in the column direction Y and arranged in a straight line in the column direction Y and arranged in a straight line in the column direction Y and arranged in a straight line in the column direction X, and RG stripe patterns of alternating red and green subject regions extending in a straight line in the column direction Y and arranged in a straight line in the column direction Y and arranged in a straight line in the column direction X, it is sometimes difficult to detect the second phase difference.

[0089] However, for subjects containing BG or RG stripe patterns, the first phase difference can be detected with high precision by using the pixel values ​​of the first phase difference detection pair. For example, if the second phase difference cannot be detected, or if the second phase difference can be detected but its reliability is below a threshold, the system control unit 11 determines the focusing lens position based on the first phase difference. In this way, the detection accuracy of the phase difference can be improved, thereby enabling high-precision focus control.

[0090] Furthermore, the system control unit 11 can control the exposure times to be different in pixel group G1, which contains the first type of phase difference detection pair, and pixel group G2, which contains the second type of phase difference detection pair. For example, the system control unit 11 controls the exposure time of pixel group G1 to a first time, and controls the exposure time of pixel group G2 to a second time, which is shorter than the first time.

[0091] The second type of phase difference pair contained in pixel group G2 has high sensitivity, thus preventing pixel saturation by shortening the exposure time. On the other hand, for the first type of phase difference pair contained in pixel group G1, the signal-to-noise ratio can be improved by extending the exposure time. These effects further improve the detection accuracy of the phase difference.

[0092] Figure 8 This is a schematic diagram showing a first modified example of imaging element 5, and is related to... Figure 3 The corresponding diagram. Figure 8 The imaging element 5 in the modified example shown is relative to Figure 3 The imaging element 5 shown is configured such that the first type of phase difference detection pair disposed in a portion of pixel group G1 is changed to a second type of phase difference detection pair, and the second type of phase difference detection pair disposed in a portion of pixel group G2 is changed to a first type of phase difference detection pair. Figure 8 In the example, in each of pixel group G1 and pixel group G2, a first type of phase difference detection pair and a second type of phase difference detection pair are alternately arranged in the row direction X.

[0093] Even if Figure 8The structure shown can also improve the detection accuracy of phase difference. Figure 8 In the case of the structure shown, the system control unit 11 can detect the phase difference based on the pixel value of the first phase difference detection pair included in the pixel group G1, and detect the phase difference based on the pixel value of the second phase difference detection pair included in the pixel group G1.

[0094] exist Figure 3 In the structure shown, with Figure 8 Compared to the structure shown, the configuration interval of the row direction X of the first type of phase difference detection pair and the configuration interval of the row direction X of the second type of phase difference detection pair can be reduced respectively. Therefore, according to Figure 3 The structure shown can further improve the detection accuracy of phase difference. Furthermore, according to... Figure 3 The structure shown contains the first phase difference detection pair and the second phase difference detection pair in different pixel groups. Therefore, even if the exposure time is changed in the first phase difference detection pair and the second phase difference detection pair, the control becomes easy, which can reduce the manufacturing cost of the digital camera 100.

[0095] In addition, Figure 3 and Figure 8 In the example, the two pixels 61 constituting the first type of phase difference detection pair correspond to the G light, but are not limited to this. For example, the two pixels 61 constituting the first type of phase difference detection pair can correspond to the R light or the B light.

[0096] Furthermore, it is also possible to configure the imaging plane 60 to include two or more of the following structures as the first type of phase difference detection pair: a phase difference detection pair corresponding to the R light, a phase difference detection pair corresponding to the G light, and a phase difference detection pair corresponding to the B light.

[0097] In the description above, it was assumed that a phase difference detection pair for detecting the phase difference in the row direction X was arranged on the imaging surface 60. The technology of the present invention can also be applied to an imaging element 5 in which a phase difference detection pair for detecting the phase difference in the column direction Y is arranged on the imaging surface 60. Furthermore, it can also be applied to an imaging element 5 in which both a phase difference detection pair for detecting the phase difference in the row direction X and a phase difference detection pair for detecting the phase difference in the column direction Y are arranged on the imaging surface 60.

[0098] Figure 9 This is a diagram showing the second modified example of imaging element 5, and it is related to... Figure 3 The corresponding diagram. Figure 9 The imaging element 5 shown has the following structure: Figure 8In the imaging element 5 shown, the position of the phase difference pixel 61FBg constituting the first phase difference detection pair is changed to be adjacent to the phase difference pixel 61FAg constituting the first phase difference detection pair in the downward direction YD, and a normal pixel 61G is arranged in the original position of the phase difference pixel 61FBg.

[0099] and, Figure 9 The imaging element 5 shown has the following structure: Figure 8 In the imaging element 5 shown, the position of the phase difference pixel 61FBw constituting the second phase difference detection pair is changed to be adjacent to the phase difference pixel 61FAw constituting the second phase difference detection pair in the downward direction YD, and a normal pixel 61G is arranged in the original position of the phase difference pixel 61FBw.

[0100] exist Figure 9 In the imaging element 5 shown, the openings of the light-shielding films LS of phase difference pixels 61FAg and 61FAw form a structure that, for example, closes the upper half of the opening of the light-shielding film LS of the ordinary pixel 61G. The openings of the light-shielding films LS of phase difference pixels 61FBg and 61FBw form a structure that, for example, closes the lower half of the opening of the light-shielding film LS of the ordinary pixel 61G.

[0101] in this way, Figure 9 The imaging element 5 shown has the following structure: a first phase difference detection pair is arranged along the column direction Y in a part of a pixel group consisting of a plurality of pixels 61 arranged along the column direction Y, and a second phase difference detection pair is arranged along the column direction Y in a part of another pixel group that does not contain the first phase difference detection pair.

[0102] The system control unit 11 performs correlation operations between pixel signal groups output from phase difference pixels 61FAg and phase difference pixels 61FBg within the same pixel group to detect a third phase difference in the column direction Y. Furthermore, the system control unit 11 performs correlation operations between pixel signal groups output from phase difference pixels 61FAw and phase difference pixels 61FBw within the same pixel group to detect a fourth phase difference in the column direction Y. Based on at least one of the third and fourth phase differences, the system control unit 11 performs focus detection processing to derive the position of the focusing lens required for focusing on the subject.

[0103] Figure 10 This is a diagram showing the third modified example of imaging element 5, and it is related to... Figure 3 The corresponding diagram. Figure 11 It means Figure 10 A schematic diagram of an example of a cross-section of range A4 shown. Figure 12 It means Figure 10A schematic diagram of an example of a cross-section of range A5 shown.

[0104] exist Figure 10 In the imaging element 5 shown, relative to Figure 3 The imaging element 5 shown has its phase difference pixel 61FAg, which constitutes the first type of phase difference detection pair, changed to phase difference pixel 61FAc (the block marked "FA(Cy)" in the figure), and its phase difference pixel 61FBg, which constitutes the first type of phase difference detection pair, changed to phase difference pixel 61FBc (the block marked "FB(Cy)" in the figure).

[0105] Furthermore, in Figure 10 In the imaging element 5 shown, relative to Figure 3 The imaging element 5 shown has its phase difference pixel 61FAw constituting the second phase difference detection pair changed to phase difference pixel 61FAy (the block marked "FA(Y)" in the figure), and its configuration position changed from the BG pixel group to the GR pixel group.

[0106] Furthermore, in Figure 10 In the imaging element 5 shown, relative to Figure 3 The imaging element 5 shown has its phase difference pixel 61FBw, which constitutes the second phase difference detection pair, changed to phase difference pixel 61FBy (the block marked "FB(Y)" in the figure), and its configuration position has changed from the BG pixel group to the GR pixel group.

[0107] The phase difference pixel 61FAc and phase difference pixel 61FBc constituting the first type of phase difference detection pair correspond to multiple wavelength bands (R light, G light, and B light) respectively (specifically, G light and B light). Phase difference pixel 61FAc and phase difference pixel 61FBc respectively receive at least a portion of the G light and at least a portion of the B light, and output a pixel signal corresponding to the amount of light. Phase difference pixel 61FAc and phase difference pixel 61FBc are capable of receiving more than... Figure 3 The phase difference pixels 61FAg and 61FBg in the imaging element 5 shown emit more wavelengths of light, thus improving sensitivity.

[0108] The phase difference pixels 61FAy and 61FBy constituting the second type of phase difference detection pair correspond to multiple wavelength bands (R light, G light, and B light) respectively (specifically, R light and G light). Phase difference pixels 61FAy and 61FBy respectively receive at least a portion of the R light and at least a portion of the G light, and output pixel signals corresponding to the light amounts. Phase difference pixels 61FAy and 61FBy are capable of receiving more than... Figure 3 The phase difference pixels 61FAg and 61FBg in the imaging element 5 shown emit more wavelengths of light, thus improving sensitivity.

[0109] exist Figure 10 In the imaging element 5 shown, in a portion of the BG pixel group among the multiple pixel groups disposed on the imaging surface 60, a phase difference pixel 61FAc is disposed at a portion of the configuration position of the ordinary pixel 61B based on the Bayer pattern, and a phase difference pixel 61FBc is disposed at a portion of the configuration position of the ordinary pixel 61G based on the Bayer pattern. The phase difference pixel 61FAc and the phase difference pixel 61FBc are adjacent. In this portion of the BG pixel group, a plurality of first-type phase difference detection pairs, consisting of the phase difference pixel 61FAc and its adjacent phase difference pixel 61FBc, are arranged at intervals in the row direction X. On the imaging surface 60, a plurality of pixel groups G3 containing the first-type phase difference detection pairs are dispersedly disposed along the column direction Y.

[0110] Furthermore, in a portion of the GR pixel groups among the multiple pixel groups disposed on the imaging surface 60, a phase difference pixel 61FAy is disposed in a portion of the configuration position of the ordinary pixel 61G based on the Bayer pattern, and a phase difference pixel 61FBy is disposed in a portion of the configuration position of the ordinary pixel 61R based on the Bayer pattern. The phase difference pixels 61FAy and 61FBy are adjacent. In this portion of the GR pixel group, a plurality of second phase difference detection pairs, consisting of the phase difference pixel 61FAy and its adjacent phase difference pixel 61FBy, are arranged at intervals in the row direction X. On the imaging surface 60, a plurality of pixel groups G4 containing the second phase difference detection pairs are dispersedly disposed along the column direction Y. Figure 10 As shown, pixel group G3 and pixel group G4 are arranged alternately in the column direction Y.

[0111] like Figure 11 As shown, in each of the phase difference pixels 61FAy and 61FBy, a color filter CF (referred to as a Y filter in the figure) is provided between the microlens ML and the light-shielding film LS, which is capable of transmitting at least a portion of the G light and at least a portion of the R light.

[0112] like Figure 12 As shown, in each of the phase difference pixels 61FAc and 61FBc, a color filter CF (referred to as a Cy filter in the figure) is provided between the microlens ML and the light-shielding film LS, which is capable of transmitting at least a portion of the G light and at least a portion of the B light.

[0113] In the digital camera 100 including the imaging element 5 with the above structure, the system control unit 11 performs focus detection processing based on the pixel value of the phase difference pixel P1.

[0114] Specifically, the system control unit 11 performs correlation operations between pixel signal groups output from phase difference pixels 61FAc included in the same pixel group G3 and pixel signal groups output from phase difference pixels 61FBc to detect a fifth phase difference in the line direction X. Furthermore, the system control unit 11 performs correlation operations between pixel signal groups output from phase difference pixels 61FAy included in the same pixel group G4 and pixel signal groups output from phase difference pixels 61FBy to detect a sixth phase difference in the line direction X. Based on at least one of the fifth and sixth phase differences, the system control unit 11 performs focus detection processing to derive the position of the focusing lens required for focusing on the subject.

[0115] exist Figure 10 The imaging element 5 shown can improve the sensitivity of both the first and second phase difference detection pairs. Therefore, the detection accuracy of the phase difference can be improved. On the other hand, since both the first and second phase difference detection pairs can receive light from multiple wavelength bands, there are subjects that are not well-suited for phase difference detection.

[0116] For example, for the subject with the BG stripe pattern described above, using the pixel values ​​of the second phase difference detection pair enables phase difference detection with higher accuracy compared to using the pixel values ​​of the first phase difference detection pair. Furthermore, for the subject with the RG stripe pattern described above, using the pixel values ​​of the first phase difference detection pair enables phase difference detection with higher accuracy compared to using the pixel values ​​of the second phase difference detection pair.

[0117] Additionally, for example, even in Figure 10 By setting the positions of phase difference pixels 61FAy and 61FAc to opposite positions, and setting the positions of phase difference pixels 61FBy and 61FBc to opposite positions, the same improvement in phase difference detection accuracy can be achieved. However, according to... Figure 10 The structure shown is advantageous from the perspective of color mixing.

[0118] For example, in Figure 10 In the imaging element 5 shown, a portion of the B-light and G-light that have passed through the color filter of the phase difference pixel 61FAc and travel along the column direction Y are incident on the photoelectric conversion unit PD of the ordinary pixel 61G on both sides of the column direction Y of the phase difference pixel 61FAc. This may result in blue color mixing in the pixel signal of the ordinary pixel 61G. However, an ordinary pixel 61B is disposed at an adjacent position in the column direction Y of the ordinary pixel 61G. A portion of the B-light that has passed through the color filter CF of the ordinary pixel 61B is incident on the photoelectric conversion unit PD of the ordinary pixel 61G. This may also result in blue color mixing.

[0119] Assuming a phase difference pixel 61FAy is positioned at the location of phase difference pixel 61FAc, a portion of the R-light that has passed through the color filter CF of the phase difference pixel 61FAy is incident into the photoelectric conversion unit PD of the ordinary pixel 61G. Therefore, it is possible for blue and red to mix in the ordinary pixel 61G. Thus, according to... Figure 10 The structure shown can set only one color to the color mixing that may occur in ordinary pixels 61G located at adjacent positions in the column direction Y of phase difference pixel 61FAc, thus improving the image quality of the camera.

[0120] Furthermore, in Figure 10 In the imaging element 5 shown, a portion of the B-light and G-light that have passed through the color filter of the phase difference pixel 61FAc and travel along the row direction X are incident on the photoelectric conversion unit PD of the ordinary pixel 61G located adjacent to the phase difference pixel 61FAc in the column direction Y. This may result in blue color mixing in the pixel signal of the ordinary pixel 61G. However, if an ordinary pixel 61B is located adjacent to the ordinary pixel 61G in the column direction Y, a portion of the B-light that has passed through the color filter CF of the ordinary pixel 61B is incident on the photoelectric conversion unit PD of the ordinary pixel 61G, which may also result in blue color mixing.

[0121] Assuming a phase difference pixel 61FAy is positioned at the location of phase difference pixel 61FAc, a portion of the R-light that has passed through the color filter CF of the phase difference pixel 61FAy is incident into the photoelectric conversion unit PD of the ordinary pixel 61G. Therefore, it is possible for blue and red to mix in the ordinary pixel 61G. Thus, according to... Figure 10 The structure shown can set only one color to the possible color mixing that may occur in ordinary pixels 61G located at adjacent positions in the row direction X of phase difference pixel 61FAc, thus improving the image quality of the camera.

[0122] This color mixing suppression effect is also achieved in the ordinary pixels 61G on both sides of the column direction Y of the phase difference pixel 61FBy and the ordinary pixel 61G on the right side of the row direction X of the phase difference pixel 61FBy. Furthermore, it is also achieved in the ordinary pixel 61R on the left side of the row direction X of the phase difference pixel 61FAy. And it is also achieved in the ordinary pixel 61B on the right side of the row direction X of the phase difference pixel 61FBc.

[0123] for Figure 10 The imaging element 5 shown, and the system control unit 11, can also control the exposure time to be different in the pixel group G3 containing the first type of phase difference detection pair and the pixel group G4 containing the second type of phase difference detection pair. This control can further improve the detection accuracy of the phase difference.

[0124] Figure 13 This is a schematic diagram illustrating a fourth modification of the imaging element 5. The pixel arrangement of the imaging element 5 in the fourth modification is... Figure 3 The pixel arrangement shown is the same, but the internal structures of phase difference pixels 61FAw and 61FBw are different. Figure 13 In the fourth variation, the imaging element 5 is shown. Figure 3 Within the six pixels 61 of range A6 shown, the state of the layer between the light-shielding film LS and the microlens ML is observed from the microlens ML side.

[0125] Figure 13 The configuration position of the phase difference pixel 61FAw shown is the configuration position of the ordinary pixel 61B based on the Bayer pattern. Furthermore, a color filter CFa is provided at the periphery of the phase difference pixel 61FAw, which is configured according to the Bayer pattern at the position where the ordinary pixel 61B should be configured, and transmits at least a portion of the wavelength band corresponding to the ordinary pixel 61B, i.e., B light.

[0126] Color filter CFa is disposed on the same layer as color filter CF. Color filter CFa is disposed in a generally C-shape around luminance filter LF. Color filter CFa includes: a first part CFau, disposed along the upper YU side edge of luminance filter LF; a second part CFad, disposed along the lower YD side edge of luminance filter LF; and a third part CFal, connecting the left XL side edges of the first part CFau and the second part CFad to each other, and disposed along the left XL side edge of luminance filter LF.

[0127] Figure 13 The configuration position of the phase difference pixel 61FBw shown becomes the configuration position of the ordinary pixel 61G based on the Bayer pattern. Moreover, a color filter CFb is provided at the periphery of the phase difference pixel 61FBw, which is configured according to the Bayer pattern at the position where the ordinary pixel 61G should be configured, and transmits at least a portion of the wavelength band corresponding to the ordinary pixel 61G, i.e., G light.

[0128] Color filter CFb is disposed on the same layer as color filter CF. Color filter CFb is disposed in a generally inverted C-shape around luminance filter LF. Color filter CFb includes: a first portion CFbu, disposed along the upper edge of luminance filter LF in the YU direction; a second portion CFbd, disposed along the lower edge of luminance filter LF in the YD direction; and a third portion CFbr, connecting the right-hand XR edge of the first portion CFbu and the second portion CFbd to each other, and disposed along the right-hand XR edge of luminance filter LF.

[0129] according to Figure 13The structures of phase difference pixels 61FAw and 61FBw shown can attenuate the R and G rays from the luminance filter LF of phase difference pixel 61FAw through the color filter CFa. As a result, most of the light incident on the photoelectric conversion unit PD of the adjacent ordinary pixel P2 of phase difference pixel 61FAw can be set to blue, and color mixing of multiple colors can be prevented in ordinary pixel P2.

[0130] Furthermore, the R and B rays in the luminance filter LF of the phase difference pixel 61FBw can be attenuated by the color filter CFb. As a result, most of the light incident on the photoelectric conversion unit PD of the adjacent ordinary pixel P2 of the phase difference pixel 61FBw can be set to green, and color mixing of multiple colors can be prevented in the ordinary pixel P2.

[0131] Furthermore, in the phase difference pixel 61FAw, the width of the column direction Y of the first part CFau of the color filter CFa, the width of the column direction Y of the second part CFad of the color filter CFa, and the width of the row direction X of the third part CFal of the color filter CFa can be set to be different depending on the configuration position of the phase difference pixel 61FAw in the imaging surface 60.

[0132] Furthermore, in the phase difference pixel 61FBw, the width of the column direction Y of the first part CFbu of the color filter CFb, the width of the column direction Y of the second part CFbd of the color filter CFb, and the width of the row direction X of the third part CFbr of the color filter CFb can be set to be different according to the configuration position of the phase difference pixel 61FBw in the imaging surface 60.

[0133] For example, regarding the phase difference pixel 61FAw and phase difference pixel 61FBw located at the ends of the imaging plane 60 on the upward YU side, such as Figure 14 As shown, compared with the phase difference pixel 61FAw and phase difference pixel 61FBw located in the central part of the column direction Y in the imaging plane 60, the width of the column direction Y of the first part CFau and the width of the column direction Y of the first part CFbu are increased respectively.

[0134] Furthermore, regarding the phase difference pixel 61FAw and phase difference pixel 61FBw located at the ends of the imaging plane 60 on the downward YD side, as follows... Figure 15 As shown, compared with the phase difference pixel 61FAw and phase difference pixel 61FBw located in the central part of the column direction Y in the imaging plane 60, the width of the column direction Y of the second part CFad and the width of the column direction Y of the second part CFbd are increased respectively.

[0135] Furthermore, regarding the phase difference pixel 61FAw and phase difference pixel 61FBw located at the left-hand XL side end of the imaging plane 60, as follows... Figure 16 As shown, compared to the phase difference pixel 61FAw located in the center of the row direction X in the imaging plane 60, the width of the row direction X of the third part CFal is increased.

[0136] Furthermore, regarding the phase difference pixel 61FAw and phase difference pixel 61FBw located at the right-hand XR side end of the imaging plane 60, as follows... Figure 17 As shown, compared to the phase difference pixel 61FBw located in the center of the row direction X in the imaging plane 60, the width of the third part CFbr in the row direction X is increased.

[0137] Thus, by changing the widths of color filters CFa and CFb according to the configuration positions of phase difference pixels 61FAw and 61FBw, it is possible to suppress the mixing of multiple colors in the adjacent pixels 61, even at the periphery of the imaging surface 60 where the incident angle of light becomes steeper.

[0138] in addition, Figures 13 to 17 The structures of the color filters CFa and CFb shown can also be applied to... Figure 10 The imaging element 5 is shown in the diagram. For example, as... Figure 18 As shown, a color filter CFa is added to the phase difference pixel 61FAy in the form of surrounding the color filter CF contained in the phase difference pixel 61FAy, which transmits at least a portion of the G light. This prevents the R light passing through the color filter CF of the phase difference pixel 61FAy from entering the photoelectric conversion unit PD of the adjacent ordinary pixel P2.

[0139] Furthermore, a color filter CFb is added to the phase difference pixel 61FBy in the form of surrounding the color filter CF contained in the phase difference pixel 61FBy, which transmits at least a portion of the R light. This prevents the G light passing through the color filter CF of the phase difference pixel 61FBy from entering the photoelectric conversion unit PD of the adjacent ordinary pixel P2.

[0140] Moreover, such as Figure 19 As shown, a color filter CFa is added to the phase difference pixel 61FAc in the form of surrounding the color filter CF contained in the phase difference pixel 61FAc, which transmits at least a portion of the B light. This prevents the G light passing through the color filter CF of the phase difference pixel 61FAc from entering the photoelectric conversion unit PD of the adjacent ordinary pixel P2.

[0141] Furthermore, a color filter CFb is added to the phase difference pixel 61FBc in the form of surrounding the color filter CF included in the phase difference pixel 61FBc, which transmits at least a portion of the G light. This prevents the B light passing through the color filter CF of the phase difference pixel 61FBc from entering the photoelectric conversion unit PD of the adjacent ordinary pixel P2.

[0142] Next, the structure of a smartphone, which is another embodiment of the camera device according to the technology of the present invention, will be described.

[0143] Figure 20 This is a diagram showing the appearance of the smartphone 200. Figure 20 The smartphone 200 shown has a flat frame 201, and a display input unit 204 is provided on one side of the frame 201, which integrates a display panel 202 as a display unit and an operation panel 203 as an input unit.

[0144] Furthermore, this frame 201 includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. However, the structure of the frame 201 is not limited to this; for example, it can also have a structure where the display unit and input unit are independent, or a structure with a folding structure or a sliding mechanism.

[0145] Figure 21 It means Figure 20 The diagram shows the structure of the smartphone 200.

[0146] like Figure 21 As shown, the main components of a smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiver unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0147] Furthermore, as a primary function of the smartphone 200, it possesses wireless communication capabilities for mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0148] The wireless communication unit 210 performs wireless communication with the base station device BS housed in the mobile communication network NW according to the instructions of the main control unit 220. Using this wireless communication, it sends and receives various types of data, such as voice data, image data, and email data, and receives web page data or streaming data.

[0149] The display input unit 204 is a so-called touch panel that displays images (static images and dynamic images) or character information to visually convey information to the user and detects the user's operation on the displayed information under the control of the main control unit 220, and includes a display panel 202 and an operation panel 203.

[0150] The display panel 202 uses LCD (Liquid Crystal Display) or OELD (Organic Electro-Luminescence Display) as display devices.

[0151] The operation panel 203 is a device that displays an image on the display surface of the display panel 202 in a visually recognizable manner and detects one or more coordinates operated by a user's finger or stylus. If the device is operated by a user's finger or stylus, a detection signal generated by the operation is output to the main control unit 220. Then, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0152] like Figure 21 As shown, in a smartphone 200 exemplified as an embodiment of the camera device of the present invention, the display panel 202 and the operation panel 203 are integrated to form a display input unit 204, but the operation panel 203 is configured to completely cover the display panel 202.

[0153] With this configuration, the operation panel 203 can also detect user operations in areas other than the display panel 202. In other words, the operation panel 203 can have a detection area (hereinafter referred to as the display area) for the overlapping portion that overlaps with the display panel 202 and a detection area (hereinafter referred to as the non-display area) for the outer edge portion that does not overlap with the display panel 202.

[0154] Alternatively, the size of the display area can be exactly the same as the size of the display panel 202, but it is not necessary for them to be identical. Furthermore, the operation panel 203 can have two sensing areas: an outer edge and an inner side. The width of the outer edge is appropriately designed based on the size of the frame 201, etc.

[0155] Furthermore, the position detection method used in the operation panel 203 can include matrix switch method, resistive film method, surface elastic wave method, infrared method, electromagnetic induction method, electrostatic capacitance method, etc., or any method can be used.

[0156] The call unit 211 includes a speaker 205 or a microphone 206, which converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes the voice data received through the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0157] And, as Figure 20 As shown, for example, by mounting the speaker 205 on the same surface as the surface where the display input section 204 is provided, the microphone 206 can be mounted on the side of the frame 201.

[0158] The operation unit 207 uses hardware keys such as push-button switches and receives instructions from the user. For example, such as... Figure 20 As shown, the operation unit 207 is mounted on the side of the frame 201 of the smartphone 200, and is a button-type switch that is turned on when pressed by a finger or the like, and turned off by the restoring force of a spring or the like when the finger is released.

[0159] Storage unit 212 stores the control program and control data of main control unit 220, application software, address data that establishes corresponding associations for names or phone numbers of communication objects, data of sent and received emails, web data downloaded through a web browser, downloaded content data, and temporarily stores streaming data, etc. Furthermore, storage unit 212 consists of an internal storage unit 217 built into the smartphone and an external storage unit 218 with a removable external memory slot.

[0160] In addition, the internal storage units 217 and external storage units 218 constituting the storage unit 212 are implemented using storage media such as flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., MicroSD memory), RAM (Random Access Memory), ROM (Read Only Memory).

[0161] The external input / output unit 213 serves as an interface to all external devices connected to the smartphone 200, and is used to connect directly or indirectly to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, etc.) or networks (e.g., Ethernet, Wireless LAN, etc.).

[0162] External devices that connect to the smartphone 200 include, for example, wired / wireless headphones, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card slots, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video devices connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video devices, wired / wireless connected smartphones, wired / wireless connected personal computers, and headphones.

[0163] The external input / output unit 213 can transmit data received from such an external device to various internal components of the smartphone 200, or transmit data from the internal components of the smartphone 200 to an external device.

[0164] According to the instructions of the main control unit 220, the GNSS receiver 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn, and performs positioning calculations based on the received multiple GNSS signals to detect the location, including the latitude, longitude, and altitude of the smartphone 200. When the GNSS receiver 214 can obtain location information from the wireless communication unit 210 or the external input / output unit 213 (e.g., a wireless LAN), it can also use that location information to detect its location.

[0165] The motion sensor unit 215, for example, includes a 3-axis accelerometer, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. This detection result is output to the main control unit 220.

[0166] The power supply unit 216 supplies power stored in the battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0167] The main control unit 220 is equipped with a microprocessor and operates according to the control program and control data stored in the storage unit 212, thereby centrally controlling all parts of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. Furthermore, in order to conduct voice or data communication via the wireless communication unit 210, the main control unit 220 has mobile communication control functions and application processing functions for controlling all parts of the communication system.

[0168] The application processing functions are implemented by the main control unit 220 according to the application software stored in the storage unit 212. Examples of application processing functions include infrared communication functions that control the external input / output unit 213 to communicate with the target device, email functions that send and receive emails, and web browsing functions that browse web pages.

[0169] Furthermore, the main control unit 220 has image processing functions such as displaying images on the display input unit 204 based on received data or downloaded streaming data (data of static or dynamic images).

[0170] The image processing function refers to the function of the main control unit 220 to decode the above-mentioned image data, perform image processing on the decoding result, and display the image on the display input unit 204.

[0171] In addition, the main control unit 220 performs display control on the display panel 202 and operation detection control for user operations via the operation unit 207 and the operation panel 203.

[0172] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for creating emails.

[0173] Additionally, the scroll bar refers to a software key used to indicate the display portion of a large image or other image that is not fully contained within the display area of ​​the display panel 202.

[0174] Furthermore, by performing operation detection control, the main control unit 220 detects user operations through the operation unit 207, or accepts operations on the icons and input of strings into the input fields of the windows through the operation panel 203, or accepts scrolling requests for the displayed images through the scroll bar.

[0175] Furthermore, by performing operation detection control, the main control unit 220 has the following touch panel control functions: determining whether the operation position of the operation panel 203 is an overlapping part (display area) that overlaps with the display panel 202, or an outer edge part (non-display area) that does not overlap with the display panel 202, and controlling the display position of the sensing area or software key of the operation panel 203.

[0176] Furthermore, the main control unit 220 can also detect gesture operations on the operation panel 203 and execute preset functions based on the detected gesture operations.

[0177] Gesture operation is not the simple touch operation of the past, but refers to the operation of drawing a trajectory with fingers or other means, or specifying multiple positions at the same time, or combining these to draw a trajectory from multiple positions to at least one position.

[0178] Camera Department 208 includes Figure 1 The lens device 40, imaging element 5, and digital signal processing unit 17 are shown.

[0179] The image data generated by the camera unit 208 can be stored in the storage unit 212, or output through the external input / output unit 213 or the wireless communication unit 210.

[0180] exist Figure 21 In the smartphone 200 shown, the camera unit 208 is mounted on the same surface as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it can also be mounted on the back of the display input unit 204.

[0181] Furthermore, the camera unit 208 can utilize various functions of the smartphone 200. For example, images captured by the camera unit 208 can be displayed on the display panel 202, or images from the camera unit 208 can be used as one of the operation inputs of the operation panel 203.

[0182] Furthermore, when the GNSS receiver 214 detects its position, it can also refer to the image from the camera unit 208 to detect the position. Moreover, it can refer to the image from the camera unit 208 to determine the optical axis direction of the smartphone 200's camera unit 208, or to determine the current operating environment, without using a 3-axis accelerometer, or in conjunction with a 3-axis accelerometer. Of course, the image from the camera unit 208 can also be used within the application software.

[0183] In addition, location information acquired by the GNSS receiver 214, voice information acquired by the microphone 206 (which can also be converted into text information by the main control unit, etc.), and posture information acquired by the motion sensor 215 can be added to the image data of static or dynamic images and stored in the storage unit 212, or output through the external input / output unit 213 or the wireless communication unit 210.

[0184] The various embodiments have been described above, but the present invention is not limited to these examples, which is obvious. Those skilled in the art will readily conceive of various modifications or alterations within the scope of the claims, and will understand that these also fall within the technical scope of the present invention. Furthermore, the constituent elements of the above embodiments can be combined arbitrarily without departing from the spirit of the invention.

[0185] Furthermore, this application is based on the contents of the Japanese patent application filed on September 29, 2023 (Japanese Patent Application 2023-169880), which are incorporated herein by reference.

[0186] Symbol Explanation 1-Imaging lens; A1, A2, A3, A4, A5, A6-Range; 4-Lens control unit; 5-Imaging element; 8-Lens drive unit; 9-Drive unit; 11-System control unit; 14, 207-Operation unit; 15-Memory control unit; 16-Memory; 17-Digital signal processing unit; 20-External memory control unit; 21-Storage medium; 22-Display device; 22a-Display controller; 22b-Display surface; 24-Control bus; 25-Data bus; 40-Lens assembly; 60-Imaging surface; 61-Pixel; 61B, 61G, 61R-Ordinary pixel; 61FAg, 61FBg, 61FAw, 61FBw-Phase pixel; 61FAc, 61FBc, 61FAy. 61FBy - Phase difference pixel, 62 - Pixel group, 63 - Driving circuit, 64 - Signal processing circuit, 100 - Digital camera, 100A - Main body, 200 - Smartphone, 201 - Frame, 202 - Display panel, 203 - Operation panel, 204 - Display input unit, 205 - Speaker, 206 - Microphone, 208 - Camera unit, 210 - Wireless communication unit, 211 - Talking unit, 212 - Storage unit, 213 - External input / output unit, 214 - GNSS receiver, 215 - Motion sensor unit, 216 - Power supply unit, 217 - Internal storage unit, 218 - External storage unit, 220 - Main control unit, P1 - Phase difference pixel, P2 - Ordinary pixel, G1, G2, G3, G4 - Pixel group.

Claims

1. An imaging element having multiple pixels, The plurality of pixels includes a first pixel used for phase difference detection and a second pixel that is different from the first pixel. The second pixel includes multiple pixels corresponding to multiple wavelength bands. The first pixel includes a first type pixel corresponding to at least one of the plurality of wavelength bands and a second type pixel that is different from the first type pixel and corresponds to a plurality of the plurality of wavelength bands.

2. The imaging element according to claim 1, wherein, The first type of pixel corresponds to any one of the plurality of wavelength bands. The second type of pixel corresponds to the plurality of wavelength bands.

3. The imaging element according to claim 2, wherein, The plurality of wavelength bands includes a first wavelength band, a second wavelength band, and a third wavelength band.

4. The imaging element according to claim 3, wherein, The first wavelength band is the red wavelength band. The second wavelength band is the green wavelength band. The third wavelength band is the blue wavelength band. The first type of pixel corresponds to the second wavelength band.

5. The imaging element according to claim 1, wherein, The first type of pixel corresponds to multiple wavelength bands.

6. The imaging element according to claim 5, wherein, The plurality of wavelength bands includes a first wavelength band, a second wavelength band, and a third wavelength band. The first type of pixel corresponds to the second wavelength band and the third wavelength band. The second type of pixel corresponds to the first wavelength band and the second wavelength band.

7. The imaging element according to claim 6, wherein, The second pixel is arranged according to a predetermined pattern. The first type of pixel is set in a portion of the configuration position of the second pixel corresponding to the second wavelength band based on the pattern and a portion of the configuration position of the second pixel corresponding to the third wavelength band based on the pattern.

8. The imaging element according to claim 7, wherein, The second type of pixel is set in a portion of the configuration position of the second pixel corresponding to the first wavelength band based on the pattern and a portion of the configuration position of the second pixel corresponding to the second wavelength band based on the pattern.

9. The imaging element according to claim 1, wherein, An imaging surface is provided in which a plurality of pixel groups, each consisting of a plurality of pixels arranged in the first direction, are arranged in a second direction intersecting the first direction. The first type of pixel and the second type of pixel are contained in different groups of pixels.

10. The imaging element according to claim 9, wherein, The first pixel is a pixel capable of detecting the phase difference in the first direction.

11. The imaging element according to claim 2, wherein, The second pixel is arranged according to a predetermined pattern. The first pixel is set in a portion of the configuration position of the second pixel based on the pattern. A filter is provided at the periphery of the second type of pixel, which transmits the wavelength band of the second pixel corresponding to the configuration position of the second type of pixel according to the pattern.

12. The imaging element according to claim 11, wherein, An imaging surface is provided in which a plurality of pixel groups, each consisting of a plurality of pixels arranged in the first direction, are arranged in a second direction intersecting the first direction. The width of the filter in the first direction at the end of the second type of pixel in the first direction and the width of the filter in the second direction at the end of the second type of pixel in the second direction are different depending on the position of the imaging surface of the second type of pixel.

13. The imaging element according to claim 5, wherein, The second pixel is arranged according to a predetermined pattern. The first pixel is set in a portion of the configuration position of the second pixel based on the pattern. A filter is provided at the periphery of the first pixel, which transmits the corresponding wavelength band of the second pixel according to the pattern and the configuration position of the first pixel.

14. The imaging element according to claim 13, wherein, An imaging surface is provided in which a plurality of pixel groups, each consisting of a plurality of pixels arranged in the first direction, are arranged in a second direction intersecting the first direction. The width of the filter in the first direction at the end of the first pixel in the first direction and the width of the filter in the second direction at the end of the first pixel in the second direction are different depending on the position of the imaging surface of the first pixel.

15. The imaging element according to claim 1, wherein, The multiple wavelength bands are included in the wavelength bands of visible light.

16. A camera device comprising: The imaging element according to any one of claims 1 to 15; and The processor performs focus detection processing based on the pixel value of the first pixel.

17. The camera device according to claim 16, wherein, The processor controls the exposure time to be different for the first type of pixel and the second type of pixel.

18. A method for controlling an imaging element, wherein, The imaging element includes a plurality of pixels, including a first pixel for phase difference detection and a second pixel different from the first pixel. The second pixel includes multiple pixels corresponding to multiple wavelength bands. The first pixel includes a first type of pixel corresponding to at least one of the multiple wavelength bands and a second type of pixel different from the first type of pixel and corresponding to multiple wavelength bands. The control method includes the following steps: controlling the exposure time to be different for the first type of pixel and the second type of pixel.

19. A control program for an imaging element, wherein, The imaging element includes a plurality of pixels, including a first pixel for phase difference detection and a second pixel different from the first pixel. The second pixel includes multiple pixels corresponding to multiple wavelength bands. The first pixel includes a first type of pixel corresponding to at least one of the multiple wavelength bands and a second type of pixel different from the first type of pixel and corresponding to multiple wavelength bands. The control program causes the processor to perform the following steps: control the exposure time to be different for the first type of pixel and the second type of pixel.

Images