Imaging element and imaging device

Abstract

Provided are an imaging element and an imaging device.　The imaging element comprises: first pixels that include a plurality of types of pixels associated with different wavelength regions; and second pixels corresponding to a narrower wavelength region than the first pixels. The plurality of types of pixels include a first-type pixel and a second-type pixel corresponding to a wavelength region relatively close to the wavelength region of the second pixels. The second-type pixel is disposed in the vicinity of the second pixels.

Description

Image Sensor and Imaging Apparatus

[0001] The technology of the present disclosure relates to an image sensor and an imaging apparatus.

[0002] Patent Document 1 describes an image sensor including: a plurality of imaging pixels that output imaging signals for forming an image signal representing a subject image; and a plurality of focus detection pixels that output focus detection signals for detecting a focus adjustment state of an optical system.

[0003] Patent Document 2 describes an image sensor including: a plurality of event pixels that generate event signals based on a change in luminance of incident light; and a plurality of gradation pixels that generate luminance signals based on a light amount of incident light.

[0004] Patent Document 3 describes a camera module including: a pixel array unit having an imaging region provided with pixels for capturing an image, and a sensing region provided with pixels capable of acquiring spectral information different from at least the pixels of the imaging region; an imaging optical system that guides light from the outside to the imaging region; and a sensing optical system that guides light from the outside to the sensing region.

[0005] Japanese Unexamined Patent Application Publication No. 2009-157198, Japanese Unexamined Patent Application Publication No. 2024-085173, Japanese Unexamined Patent Application Publication No. 2019-029851

[0006] An image sensor and an imaging apparatus according to one embodiment of the technology of the present disclosure are as follows.

[0007] (1) An image sensor comprising: first pixels including a plurality of types of pixels corresponding to different wavelength ranges; and a second pixel corresponding to a narrower wavelength range than the first pixels, wherein the plurality of types of pixels include a first type pixel and a second type pixel corresponding to a wavelength range relatively closer to the wavelength range of the second pixel, and the second type pixel is disposed on at least a part of a periphery of the second pixel.

[0008] (2) The image sensor according to (1), wherein only the second type pixel among the first type pixel and the second type pixel is disposed around the second pixel.

[0009] (3) An image sensor according to (1) or (2), wherein the second type pixel includes a pixel corresponding to a wavelength range longer than the wavelength range corresponding to the second pixel, and a pixel corresponding to a wavelength range shorter than the wavelength range corresponding to the second pixel.

[0010] (4) An image sensor as described in (1), wherein the second type pixel includes a plurality of pixels with different corresponding wavelength ranges, and a pixel corresponding to the wavelength range closest to the wavelength range corresponding to the second pixel is arranged next to the second pixel in the first direction.

[0011] (5) An image sensor as described in (1), wherein the second type pixel includes a plurality of pixels with different corresponding wavelength ranges, and a pixel corresponding to the shortest wavelength range among the plurality of pixels of the second type pixel is arranged next to the second pixel in the first direction.

[0012] (6) An image sensor according to any one of (1) to (5), wherein the second type pixel includes a plurality of pixels with different corresponding wavelength ranges, and the second pixel includes a configuration in which the second type pixel is arranged next to the first direction and the first type pixel is arranged next to the second direction, and a configuration in which the second type pixel is arranged next to the second direction and the first type pixel is arranged next to the first direction.

[0013] (7) An image sensor as described in (6), wherein a second type pixel is arranged next to the first direction, and the second pixel, which is arranged next to the second direction, is located at the end of the first direction, and a second type pixel is arranged next to the second direction, and the second pixel, which is arranged next to the first direction, is located at the end of the second direction.

[0014] (8) The image sensor described in (7), wherein the second pixel is located only at the end in the first direction and the end in the second direction.

[0015] (9) An image sensor as described in (1), wherein the second pixel is surrounded by a smaller number of first type pixels than the second type pixels.

[0016] (10) An image sensor according to any one of (1) to (9), wherein the second pixel shares a conversion unit that converts electric charge into a signal with the second pixel.

[0017] (11) An image sensor according to any one of (1) to (10), wherein a first pixel row in which the first pixel and the second pixel are aligned in one direction, and a second pixel row in which only the first pixel among the first pixel and the second pixel are aligned in the one direction, are arranged in a direction that intersects in the one direction.

[0018] (12) An image sensor as described in (11), wherein the second pixel shares a conversion unit that converts electric charge into a signal with a plurality of first pixels, and in each group of the second pixel and the plurality of first pixels that share the conversion unit, the positional relationship between the second pixel and the first pixels is the same.

[0019] (13) An imaging device comprising an image sensor as described in any one of (1) to (12), and a processor, wherein the processor determines the subject light source based on the signal read from the second pixel.

[0020] (14) An imaging device comprising an image sensor as described in (11) or (12), and a processor, wherein the processor determines the subject light source based on the signal read from the first row of pixels, and processes the signal read from the second row of pixels to generate image data.

[0021] Figure 1 is a diagram showing the schematic configuration of a digital camera 100, which is an embodiment of an imaging device according to the technology of this disclosure. Figure 2 is a schematic plan view showing the schematic configuration of the image sensor 5 shown in Figure 1. Figure 3 is a schematic diagram showing an enlarged view of an example of the pixel arrangement on the imaging surface 60 of the image sensor 5 shown in Figure 2. Figure 4 is a schematic diagram showing a first modified example of the pixel arrangement shown in Figure 3. Figure 5 is a schematic diagram showing a second modified example of the pixel arrangement shown in Figure 3. Figure 6 is a schematic diagram showing a modified image sensor 5. Figure 7 is a schematic diagram showing an enlarged view of the upper and lower end regions 603 shown in Figure 6. Figure 8 is a schematic diagram showing an enlarged view of the left and right end regions 602 shown in Figure 6. Figure 9 is a diagram showing modified examples of the arrangement positions of the second pixels P2 in the upper and lower end regions 603 and left and right end regions 602 of the image sensor 5 shown in Figure 6. Figure 10 is a diagram showing an example of the arrangement of cyan pixels 61C in the upper and lower end regions 603 shown in Figure 6 when cyan pixels 61C are used as the second pixels P2. Figure 11 shows an example of the arrangement of cyan pixels 61C in the left and right edge regions 602 shown in Figure 6 when cyan pixels 61C are used as the second pixels P2. Figure 12 shows a modified example of the arrangement position of the second pixels P2 in the upper and lower edge regions 603 and left and right edge regions 602 of the image sensor 5 shown in Figure 6 when cyan pixels 61C are used as the second pixels P2. Figure 13 shows an example of an array pattern other than the Bayer pattern for the first pixels P1 arranged on the imaging surface 60. Figure 14 is a schematic diagram showing an example of the pixel arrangement on the imaging surface 60 when yellow pixels 61Y are provided as the second pixels P2 in the array pattern shown in Figure 13. Figure 15 shows an example in the image sensor 5 shown in Figure 14 in which four adjacent pixels 61 share a conversion unit. Figure 16 shows another example in the image sensor 5 shown in Figure 14 in which four adjacent pixels 61 share a conversion unit. Figure 17 is a schematic diagram showing another example of the pixel arrangement on the imaging surface 60 when a yellow pixel 61Y is provided as the second pixel P2 in the array pattern shown in Figure 13. Figure 18 is a schematic diagram showing an example in which four adjacent pixels 61 share the conversion unit 66 in the pixel arrangement shown in Figure 17. Figure 19 shows the external appearance of the smartphone 200. Figure 20 is a block diagram showing the configuration of the smartphone 200 shown in Figure 19.

[0022] Figure 1 is a diagram showing the schematic configuration of a digital camera 100, which is one embodiment of an imaging device according to the technology of the present disclosure. The digital camera 100 shown in Figure 1 comprises a lens device 40 having an imaging lens 1, an aperture 2, a lens drive unit 8 that drives the imaging lens 1, an aperture drive unit 9 that drives the aperture 2, and a lens control unit 4 that controls the lens drive unit 8 and the aperture drive unit 9, and a main body 100A.

[0023] The main unit 100A includes an image sensor 5, a system control unit 11 that provides overall control of the entire electrical control system of the digital camera 100, an operation unit 14, a display device 22, a memory 16 including RAM (Random Access Memory) and ROM (Read-only memory), a memory control unit 15 that controls data storage in the memory 16 and data reading from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 that controls data storage in the storage medium 21 and data reading from the storage medium 21.

[0024] The lens device 40 may be detachable from the main body 100A, or it may be integrated with the main body 100A.

[0025] The lens control unit 4 of the lens device 40 controls the lens drive unit 8 based on the lens drive signal transmitted from the system control unit 11 to change the focus lens position. The lens control unit 4 of the lens device 40 controls the aperture drive unit 9 based on the drive control signal transmitted from the system control unit 11 to change the aperture amount of the aperture 2.

[0026] The image sensor 5 captures an image of the subject through an imaging optical system placed between it and the subject. The image sensor 5 has an imaging surface 60 (see Figure 2) in which multiple pixels are arranged in two dimensions, and the imaging optical system converts the image of the subject formed on this imaging surface 60 into an image signal using these multiple pixels and outputs it. The output of the pixels included in the image sensor 5 is referred to as a pixel signal, and the collection of pixel signals is referred to as an image signal.

[0027] The image sensor 5 may be, for example, a CMOS (complete metal-oxide semiconductor) image sensor or a CCD (charge coupled device) image sensor. The following example describes a case where the image sensor 5 is a CMOS image sensor.

[0028] The system control unit 11 provides overall control for the digital camera 100, and its hardware structure consists of various processors that execute programs and perform processing. The programs executed by the system control unit 11 are stored in the ROM (non-temporary storage medium) of the memory 16.

[0029] The system control unit 11 drives the image sensor 5 and the lens device 40, and outputs the subject image captured through the imaging optical system of the lens device 40 as an image signal. The image signal output from the image sensor 5 is processed by the digital signal processing unit 17 to generate image data that is suitable for display on the display device 22 or suitable for storage on the storage medium 21.

[0030] The system control unit 11 receives instruction signals from the user through the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b, as well as various buttons and the like.

[0031] The display device 22 comprises a display surface 22b composed of an organic EL (electroluminescence) panel or a liquid crystal panel, and a display controller 22a that controls the display on the display surface 22b.

[0032] The memory control unit 15, the digital signal processing unit 17, the external memory control unit 20, and the display controller 22a are interconnected by a control bus 24 and a data bus 25, and are controlled by commands from the system control unit 11.

[0033] Figure 2 is a schematic plan view showing the general configuration of the image sensor 5 shown in Figure 1. The image sensor 5 comprises an imaging surface 60 in which multiple pixel rows 62, each composed of multiple pixels 61 arranged in the row direction X, are arranged in the column direction Y intersecting the row direction X; a drive circuit 63 for driving the pixels 61 arranged on the imaging surface 60; and a signal processing circuit 64 for processing the pixel signals read out from each pixel 61 of the pixel rows 62 arranged on the imaging surface 60 to the signal lines. Each pixel 61 includes a photoelectric conversion unit such as a photodiode and a spectral filter provided above the photoelectric conversion unit.

[0034] In the example in Figure 2, the row direction X and the column direction Y are orthogonal. One direction in the row direction X is denoted as the right direction XR, and the other direction in the row direction X is denoted as the left direction XL. One direction in the column direction Y is denoted as the upward direction YU, and the other direction in the column direction Y is denoted as the downward direction YD. One direction in the column direction Y and one direction in the row direction X constitute the first direction, and one direction in the column direction Y and the other direction in the row direction X constitute the second direction.

[0035] The multiple pixels 61 arranged on the imaging surface 60 include a first pixel P1 which contains multiple types of pixels with different corresponding wavelength ranges, and a second pixel P2 which corresponds to a narrower wavelength range than the first pixel P1.

[0036] A pixel 61 corresponding to an arbitrary wavelength range receives at least a portion of that wavelength range and outputs a pixel signal corresponding to the amount of light received. The wavelength range corresponding to a pixel 61 is determined, for example, by the characteristics of the spectral filter included in that pixel 61.

[0037] The comparison of the widths of the two wavelength ranges is performed, for example, using the wavelength range corresponding to 50% reflectance in a graph showing the spectral reflectance characteristics of the spectral filter of pixel 61 (a graph with wavelength on the horizontal axis and reflectance on the vertical axis). If there is a pixel 61 corresponding to a first wavelength range and a pixel 61 corresponding to a second wavelength range which is narrower than the first wavelength range, the wavelength range corresponding to 50% reflectance of the spectral filter of the pixel 61 corresponding to the second wavelength range is smaller than the wavelength range corresponding to 50% reflectance of the spectral filter of the pixel 61 corresponding to the first wavelength range.

[0038] The plurality of types of pixels included in the first pixel P1 include a first-type pixel and a second-type pixel corresponding to a wavelength range relatively closer to the wavelength range of the second pixel P2 than the first-type pixel. The proximity of the wavelength ranges respectively corresponding to the two pixels 61 is compared at the wavelength corresponding to the maximum sensitivity value in the spectral sensitivity characteristic of the pixels 61.

[0039] FIG. 3 is an enlarged schematic diagram showing an example of a pixel arrangement on an imaging surface 60 of the image sensor 5 shown in FIG. 2. In the example shown in FIG. 3, the imaging surface 60 is provided with three types of first pixels P1 and one type of second pixel P2.

[0040] The first pixel P1 includes three types: a blue pixel 61B corresponding to a blue wavelength range (the block marked with "B" in the figure), a green pixel 61G corresponding to a green wavelength range (the block marked with "G" in the figure), and a red pixel 61R corresponding to a red wavelength range (the block marked with "R" in the figure).

[0041] The second pixel P2 is provided as one type, which is a yellow pixel 61Y corresponding to a yellow wavelength range (the block marked with "Y" in the figure).

[0042] The green wavelength range is a shorter-wavelength wavelength range than the yellow wavelength range. The red wavelength range is a longer-wavelength wavelength range than the yellow wavelength range.

[0043] The wavelength corresponding to the maximum sensitivity value of each pixel 61 increases in length in the order of blue pixel 61B, green pixel 61G, yellow pixel 61Y, and red pixel 61R from the shorter-wavelength side.

[0044] Among the first pixels P1, the respectively corresponding wavelength ranges of the green pixel 61G and the red pixel 61R are closer to the wavelength range corresponding to the yellow pixel 61Y than the wavelength range corresponding to the blue pixel 61B. Therefore, the red pixel 61R and the green pixel 61G are each a second-type pixel, and the blue pixel 61B is the first-type pixel.

[0045] On the imaging surface 60, red pixels 61R, green pixels 61G, and blue pixels 61B are arranged based on a predetermined pattern (a Bayer pattern in the example shown in the drawing). That is, on the imaging surface 60, an RG pixel group in which green pixels 61G and red pixels 61R are alternately arranged in the row direction X, and a GB pixel group in which blue pixels 61B and green pixels 61G are alternately arranged in the row direction X are alternately arranged in the column direction Y. The arrangement pattern of the first pixels P1 disposed on the imaging surface 60 is not limited to the Bayer pattern, as long as it is a pattern capable of capturing a color image.

[0046] The second pixels P2 are provided at part of the arrangement positions of the first pixels P1 based on the Bayer pattern. Specifically, in some GB pixel groups among the plurality of pixel groups provided on the imaging surface 60, yellow pixels 61Y are disposed at part of the arrangement positions of blue pixels 61B based on the Bayer pattern.

[0047] The first pixels P1 and the second pixels P2 have different usage purposes for pixel signals output from each of them. The main purpose of the first pixels P1 is to capture a subject image as a color image. The main purpose of the second pixels P2 is to improve the determination accuracy of a captured subject light source. For example, the system control unit 11 processes the pixel signals of the first pixels P1 among the pixel signals output from the image sensor 5 to generate image data, and determines the subject light source based on the pixel signals output from the second pixels P2.

[0048] In order to improve the determination accuracy of the subject light source, the corresponding wavelength range of the second pixels P2 is narrower than that of the first pixels P1. For this reason, the influence of color mixing caused by light outside the corresponding wavelength range incident on the second pixels P2 is greater than that on the first pixels P1. In order to reduce the influence of color mixing in the second pixels P2, it is preferable to arrange the first pixels P1, which correspond to a wavelength range close to the wavelength range corresponding to the second pixels P2, near the second pixels P2.

[0049] In the example of FIG. 3, around the second pixels P2 (yellow pixels 61Y), red pixels 61R and green pixels 61G whose corresponding wavelength ranges are relatively closer to the yellow pixels 61Y than the blue pixels 61B are arranged, and blue pixels 61B whose corresponding wavelength range is farthest from the yellow pixels 61Y are not arranged.

[0050] In other words, around the second pixel P2 (yellow pixel 61Y), only the second type pixels (green pixels 61G and red pixels 61R) of the first pixel P1 are arranged, out of the first type pixels (blue pixels 61B) and second type pixels (green pixels 61G and red pixels 61R).

[0051] The area surrounding the second pixel P2 refers to the positions of the eight pixels 61 located to the right, left, above, below, upper right, lower right, upper left, and lower left of the second pixel P2.

[0052] In this way, by arranging second type pixels (red pixels 61R and green pixels 61G) that detect wavelengths close to the wavelength detected by the second pixel P2 (yellow pixel 61Y) around the second pixel P2 (yellow pixel 61Y), the effect of color mixing in the second pixel P2 (yellow pixel 61Y) can be reduced.

[0053] Furthermore, because the area surrounding the second pixel P2 (yellow pixel 61Y) is entirely composed of second-type pixels, the effects of color mixing from light coming from any direction can be reduced. As a result, the accuracy of determining the subject light source using the pixel signal of the second pixel P2 (yellow pixel 61Y) can be improved.

[0054] Furthermore, in the configuration example shown in Figure 3, a first pixel row is arranged in the row direction X, where the first pixel P1 and the second pixel P2 are aligned in one direction, and a second pixel row is arranged in the column direction Y, where only the first pixel P1 of the first pixel P1 and the second pixel P2 are aligned in the row direction X.

[0055] With this configuration, the system control unit 11 can process the pixel signals read from the second pixel row to generate image data for display and recording, and can perform subject light source determination processing based on the pixel signals read from the first pixel row. Since the pixel signals of the second pixel P2 are not included when generating image data, pixel interpolation processing is unnecessary, and the quality of the image data can be improved. In addition, by making the number of first pixel rows less than the number of second pixel rows, the signals of the first pixel row can be read faster than those of the second pixel row. Therefore, subject light source determination processing can be performed at high speed. Furthermore, the exposure can be changed between the first and second pixel rows. For example, by specifically adjusting the exposure of the second pixel P2, the accuracy of subject light source determination can be improved.

[0056] Figure 4 is a schematic diagram showing a first modified example of the pixel arrangement shown in Figure 3. In the modified example shown in Figure 4, a cyan pixel 61C corresponding to the cyan wavelength range is provided as the second pixel P2, instead of the yellow pixel 61Y.

[0057] The wavelength range for blue is shorter wavelengths than the wavelength range for cyan. The wavelength range for green is longer wavelengths than the wavelength range for cyan.

[0058] In the modified example shown in Figure 4, the wavelengths corresponding to the maximum sensitivity of each pixel 61 increase in length from the shortest wavelength side, in the order of blue pixel 61B, cyan pixel 61C, green pixel 61G, and red pixel 61R.

[0059] Of the first pixels P1, the blue pixels 61B and green pixels 61G each have wavelength ranges that are closer to the wavelength range corresponding to the cyan pixel 61C than to the wavelength range corresponding to the red pixel 61R. Therefore, in the modified example shown in Figure 4, the blue pixels 61B and green pixels 61G are second-type pixels, and the red pixel 61R is a first-type pixel.

[0060] In the example shown in Figure 4, the second pixel P2, a cyan pixel 61C, is located in a portion of the position of the first pixel P1 based on the Bayer pattern. Specifically, in a portion of the RG pixel group in a group of multiple pixels provided on the imaging surface 60, the cyan pixel 61C is located in a portion of the position of the red pixel 61R based on the Bayer pattern.

[0061] In the example shown in Figure 4, blue pixels 61B and green pixels 61G, whose corresponding wavelength ranges are relatively closer to the cyan pixel 61C than to the red pixel 61R, are arranged around the second pixel P2 (cyan pixel 61C), while the red pixel 61R, whose corresponding wavelength range is furthest from the cyan pixel 61C, is not present.

[0062] In other words, around the second pixel P2 (cyan pixel 61C), only the second type pixels (blue pixels 61B and green pixels 61G) of the first pixel P1 are arranged, out of the first type pixels (red pixels 61R) and second type pixels (blue pixels 61B and green pixels 61G).

[0063] In this way, by arranging second type pixels (blue pixels 61B and green pixels 61G) that detect wavelengths close to the wavelength detected by the second pixel P2 (cyan pixel 61C) around the second pixel P2 (cyan pixel 61C), the effect of color mixing in the second pixel P2 (cyan pixel 61C) can be reduced.

[0064] Furthermore, because the area surrounding the second pixel P2 (cyan pixel 61C) is entirely composed of second-type pixels, the effect of color mixing from light coming from any direction can be reduced. As a result, the accuracy of determining the subject light source using the pixel signal of the second pixel P2 (cyan pixel 61C) can be improved.

[0065] Figure 5 is a schematic diagram showing a second modified example of the pixel arrangement shown in Figure 3. In the modified example shown in Figure 5, in addition to the yellow pixel 61Y, a cyan pixel 61C, as described in Figure 4, is provided as the second pixel P2. In the modified example shown in Figure 5, the yellow pixel 61Y is provided in a position where only red pixels 61R and green pixels 61G are arranged around it, and the cyan pixel 61C is provided in a position where only green pixels 61G and blue pixels 61B are arranged around it. By providing multiple types of second pixels P2 in this way, the accuracy of discriminating the subject light source can be further improved.

[0066] Figure 6 is a schematic diagram showing a third modified example of the image sensor 5. The imaging surface 60 has left and right edge regions 602 and upper and lower edge regions 603, which are areas where color mixing due to oblique light is likely to occur. In the third modified example, the second pixel P2 is not arranged in the central region 601 of the imaging surface 60, excluding the left and right edge regions 602 and the upper and lower edge regions 603. In the left and right edge regions 602 and the upper and lower edge regions 603, the second pixel P2 is arranged as follows.

[0067] Figure 7 is a schematic diagram showing an enlarged view of the upper and lower end regions 603 shown in Figure 6. Figure 8 is a schematic diagram showing an enlarged view of the left and right end regions 602 shown in Figure 6. Figures 7 and 8 show an example in which one type of yellow pixel 61Y is provided as the second pixel P2.

[0068] As shown in Figure 7, in the upper and lower end regions 603, in some GB pixel groups of the multiple pixel groups provided there, yellow pixels 61Y as second pixels P2 are positioned at some of the locations where green pixels 61G are arranged based on the Bayer pattern. Therefore, red pixels 61R are positioned above and below the yellow pixels 61Y, and blue pixels 61B are positioned to the left and right of the yellow pixels 61Y.

[0069] Color mixing in the upper and lower edge regions 603 is predominantly caused by oblique light traveling from the center of the imaging surface 60 in the column direction Y. Above and below the yellow pixel 61Y, red pixels 61R are positioned, whose wavelength is closer to that of the yellow pixel 61Y than the blue pixels 61B. Therefore, the occurrence of color mixing in the yellow pixel 61Y due to light traveling in the column direction Y can be suppressed.

[0070] As shown in Figure 8, in the left and right edge regions 602, in some of the RG pixel groups in the multiple pixel groups provided there, yellow pixels 61Y as second pixels P2 are positioned in some of the locations where green pixels 61G are arranged based on the Bayer pattern. Therefore, red pixels 61R are positioned to the left and right of the yellow pixels 61Y, and blue pixels 61B are positioned above and below them.

[0071] Color mixing in the left and right edge regions 602 is predominantly caused by oblique light traveling in the row direction X from the center of the imaging surface 60. Red pixels 61R, which are closer in wavelength to the yellow pixel 61Y than the blue pixels 61B, are positioned to the left and right of the yellow pixel 61Y. Therefore, the occurrence of color mixing in the yellow pixel 61Y due to light traveling in the row direction X can be suppressed.

[0072] As shown in Figures 7 and 8, some of the relatively numerous green pixels 61G in the first pixel P1 may be replaced with yellow pixels 61Y. With this configuration, compared to the configuration in Figure 3 where some of the blue pixels 61B are replaced with green pixels 61G, the number of blue pixels 61B can be increased, and the quality of the image data can be improved.

[0073] In the modified example shown in Figure 6, the central region 601 does not have a second pixel P2. Therefore, for example, by reading pixel signals only from the central region 601 and generating image data, it is not necessary to interpolate the pixel signals, and the quality of the image data can be improved.

[0074] Furthermore, the positions of the yellow pixels 61Y in the left and right edge regions 602 and the top and bottom edge regions 603 may be part of the positions where the blue pixels 61B should be placed (positions where only green pixels 61G and red pixels 61R are placed around them), as shown in Figure 3. By doing so, color mixing in the second pixel P2 can be further suppressed, and the accuracy of determining the subject light source can be improved.

[0075] Figure 9 shows a modified example of the arrangement of the second pixel P2 in the upper and lower end regions 603 and the left and right end regions 602 of the image sensor 5 shown in Figure 6.

[0076] In the example shown in Figure 9, in some of the RG pixel groups in the multiple pixel groups provided in the left and right edge regions 602 and the top and bottom edge regions 603, yellow pixels 61Y as second pixels P2 are positioned in some of the locations where red pixels 61R are positioned based on the Bayer pattern. Therefore, green pixels 61G, whose corresponding wavelength range is closer to that of the yellow pixel 61Y than that of the blue pixel 61B, are positioned next to the yellow pixel 61Y in all directions (up, down, left, and right). As a result, color mixing can be suppressed in the yellow pixel 61Y in both the left and right edge regions 602 and the top and bottom edge regions 603. In the example shown in Figure 9, the number of green pixels 61G corresponding to the green wavelength range, which is highly correlated with the luminance component of the image, can be increased, thereby improving the quality of the image data.

[0077] The color mixing in the second pixel P2 becomes more pronounced as the corresponding wavelength range of the adjacent pixel 61 becomes longer. This is because, if the photoelectric conversion unit included in pixel 61 is, for example, a photodiode, longer wavelength light is more likely to be converted into electrons even at deeper locations within the photoelectric conversion unit.

[0078] In the image sensor 5 shown in Figure 6, as examples of the arrangement of the second pixels P2 in the left and right edge regions 602 and the top and bottom edge regions 603, either the example shown in Figures 7 and 8 (a configuration in which red pixels 61R are arranged above and below or to the left and right of the yellow pixels 61Y) or the example shown in Figure 9 (a configuration in which green pixels 61G are arranged above, below, to the left and right of the yellow pixels 61Y) can be adopted.

[0079] In the example shown in Figure 9, the wavelength range corresponding to the pixel 61 adjacent to the yellow pixel 61Y (adjacent to the direction of incident light that could cause color mixing) is on the shorter wavelength side, compared to the examples shown in Figures 7 and 8. Therefore, the effect of color mixing can be reduced compared to the examples shown in Figures 7 and 8.

[0080] The red pixel 61R and the green pixel 61G have corresponding wavelength ranges that are relatively closer to the yellow pixel 61Y than to the blue pixel 61B. However, the effect of suppressing the color mixing effect of the yellow pixel 61Y differs depending on whether the red pixel 61R or the green pixel 61G has a corresponding wavelength range that is closer to the yellow pixel 61Y.

[0081] For example, consider a case where the wavelength range corresponding to the red pixel 61R is closer to the wavelength range corresponding to the yellow pixel 61Y than the wavelength range corresponding to the green pixel 61G. This case is when, among the second type pixels, the red pixel 61R and the green pixel 61G, the pixel whose corresponding wavelength range is closest to the second type pixel P2 is the red pixel 61R. In this case, as shown in the examples in Figures 7 and 8, by adopting a configuration in which the red pixel 61R is adjacent to the yellow pixel 61Y, the effect of color mixing can be reduced compared to the example shown in Figure 9.

[0082] Furthermore, we consider the case where, of the wavelength ranges corresponding to the red pixel 61R and the green pixel 61G, the wavelength range corresponding to the green pixel 61G is closer to the wavelength range corresponding to the yellow pixel 61Y. In this case, of the second type pixels, the red pixel 61R and the green pixel 61G, the pixel whose corresponding wavelength range is closest to the second pixel P2 is the green pixel 61G. In this case, as shown in the example in Figure 9, by adopting a configuration in which the green pixel 61G is adjacent to the yellow pixel 61Y, the effect of color mixing can be reduced compared to the examples shown in Figures 7 and 8.

[0083] Figure 10 shows an example of the arrangement of cyan pixels 61C in the upper and lower end regions 603 shown in Figure 6, when cyan pixels 61C are used as the second pixels P2. As shown in Figure 10, in the upper and lower end regions 603, in some of the RG pixel groups in the plurality of pixel groups provided there, cyan pixels 61C as the second pixels P2 are arranged in some of the positions of green pixels 61G based on the Bayer pattern. Therefore, blue pixels 61B are arranged above and below the cyan pixels 61C, and red pixels 61R are arranged to the left and right of the cyan pixels 61C.

[0084] Color mixing in the upper and lower edge regions 603 is predominantly caused by oblique light traveling in the column direction Y from the center of the imaging surface 60. Blue pixels 61B are positioned above and below the cyan pixels 61C. Therefore, the occurrence of color mixing in the cyan pixels 61C due to light traveling in the column direction Y can be suppressed.

[0085] Figure 11 shows an example of the arrangement of cyan pixels 61C in the left and right edge regions 602 shown in Figure 6, when cyan pixels 61C are used as the second pixels P2. As shown in Figure 11, in the left and right edge regions 602, in some GB pixel groups of the multiple pixel groups provided there, cyan pixels 61C as the second pixels P2 are arranged in some of the positions of green pixels 61G based on the Bayer pattern. Therefore, blue pixels 61B are arranged to the left and right of the cyan pixels 61C, and red pixels 61R are arranged to the top and bottom of the cyan pixels 61C.

[0086] Color mixing in the left and right edge regions 602 is predominantly caused by oblique light traveling in the row direction X from the center of the imaging surface 60. Blue pixels 61B are positioned to the left and right of the cyan pixels 61C. Therefore, the occurrence of color mixing due to light traveling in the row direction X can be suppressed in the cyan pixels 61C.

[0087] As shown in Figures 10 and 11, in a configuration in which a portion of the relatively numerous green pixels 61G in the first pixel P1 are replaced with cyan pixels 61C, the number of red pixels 61R can be increased compared to a configuration in Figure 4 in which a portion of the red pixels 61R are replaced with green pixels 61G, thereby improving the quality of the image data.

[0088] Figure 12 shows a modified example of the arrangement position of the second pixel P2 in the upper and lower end regions 603 and the left and right end regions 602 of the image sensor 5 shown in Figure 6, when a cyan pixel 61C is used as the second pixel P2.

[0089] In the example shown in Figure 12, in some of the GB pixel groups in the multiple pixel groups provided in the left and right edge regions 602 and the top and bottom edge regions 603, cyan pixels 61C as second pixels P2 are positioned in some of the locations where blue pixels 61B are positioned based on the Bayer pattern. Therefore, green pixels 61G are positioned adjacent to the cyan pixels 61C in the top, bottom, left, and right directions.

[0090] As a result, color mixing can be suppressed in the cyan pixels 61C in both the left and right edge regions 602 and the top and bottom edge regions 603. In the example shown in Figure 12, the number of green pixels 61G corresponding to the green wavelength range, which is highly correlated with the luminance component of the image, can be increased, thereby improving the quality of the image data.

[0091] In the image sensor 5 shown in Figure 6, as an example of the arrangement of the second pixels P2 in the left and right edge regions 602 and the top and bottom edge regions 603, either the example shown in Figures 10 and 11 (a configuration in which blue pixels 61B are arranged above and below or to the left and right of the cyan pixels 61C) or the example shown in Figure 12 (a configuration in which green pixels 61G are arranged above, below, to the left and right of the cyan pixels 61C) can be adopted.

[0092] In the examples shown in Figures 10 and 11, the wavelength range corresponding to the pixel 61 adjacent to the cyan pixel 61C that affects color mixing is on the shorter wavelength side, compared to the example shown in Figure 12. Therefore, the effect of color mixing can be reduced compared to the example shown in Figure 12.

[0093] The blue pixel 61B and the green pixel 61G have corresponding wavelength ranges that are relatively closer to the cyan pixel 61C than to the red pixel 61R. However, the effect of suppressing color mixing in the cyan pixel 61C differs depending on whether the corresponding wavelength ranges of the blue pixel 61B or the green pixel 61G are closer to those of the cyan pixel 61C.

[0094] For example, consider a case where, among the wavelength ranges corresponding to the blue pixel 61B and the wavelength ranges corresponding to the green pixel 61G, the wavelength range corresponding to the blue pixel 61B is closer to the wavelength range corresponding to the cyan pixel 61C. This case is when, among the second type pixels, the blue pixel 61B and the green pixel 61G, the pixel whose corresponding wavelength range is closest to the second pixel P2 is the blue pixel 61B.

[0095] In this case, by adopting a configuration in which a blue pixel 61B is adjacent to a cyan pixel 61C, as shown in the examples in Figures 10 and 11, the effect of color mixing can be reduced compared to the example shown in Figure 12.

[0096] Furthermore, we consider the case where, among the wavelength ranges corresponding to the blue pixel 61B and the wavelength ranges corresponding to the green pixel 61G, the wavelength range corresponding to the green pixel 61G is closer to the wavelength range corresponding to the cyan pixel 61C. In this case, of the second type pixels, the blue pixel 61B and the green pixel 61G, the pixel whose corresponding wavelength range is closest to the second pixel P2 is the green pixel 61G.

[0097] In this case, as shown in the example in Figure 12, by adopting a configuration in which a green pixel 61G is adjacent to a cyan pixel 61C, the effect of color mixing can be reduced compared to the examples shown in Figures 10 and 11.

[0098] Figure 13 shows an example of an arrangement pattern other than the Bayer pattern for the first pixels P1 arranged on the imaging surface 60. In the example shown in Figure 13, the imaging surface 60 is provided with a first pixel group GR1, which consists of nine pixels 61, each consisting of two red pixels 61R, five green pixels 61G, and two blue pixels 61B, arranged in three groups in the column direction Y and three groups in the row direction X, and a second pixel group GR2, which is the same as the first pixel group GR1 but with the positions of the red pixels 61R and blue pixels 61B reversed, and these are arranged in a checkerboard pattern.

[0099] The first pixel group GR1 consists of nine pixels 61 arranged in a 3x3 grid. A green pixel 61G is positioned in the center, with other green pixels 61G positioned to its right, lower right, lower, and upper left. A blue pixel 61B is positioned above the central green pixel 61G, and a red pixel 61R is positioned to the right of this blue pixel 61B. A red pixel 61R is positioned to the left of the central green pixel 61G, and a blue pixel 61B is positioned below this red pixel 61R.

[0100] Even with the pixel arrangement shown in Figure 13, color image data can be generated. An example of the arrangement of the second pixel P2 when the first pixel P1 is arranged according to such an arrangement pattern will be explained with reference to Figure 14. Figure 14 is a schematic diagram showing an example of the pixel arrangement of the imaging surface 60 when a yellow pixel 61Y is provided as the second pixel P2.

[0101] In the array pattern shown in Figure 13, one of the four green pixels 61G in a portion of the first pixel group GR1—the central green pixel 61G and the green pixels 61G to its right, lower right, and lower—can be replaced with a yellow pixel 61Y.

[0102] In the example shown in Figure 14, in a portion of the first pixel group GR1, the green pixel 61G adjacent to the central green pixel 61G is replaced with a yellow pixel 61Y. In this configuration, of the eight first pixels P1 surrounding the yellow pixel 61Y, two become blue pixels 61B (first-type pixels), and the remaining two become red pixels 61R and green pixels 61G (second-type pixels). In other words, there are more second-type pixels than first-type pixels surrounding the yellow pixel 61Y. Therefore, the effect of color mixing in the yellow pixel 61Y can be reduced.

[0103] Here, we have described the case where a yellow pixel 61Y is provided as the second pixel P2. However, when a cyan pixel 61C is provided, the influence of color mixing in the cyan pixel 61C can be reduced by providing the cyan pixel 61C in the same position as the yellow pixel 61Y.

[0104] Each pixel 61 is equipped with a conversion unit that converts the charge accumulated in the photoelectric conversion unit into a voltage signal. The charge generated in the photoelectric conversion unit is sent to a floating diffusion unit, which acts as a conversion unit, and converted into a voltage signal. This voltage signal is read out as a pixel signal on a signal line by a circuit combining transistors. The conversion unit provided in each pixel 61 may be provided individually for each pixel 61, but it may also be shared by multiple pixels 61.

[0105] Figure 15 shows an example in which four adjacent pixels 61 share a conversion unit in the image sensor 5 shown in Figure 14. In the example in Figure 15, in the first pixel group GR1, the conversion unit 66 is shared by a total of four green pixels 61G: the central green pixel 61G and the green pixels 61G to its right, lower right, and lower.

[0106] In the example shown in Figure 15, the yellow pixel 61Y shares the conversion unit 66 only with the green pixel 61G, which is a Type 2 pixel with a relatively close wavelength range. When multiple pixels 61 share the conversion unit 66, color mixing between these multiple pixels 61 may easily occur due to constraints on the layout of the photoelectric conversion unit and wiring. However, with the configuration shown in Figure 15, even if color mixing occurs in the yellow pixel 61Y, it occurs with the green pixel 61G, which corresponds to a wavelength range close to that of the yellow pixel 61Y, thus reducing the impact of color mixing in the yellow pixel 61Y.

[0107] For example, as shown in Figure 16, the conversion unit 66 can be shared between the yellow pixel 61Y and the adjacent pixels 61 to its right, lower right, and lower. In the example in Figure 16, the yellow pixel 61Y also shares the conversion unit 66 with the blue pixel 61B. Therefore, compared to the example in Figure 15, the effect of reducing the influence of color mixing in the yellow pixel 61Y is weakened.

[0108] As mentioned above, the yellow pixel 61Y can be located in any of the four green pixels 61G in the lower right of the first pixel group GR1. In the example in Figure 14, the position of the yellow pixel 61Y is the same in each of the multiple first pixel groups GR1 in which the yellow pixel 61Y is located. However, the position of the yellow pixel 61Y may differ for each first pixel group GR1. For example, the configuration shown in Figure 17 can also be adopted.

[0109] In the configuration shown in Figure 17, if the conversion unit 66 is shared as shown in Figure 15, the result will be as shown in Figure 18. In the example shown in Figure 18, there are yellow pixels 61Y that are in different positions relative to the conversion unit 66. Therefore, it takes time to read out the pixel signal from the yellow pixels 61Y. In contrast, in the configurations shown in Figures 15 and 16, the positional relationship between the yellow pixels 61Y and the other first pixels P1 is the same for the four pixels 61 that share the conversion unit 66. Therefore, the reading out of the pixel signal from the yellow pixels 61Y can be done faster than in the example in Figure 18.

[0110] Next, we will describe the configuration of a smartphone, which is another embodiment of the imaging device related to the technology of this disclosure.

[0111] Figure 19 shows the external appearance of the smartphone 200. The smartphone 200 shown in Figure 19 has a flat casing 201, and one side of the casing 201 is equipped with a display input unit 204 which is an integrated display panel 202 as a display unit and an operation panel 203 as an input unit.

[0112] Furthermore, such a housing 201 includes a speaker 205, a microphone 206, an operating unit 207, and a camera unit 208. However, the configuration of the housing 201 is not limited to this; for example, a configuration in which the display unit and input unit are independent, or a configuration having a folding structure or a sliding mechanism, can also be adopted.

[0113] Figure 20 is a block diagram showing the configuration of the smartphone 200 shown in Figure 19.

[0114] As shown in Figure 20, the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiving unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0115] Furthermore, the smartphone 200 has a primary function of providing wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0116] The wireless communication unit 210 performs wireless communication with base station equipment BS connected to the mobile communication network NW, in accordance with instructions from the main control unit 220. This wireless communication is used to send and receive various file data such as voice data and image data, email data, etc., and to receive web data or streaming data, etc.

[0117] The display input unit 204 is a so-called touch panel that, under the control of the main control unit 220, displays images (still images and moving images) or text information to visually convey information to the user and detects user operations on the displayed information, and comprises a display panel 202 and an operation panel 203.

[0118] The display panel 202 uses an LCD (Liquid Crystal Display), an OLED (Organic Electro-Luminescence Display), or the like as a display device.

[0119] The operation panel 203 is a device that detects one or more coordinates operated by the user's finger or stylus, and is positioned so as to be visible on the display surface of the display panel 202. When this device is operated by the user's finger or stylus, it outputs a detection signal generated by the operation to the main control unit 220. The main control unit 220 then detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0120] As shown in Figure 20, the display panel 202 and the operation panel 203 of the smartphone 200, which is illustrated as one embodiment of the imaging device of the present invention, are integrated to form a display input unit 204, but the operation panel 203 is positioned to completely cover the display panel 202.

[0121] When such an arrangement is adopted, the operation panel 203 may also have a function to detect user operations in areas outside the display panel 202. In other words, the operation panel 203 may have a detection area for the overlapping portion that overlaps with the display panel 202 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap with the display panel 202 (hereinafter referred to as the non-display area).

[0122] The size of the display area and the size of the display panel 202 may be made to match perfectly, but it is not necessary for them to match. Furthermore, the operation panel 203 may have two sensitive areas: an outer edge portion and an inner portion. The width of the outer edge portion is designed appropriately according to the size of the housing 201, etc.

[0123] Furthermore, the position detection methods used in the control panel 203 include matrix switch methods, resistive film methods, surface acoustic wave methods, infrared methods, electromagnetic induction methods, and capacitive methods, and any of these methods can be adopted.

[0124] The communication unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into audio data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes audio data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0125] Furthermore, as shown in Figure 19, for example, the speaker 205 can be mounted on the same side as the display input unit 204, and the microphone 206 can be mounted on the side of the housing 201.

[0126] The operation unit 207 is a hardware key using a key switch or the like, which receives instructions from the user. For example, as shown in Figure 19, the operation unit 207 is mounted on the side of the casing 201 of the smartphone 200 and is a push-button type switch that turns on when pressed with a finger or the like, and turns off when the finger is released due to a restoring force such as a spring.

[0127] The memory unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with the name or telephone number of the communication partner, sent and received email data, web data downloaded through web browsing, downloaded content data, and also temporarily stores streaming data. The memory unit 212 is composed of an internal memory unit 217 built into the smartphone and an external memory unit 218 with a removable external memory slot.

[0128] The internal storage units 217 and external storage units 218 that constitute the storage unit 212 are implemented using storage media such as flash memory type, hard disk type, multimedia card micro type, card type memory (for example, MicroSD® memory), RAM (Random Access Memory), and ROM (Read Only Memory).

[0129] The external input / output unit 213 serves as an interface for all external devices connected to the smartphone 200, and is intended for direct or indirect connection to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth®, RFID (Radio Frequency Identification), Infrared Data Association (IrDA)®, UWB (Ultra Wideband)®, ZigBee®, etc.) or network (e.g., Ethernet®, Wireless LAN (Local Area Network), etc.).

[0130] External devices that can be connected to the smartphone 200 include, for example, wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card sockets, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video equipment connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video equipment, wired / wireless connected smartphones, wired / wireless connected personal computers, wired / wireless connected personal computers, earphones, etc.

[0131] The external input / output unit 213 can transmit data received from such external devices to the various internal components of the smartphone 200, or enable data from inside the smartphone 200 to be transmitted to external devices.

[0132] The GNSS receiving unit 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn in accordance with instructions from the main control unit 220, performs positioning calculation processing based on the received GNSS signals, and detects the position of the smartphone 200, consisting of latitude, longitude, and altitude. When the GNSS receiving unit 214 can acquire position information from the wireless communication unit 210 or the external input / output unit 213 (for example, wireless LAN), it can also use that position information to detect the position.

[0133] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection results are output to the main control unit 220.

[0134] The power supply unit 216 supplies power stored in a battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0135] The main control unit 220 is equipped with a microprocessor and operates according to the control program and control data stored in the memory unit 212, and comprehensively controls each part of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. In addition, the main control unit 220 is equipped with a mobile communication control function that controls each part of the communication system for voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0136] The application processing function is realized by the operation of the main control unit 220 according to the application software stored in the memory unit 212. Examples of application processing functions include an infrared communication function that controls the external input / output unit 213 to communicate data with a counterpart device, an email function that sends and receives emails, and a web browsing function that displays web pages.

[0137] Furthermore, the main control unit 220 is equipped with image processing functions, such as displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0138] The image processing function refers to the function in which the main control unit 220 decodes the image data, applies image processing to the decoded result, and displays the image on the display input unit 204.

[0139] Furthermore, the main control unit 220 performs display control for the display panel 202 and operation detection control to detect user operations through the operation unit 207 and the operation panel 203.

[0140] By executing the display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for composing an email.

[0141] A scroll bar is a software key that accepts instructions to move the display portion of an image, such as a large image that does not fit within the display area of ​​the display panel 202.

[0142] Furthermore, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the icons and input of strings into the input fields of the window through the operation panel 203, or accepts requests to scroll the displayed image through the scroll bar.

[0143] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is in the overlapping portion (display area) that overlaps with the display panel 202 or in the outer edge portion (non-display area) that does not overlap with the display panel 202, and has a touch panel control function that controls the display position of the sensitive area of ​​the operation panel 203 or the software key.

[0144] Furthermore, the main control unit 220 can detect gesture operations on the operation panel 203 and execute pre-set functions in response to the detected gesture operations.

[0145] Gesture control refers to operations that differ from traditional simple touch operations, such as drawing a path with a finger or other object, specifying multiple locations simultaneously, or combining these to draw a path from at least one of multiple locations.

[0146] The camera unit 208 includes the lens device 40, image sensor 5, and digital signal processing unit 17 shown in Figure 1.

[0147] The image data generated by the camera unit 208 can be stored in the storage unit 212 or output via the external input / output unit 213 or the wireless communication unit 210.

[0148] In the smartphone 200 shown in Figure 19, the camera unit 208 is mounted on the same side as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it may also be mounted on the back of the display input unit 204.

[0149] Furthermore, the camera unit 208 can be used for various functions of the smartphone 200. For example, images acquired by the camera unit 208 can be displayed on the display panel 202, or images from the camera unit 208 can be used as one of the inputs for the operation panel 203.

[0150] Furthermore, when the GNSS receiver 214 detects position, it can also detect position by referring to the image from the camera unit 208. Moreover, by referring to the image from the camera unit 208, it is possible to determine the optical axis direction of the camera unit 208 of the smartphone 200, or to determine the current usage environment, either without using a 3-axis accelerometer or in combination with a 3-axis accelerometer. Of course, the image from the camera unit 208 can also be used within the application software.

[0151] In addition, position information acquired by the GNSS receiver 214, audio information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., can be added to still image or video image data and stored in the storage unit 212 or output through the external input / output unit 213 or the wireless communication unit 210.

[0152] In this embodiment, each process (each control) performed by the system control unit 11 is executed on any computer. Furthermore, any computer may execute these processes using a processor, a program, or a combination thereof. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other hardware element capable of executing a program.

[0153] The processor may be composed of one or more hardware components, and the type of hardware is not limited. For example, the processor may be composed of programmable logic devices such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), dedicated circuits for executing specific processes such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). The processor also has various parts (Units) or means (Means) that execute the various processes in this embodiment. Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware components are composed of electrical circuits (circuits) and the like, which are combinations of circuit elements such as semiconductor elements.

[0154] Furthermore, this embodiment may be implemented by hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are composed of a program. The program may also be, for example, a group of program modules, each of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. The program code or code segment may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. The program code or code segment may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0155] Although various embodiments have been described above, it goes without saying that the present invention is not limited to these examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the invention.

[0156] This application is based on a Japanese patent application (Patent Application No. 2025-054339) filed on March 27, 2025, the contents of which are incorporated herein by reference.

[0157] 1 Imaging lens 2 Aperture 4 Lens control unit 5 Image sensor 8 Lens drive unit 9 Aperture drive unit 11 System control unit 14, 207 Operation unit 15 Memory control unit 16 Memory 17 Digital signal processing unit 20 External memory control unit 21 Storage medium 22 Display device 22a Display controller 22b Display surface 24 Control bus 25 Data bus 40 Lens device 50 Reflectance 60 Imaging surface 61 Pixel 61B Blue pixel 61C Cyan pixel 61G Green pixel 61R Red pixel 61Y Yellow pixel 62 Pixel row 63 Drive circuit 64 Signal processing circuit 66 Conversion unit 100 Digital camera 100A Main unit 200 Smartphone 201 Housing 202 Display panel 203 Operation panel 204 Display input unit 205 Speaker 206 Microphone 208 Camera unit 210 Wireless communication unit 211 Communication unit 212 Memory unit 213 External input / output unit 214 GNSS receiver unit 215 Motion sensor unit 216 Power supply unit 217 Internal memory unit 218 External memory unit 220 Main control unit 601 Central area 602 Left and right end areas 603 Upper and lower end areas P1 First pixel P2 Second pixel GR1 First pixel group GR2 Second pixel group

Claims

1. An image sensor comprising: a first pixel including multiple types of pixels with different corresponding wavelength ranges; and a second pixel corresponding to a narrower wavelength range than the first pixel, wherein the multiple types of pixels include a first type pixel and a second type pixel corresponding to a wavelength range relatively close to the wavelength range of the second pixel, and the second type pixels are arranged in at least a portion of the area surrounding the second pixel.

2. An image sensor according to claim 1, wherein only the second type pixels among the first type pixels and the second type pixels are arranged around the second pixel.

3. An image sensor according to claim 1, wherein the second type of pixel includes a pixel corresponding to a wavelength range longer than the wavelength range corresponding to the second pixel, and a pixel corresponding to a wavelength range shorter than the wavelength range corresponding to the second pixel.

4. An image sensor according to claim 1, wherein the second type pixel includes a plurality of pixels with different corresponding wavelength ranges, and a pixel corresponding to the wavelength range closest to the wavelength range corresponding to the second pixel is arranged next to the second pixel in the first direction among the plurality of pixels of the second type pixel.

5. An image sensor according to claim 1, wherein the second type of pixel includes a plurality of pixels with different corresponding wavelength ranges, and a pixel corresponding to the shortest wavelength range among the plurality of pixels of the second type of pixel is arranged next to the second pixel in the first direction.

6. An image sensor according to any one of claims 1 to 5, wherein the second type pixel includes a plurality of pixels with different corresponding wavelength ranges, and the second pixel includes a configuration in which the second type pixel is arranged next to a first direction and the first type pixel is arranged next to a second direction, and a configuration in which the second type pixel is arranged next to a second direction and the first type pixel is arranged next to a first direction.

7. An image sensor according to claim 6, wherein a second type pixel is arranged next to the first direction, the second pixel which is arranged next to the first type pixel is arranged at the end of the first direction, and a second type pixel which is arranged next to the second direction is arranged at the end of the second direction.

8. The image sensor according to claim 7, wherein the second pixel is located only at the end in the first direction and the end in the second direction.

9. An image sensor according to claim 1, wherein the second pixel is surrounded by a number of first type pixels less than the number of second type pixels.

10. An image sensor according to any one of claims 1 to 5, wherein the second pixel shares a conversion unit that converts electric charge into a signal with the second pixel.

11. An image sensor according to any one of claims 1 to 5, wherein a first pixel row in which the first pixels and the second pixels are aligned in one direction, and a second pixel row in which only the first pixels among the first and second pixels are aligned in the one direction, are arranged in a direction that intersects the one direction.

12. An image sensor according to claim 11, wherein the second pixel shares a conversion unit that converts electric charge into a signal with a plurality of first pixels, and in each group of the second pixel and the plurality of first pixels that share the conversion unit, the positional relationship between the second pixel and the first pixels is the same.

13. An imaging device comprising an image sensor according to any one of claims 1 to 5, and a processor, wherein the processor determines a subject light source based on a signal read from the second pixel.

14. An imaging device comprising the image sensor according to claim 11 and a processor, wherein the processor determines the subject light source based on the signal read from the first pixel row and processes the signal read from the second pixel row to generate image data.

Images