Solid-state imaging device and electronic device

Abstract

The invention provides a light detection device. The light detection device includes a pixel array having a main pixel and a sub-pixel. More than half of the primary pixels may have transparent color filters, yellow color filters, or no color filters. The sub-pixels may have color filters of different colors, such as red, blue, and green color filters. Some of the primary pixels may have a red color filter. A processor may be used to generate image data based on signals from the main pixels and the sub-pixels. Interpolation may be used to calculate an intermediate main pixel signal of a position between the main pixels and an intermediate sub-pixel signal of a position between the sub-pixels. The light detection device may be included in an automotive camera system and in a vehicle. The vehicle control system may control the vehicle based on image data generated using the light detection device.

Description

Technical Field

[0001] The present invention relates to a solid-state imaging device and an electronic device.

[0002] <Cross-reference to related applications>

[0003] This application claims the benefit of Japanese Priority Patent Application JP 2022-099409 filed on Jun. 21, 2022, which is hereby incorporated by reference herein in its entirety. Background Art

[0004] Generally, there are color filters of three colors, for example, red, green, and blue, or red, yellow, and cyan, in a solid-state imaging device (solid-state imaging element) such as an image sensor.

[0005] Reference List

[0006] Patent Literature

[0007] PTL 1: JP 2015-65270A Summary of the invention

[0008] Technical issues

[0009] In some conventional solid-state imaging devices, it is difficult to increase the sensitivity and dynamic range on the low illumination side without increasing the size of the pixel portion.

[0010] The present technology has been created in view of such circumstances, and makes it possible to increase the sensitivity and dynamic range on the low illumination side without increasing the size of the pixel portion when the pixel portion is constituted by two types of pixels having different light receiving areas.

[0011] Solution to the problem

[0012] According to aspects of the present disclosure, a light detection device is provided, comprising: a pixel array including a plurality of main pixels and a plurality of sub-pixels having color filters, wherein more than half of the plurality of main pixels have transparent color filters, yellow color filters, or no color filters.

[0013] In some embodiments, the plurality of sub-pixels include a first group of sub-pixels having a color filter of a first color, a second group of sub-pixels having a color filter of a second color, and a third group of sub-pixels having a color filter of a third color. In some embodiments, the color filter of the first color includes a red color filter. In some embodiments, the color filter of the second color includes a blue color filter and the color filter of the third color includes a green color filter. In some embodiments, the color filter of the first color includes a red color filter, the color filter of the second color includes a yellow color filter, and the color filter of the third color includes a cyan color filter. In some embodiments, the light receiving area of ​​each of the plurality of main pixels includes an octagonal shape. In some embodiments, each of the plurality of main pixels has a transparent color filter or does not have a color filter. In some embodiments, at least one of the plurality of main pixels has a red color filter. In some embodiments, the light detection device further includes a column signal line extending in a column direction and a pixel control signal line extending in a horizontal direction, wherein the plurality of main pixels and the plurality of sub-pixels are arranged in a plurality of pixel columns in the column direction and in a plurality of pixel rows in the horizontal direction. In some embodiments, at least one of the plurality of primary pixels has a red color filter. In some embodiments, at least one of the plurality of primary pixels has a red color filter, and at least one of the plurality of primary pixels has a blue color filter. In some embodiments, each of the plurality of pixel rows includes at least one pixel having a transparent color filter or having no color filter. In some embodiments, each of the plurality of pixel columns includes at least one pixel having a transparent color filter or having no color filter. In some embodiments, more than half of the primary pixels in the plurality of primary pixels have a yellow color filter, at least one of the plurality of primary pixels has a red color filter, and at least one of the plurality of primary pixels has a cyan color filter. In some embodiments, the light detection device further includes a processor configured to generate image data based on signals from the plurality of primary pixels and signals from the plurality of sub-pixels. In some embodiments, the processor is configured to calculate using interpolation: intermediate primary pixel signals of positions between primary pixels of the plurality of primary pixels and intermediate sub-pixel signals of positions between sub-pixels of the plurality of sub-pixels, and the processor is configured to generate image data based on signals from the plurality of primary pixels, signals from the plurality of sub-pixels, intermediate primary pixel signals, and intermediate sub-pixel signals.

[0014] According to aspects of the present disclosure, a vehicle is provided, comprising: a light detection device configured to generate a signal; a processor configured to generate image data based on the signal from the light detection device; and a vehicle control system configured to control the vehicle based on the image data, wherein the light detection device comprises a pixel array, the pixel array comprising a plurality of main pixels and a plurality of sub-pixels having a color filter, wherein more than half of the plurality of main pixels have a transparent color filter, a yellow color filter, or no color filter.

[0015] According to various aspects of the present disclosure, an automotive camera system is provided, comprising a lens, a light detection device, the light detection device comprising a pixel array, the pixel array comprising a plurality of main pixels and a plurality of sub-pixels having color filters, wherein more than half of the plurality of main pixels have a transparent color filter, a yellow color filter, or no color filter, and a processor configured to generate image data based on signals from the plurality of main pixels and signals from the plurality of sub-pixels.

[0016] In some embodiments, at least one of the plurality of primary pixels has a red filter. In some embodiments, the processor is configured to calculate using interpolation: an intermediate primary pixel signal at a position between primary pixels of the plurality of primary pixels and an intermediate sub-pixel signal at a position between sub-pixels of the plurality of sub-pixels, and the processor is configured to generate image data based on the signals from the plurality of primary pixels, the signals from the plurality of sub-pixels, the intermediate primary pixel signal, and the intermediate sub-pixel signal.

[0017] According to the first aspect of the present invention, a solid-state imaging device includes: a pixel array, wherein a plurality of pixel units are arranged in an array, each pixel unit is composed of a sub-pixel having a first light receiving area (area) and a main pixel having a second light receiving area larger than the first light receiving area, wherein a first color filter group composed of color filters of each of the plurality of sub-pixels arranged in the pixel array is composed of three types of color filters; at least some of the color filters of the second color filter group configured by the color filters of each of the plurality of main pixels arranged in the pixel array are broadband filters that transmit light in a band including a green light band; and the proportion of broadband filters in the second color filter group is greater than 50%.

[0018] In a first aspect of the present invention, a pixel array is provided, in which a plurality of pixel units are arranged, each pixel unit is composed of a sub-pixel having a first light receiving area and a main pixel having a second light receiving area larger than the first light receiving area, a first color filter group composed of color filters of each of the plurality of sub-pixels arranged in the pixel array is composed of three types of color filters; at least some of the color filters of the second color filter group configured by the color filters of each pixel in the plurality of main pixels arranged in the pixel array are broadband filters that transmit light in a band including a green light band; and the proportion of broadband filters in the second color filter group is greater than 50%.

[0019] An electronic device according to a second aspect of the present invention includes: a solid-state imaging element, configured to include: a pixel array, wherein a plurality of pixel portions are arranged in an array, each pixel portion is composed of a sub-pixel having a first light receiving area and a main pixel having a second light receiving area larger than the first light receiving area, a first color filter group composed of a color filter of each of the plurality of sub-pixels arranged in the pixel array is composed of three types of color filters; at least some of the color filters in the second color filter group are configured as broadband color filters that transmit light in a band including a green light band by the color filter of each of the plurality of main pixels arranged in the pixel array; the proportion of the broadband color filters in the second color filter group is greater than 50%; and a generating unit that generates an image by using a sub-pixel signal and a main pixel signal, the sub-pixel signal being a pixel signal corresponding to the light received by the sub-pixel, and the main pixel signal being a pixel signal corresponding to the light received by the main pixel.

[0020] In a second aspect of the present technology, a pixel array is provided, wherein a plurality of pixel units are arranged in an array, each pixel unit is composed of a sub-pixel having a first light receiving area and a main pixel having a second light receiving area larger than the first light receiving area, a first color filter group composed of color filters arranged in each of the plurality of sub-pixels in the pixel array is composed of three types of color filters; at least some of the color filters of the second color filter group configured by the color filters of each of the plurality of main pixels arranged in the pixel array are broadband color filters that transmit light in a band including a green light band; and the proportion of the broadband color filters in the second color filter group is greater than 50%. In addition, an image is generated using a sub-pixel signal and a main pixel signal, the sub-pixel signal being a pixel signal corresponding to light received by the sub-pixel, and the main pixel signal being a pixel signal corresponding to light received by the main pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 : is a block diagram showing an example of the configuration of a first embodiment of an imaging device as an electronic device to which the present technology is applied.

[0022] Figure 2 Shows Figure 1 An example of a circuit configuration of a solid-state imaging element in FIG.

[0023] Figure 3 is a top view showing an example of the first color filter array in the first embodiment.

[0024] Figure 4 It shows that Figure 3 Top view of an exemplary array of color filters when the array of large color filter groups in is a Bayer array.

[0025] Figure 5 is a top view showing an example of the second color filter array in the first embodiment.

[0026] Figure 6 : is a block diagram showing an example of the configuration of a second embodiment of an imaging device as an electronic device to which the present technology is applied.

[0027] Figure 7 It is shown Figure 6 A top view of an example of the configuration of a pixel array portion of a solid-state imaging element.

[0028] Figure 8 Shows Figure 2 The spacing between the pixels in the row and column directions.

[0029] Fig. 9 Shown in Figure 7 The spacing between adjacent primary pixels arranged side by side in the array direction.

[0030] Fig.10 : is a top view showing an example of an array of color filters when an array of a large color filter group is obtained by tilting the Bayer array 45 degrees to the right.

[0031] Fig.11 is a top view showing another example of the first color filter array in the second embodiment.

[0032] Fig.12 is a top view showing another example of the second color filter array in the second embodiment.

[0033] Fig.13 is a top view showing another example of the third color filter array in the second embodiment.

[0034] Fig.14 is a top view showing another example of the fourth color filter array in the second embodiment.

[0035] Fig.15 is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0036] Fig.16 2 is a diagram showing an example of the installation positions of the vehicle exterior information detection section and the imaging section. DETAILED DESCRIPTION

[0037] Hereinafter, a mode for executing the present technology (hereinafter referred to as an embodiment) will be described. Explanations are given in the following order: 1. First embodiment (imaging device in which the arrangement direction of color filters of adjacent primary pixels is in the row direction); 2. Second embodiment (imaging device in which the array direction of color filters of adjacent primary pixels is inclined 45 degrees relative to the row direction); and 3. Example applied to a moving object.

[0038] In the drawings mentioned in the following description, the same or similar parts are denoted by the same or similar reference numerals. However, the drawings are schematic and may include parts having different dimensional relationships and ratios between the drawings.

[0039] In addition, the definition of directions (such as up and down directions) in the following description is given only for the convenience of description and does not limit the technical concept of the present disclosure. For example, when the object is rotated 90° and observed, the top and bottom are read as converted to left and right, and when the object is rotated 180° and observed, the top and bottom are read as reversed.

[0040] The present disclosure relates to a solid-state imaging device and an electronic device, and more specifically, to a solid-state imaging device and an electronic device, so that when a pixel portion is composed of two types of pixels with different light receiving areas, the sensitivity and dynamic range can be increased on the low illumination side without increasing the size of the pixel portion.

[0041] A method of increasing the pixel size is used to capture a high-sensitivity image in a solid-state imaging device. However, with this method, the size of the solid-state imaging device increases without changing the number of pixels, which increases the cost of the solid-state imaging device and increases the size of a camera including the solid-state imaging device, thereby imposing restrictions on the installation position of the camera. At the same time, without changing the size of the solid-state imaging device, the number of pixels decreases and the resolution decreases.

[0042] A method of increasing the number of pixels is used to capture high-resolution images in a solid-state imaging device. However, with this method, the size of the solid-state imaging device increases without changing the size of the pixel, which increases the cost of the solid-state imaging device and increases the size of a camera including the solid-state imaging device, thereby imposing restrictions on the installation position of the camera. Meanwhile, when the size of the pixel is reduced, high resolution can be achieved while suppressing an increase in the size of the solid-state imaging device, but the sensitivity decreases.

[0043] There is a solid-state imaging device that has pixels with a large aperture and pixels with a small aperture, and in which two types of adjacent pixels have color filters of the same color (red, green, or blue) to increase the dynamic range (see, for example, PTL 1).

[0044] <1. First Embodiment>

[0045] <Configuration Example of Imaging Device>

[0046] Light detection devices are described herein.The light detection device may include an imaging device. Figure 1 is a block diagram showing an exemplary configuration of a first embodiment of an imaging device as an electronic device to which the present technology is applied.

[0047] Figure 1 The imaging device 11 in the embodiment is configured by an optical system 12, a shutter device 13, a solid-state imaging element 14, a control circuit 15, a signal processing circuit 16, a monitor 17, and a memory 18. The imaging device 11 captures an image of an object and displays or records an HDR image (which is a high dynamic range image) as a captured image.

[0048] Specifically, the optical system 12 has one or more lenses, guides light from an object to the solid-state imaging element 14 , and forms an image on a light receiving surface of the solid-state imaging element 14 .

[0049] The shutter device 13 is arranged between the optical system 12 and the solid-state imaging element 14 , and controls a period in which the solid-state imaging element 14 is irradiated with light and a period in which the light is blocked according to control of the control circuit 15 .

[0050] The solid-state imaging element 14 is, for example, a CMOS (complementary metal oxide semiconductor) image sensor. The solid-state imaging element 14 is configured by arranging a plurality of pixel units consisting of main pixels and sub-pixels having different light receiving areas in a matrix form on a semiconductor substrate using silicon (Si). The light receiving area of ​​the main pixel is larger than the light receiving area of ​​the sub-pixel. In the case where there is no need to distinguish between the main pixel and the sub-pixel, these are collectively referred to as "pixels".

[0051] Each pixel has a color filter. Light incident through the optical system 12 and the shutter device 13 is received by each pixel through the color filter. Under the control of the control circuit 15, the solid-state imaging element 14 outputs a pixel signal as a digital signal corresponding to the light received by each pixel to the signal processing circuit 16.

[0052] The control circuit 15 controls the shutter device 13 and the solid-state imaging element 14 .

[0053] The signal processing circuit 16 supplies the pixel signals provided from the solid-state imaging device 14 to the memory 18 to be stored therein. If necessary, the signal processing circuit 16 reads the pixel signals stored in the memory 18, and performs demosaicing processing by using the pixel signals to generate sub-pixel signals for each color component of the color filter having sub-pixels. Similarly, the signal processing circuit 16 generates main pixel signals that have the color components of the color filter of the main pixels with respect to the main pixels. The signal processing circuit 16 (generation unit) generates an HDR (high dynamic range) image composed of pixels corresponding to each pixel unit by using the sub-pixel signals and the main pixel signals of each pixel unit. The signal processing circuit 16 provides the HDR image to be displayed as a captured image on the display 17 or recorded on a recording medium (not shown).

[0054] The monitor 17 displays the captured image provided from the signal processing circuit 16. The memory 18 stores the pixel signals provided from the signal processing circuit 16.

[0055] As described above, each pixel unit of the solid-state imaging device 14 is composed of a main pixel and a sub-pixel having different light-receiving areas. Therefore, the signal processing circuit 16 can expand the dynamic range of the captured image by generating a captured image by using the pixel signals of the main pixels and the sub-pixels constituting each pixel unit.

[0056] <Exemplary Configuration of CMOS Image Sensor>

[0057] Figure 2 Shows Figure 1 an example of the circuit configuration of the solid-state imaging device 14 in

[0058] The solid-state imaging device 14 includes a pixel array unit 32 in which a plurality of pixel units 31 are arranged in a matrix, a vertical drive circuit 33, a column signal processing circuit 34, a horizontal drive circuit 35, an output circuit 36, a control circuit 37, an input / output terminal 38, and the like.

[0059] The pixel unit 31 is composed of a main pixel 31a and a sub-pixel 31b. The main pixel 31a and the sub-pixel 31b each have a photodiode as a photoelectric conversion element and a plurality of pixel transistors. The light-receiving area of the photodiode of the main pixel 31a is larger than the light-receiving area of the photodiode of the sub-pixel 31b.

[0060] The vertical drive circuit 33 is configured by, for example, a shift register and is connected to the pixel section 31 of each row through a pixel drive wiring 39. The vertical drive circuit 33 selects a predetermined pixel drive wiring 39 according to a clock signal or a control signal supplied from the control circuit 37, and supplies a pulse for driving the pixel section 31 to the pixel drive wiring 39, thereby driving the pixel section 31 in row units. Specifically, the vertical drive circuit 33 successively selects and scans the pixel section 31 of the pixel array section 32 in the vertical direction in row units. Therefore, a pixel signal based on a signal charge generated according to the amount of received light in the photodiode of each main pixel 31a and each sub-pixel 31b of the pixel section 31 is output to the column signal processing circuit 34 in row units through a separate vertical signal line 40.

[0061] The column signal processing circuit 34 is provided for each column of the pixel section 31, and is connected to each of the main pixel 31a and the sub-pixel 31b of the pixel section 31 of the column corresponding thereto through the vertical signal line 40. The column signal processing circuit 34 performs signal processing such as noise removal on the pixel signal input from each of the main pixel 31a and the sub-pixel 31b of the pixel section 31 of the column corresponding thereto through the vertical signal line 40, according to the clock signal or the control signal supplied from the control circuit 37. Examples of such signal processing include CDS (correlated double sampling) for removing fixed pattern noise inherent to pixels, AD (analog-to-digital) conversion, and the like.

[0062] The horizontal driving circuit 35 is composed of, for example, a shift register. The horizontal driving circuit 35 continuously outputs a horizontal scanning pulse to each column signal processing circuit 34 according to a clock signal or a control signal provided from the control circuit 37. Therefore, the horizontal driving circuit 35 sequentially selects each column signal processing circuit 34 and outputs the pixel signals of the main pixel 31a and the sub-pixel 31b from each column signal processing circuit 34 to the corresponding horizontal signal line 41.

[0063] The output circuit 36 ​​performs signal processing on each pixel signal of the main pixel 31a and the sub-pixel 31b continuously supplied from each column signal processing circuit 34 via the horizontal signal line 41 and outputs the processed signal. The output circuit 36 ​​may perform, for example, only buffering or may perform black level adjustment, column expansion correction, various types of digital signal processing, etc. The input / output terminal 38 exchanges signals with the outside.

[0064] The control circuit 37 is Figure 1The control circuit 15 in the vertical drive circuit 33 inputs a vertical synchronization signal, a horizontal synchronization signal, a master clock, etc. The control circuit 37 generates a clock signal and a control signal based on the vertical synchronization signal, the horizontal synchronization signal and the master clock as a reference for the operation of the vertical drive circuit 33, the column signal processing circuit 34 and the horizontal drive circuit 35. The control circuit 37 outputs these clock signals and control signals to the vertical drive circuit 33, the column signal processing circuit 34 and the horizontal drive circuit 35.

[0065] [Example of the First Color Filter Array]

[0066] Figure 3 It shows the settings in Figure 2 0 is a top view of an example of a first color filter array in the pixel array section 32 in FIG. Figure 3 An embodiment is shown in which more than half of the primary pixels in a pixel array may have a transparent color filter, a yellow color filter, or no color filter.

[0067] exist Figure 3 In FIG. 1 , only the pixel section 31 of the 4×4 color filter as a part of the pixel section provided in the pixel array section 32 is shown, but the same applies to the color filters of the other pixel sections 31. This also applies to the following description Figure 4 , 5 and 8.

[0068] exist Figure 3 In the example of , when viewed from above, the color filter 61a of each primary pixel 31a has a regular octagonal shape. All color filters 61a of the large color filter group (second color filter group) composed of the color filters 61a of the primary pixels 31a arranged in the pixel array section 32 are white (transparent) (W) filters that transmit light of all visible colors including green as a brightness component. In some embodiments, the pixel may not have a color filter.

[0069] The color filter 61b of each sub-pixel 31b is formed into a square that contacts the lower right side of the regular octagon of the color filter 61a when viewed from above. The array of the small color filter group (first color filter group) composed of the color filters 61b of the sub-pixels 31b arranged in the pixel array unit 32 is a Bayer array. Specifically, when the small color filter group is divided into an array unit 62 of 2 (rows) × 2 (columns) color filters 61b in sequence from the upper left color filter, the color of the upper left color filter 61b in the array unit 62 is red (R). The colors of the upper right and lower left color filters 61b are green (G), and the color of the lower right color filter 61b is blue (B). That is, the small color filter group is configured by three types of color filters of red, green and blue.

[0070] <Effect of an Array of Large Color Filter Groups>

[0071] The following will refer to Figure 4 Description by Figure 3 The effect is produced by an array of large color filter groups.

[0072] In the case where the array of large color filter groups is a Bayer array, similar to Figure 4 With the array of small color filter groups shown, the light received by the photodiode of the main pixel 31a is only red, green or blue. Therefore, the dynamic range on the low illumination (low brightness) side is not increased.

[0073] On the contrary, for Figure 3 In the array shown in , all filters 61a of the large filter group are white filters (in other words, transparent filters) that transmit light of all visible colors. Therefore, the light received by the photodiode of the main pixel 31a is light of all visible colors. Therefore, the imaging device 11 can increase the dynamic range on the low illumination side and increase the SN (single / noise) ratio of the brightness on the low illumination side. Therefore, the visibility of the object captured in the HDR image at low illumination can be improved.

[0074] However, because the light received by the photodiode of the main pixel 31a is light of all visible colors, the main pixel signal has only a white component, that is, no color component. Therefore, the color of an object with low illumination in the HDR image cannot be distinguished. However, since the array of the small color filter group is a Bayer array, the sub-pixel signal has red, green, and blue components. Therefore, color reproduction of a high-brightness object in an HDR image can be performed.

[0075] The array of small color filter groups may be different from the Bayer array, provided that color reproduction can be performed using sub-pixel signals. In addition, the type of color of the color filter 61b is not limited to the three colors of red, green and blue, and may be three colors of yellow, red and cyan, for example.

[0076] The color filter 61a is not limited to a white filter (non-color filter) as long as it is a broadband filter that transmits light in a band including the light transmitted by the green filter (i.e., light that contributes to the brightness value). For example, the color of the color filter 61a may be yellow, green, cyan, etc. In this case, since the light received by the photodiode of the main pixel 31a contributes to the brightness value, the dynamic range on the low illumination side can also be increased.

[0077] <Example of Second Color Filter Array>

[0078] Figure 5 is a plan view showing an example of a second color filter array provided in the pixel array section 32 . Figure 5 Another embodiment is shown where more than half of the primary pixels in a pixel array may have a transparent color filter, a yellow color filter, or no color filter.

[0079] exist Figure 5 In the array instance of Figure 3 The same reference numerals are assigned to those parts of the array instance in FIG. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to the array instance in FIG. Figure 3 The parts in the pattern instance are on different parts. Figure 5 The array instance in Figure 3 The array example shown in FIG. 1 differs in that some of the filters in the large filter group are red filters and the other configurations are the same as in FIG. Figure 3 The array instances in are the same.

[0080] Specifically, when the large color filter group is divided into array units 102 of 2 (rows) x 2 (columns) color filters 101 in sequence from the upper left color filter, the color of the upper left color filter 101 in the array unit 102 is red (R). The colors of the other three color filters 101 are white (W).

[0081] That is, only some of the color filters 101 in the large color filter group are white filters. Figure 4 When the array of the large color filter group shown is a Bayer array, the ratio of the white color filter 101 in the large color filter group is greater than the ratio of the green color filter 81 in the large color filter group. Specifically, Figure 5 The ratio of the white filter 101 in the large filter group shown in FIG. 1 is 75% (3 / 4), which is greater than Figure 4 The ratio of the green filter 81 in the large filter group in FIG. 5 is 50% (2 / 4).

[0082] From the above, we can see that using Figure 5 , with Figure 4 Compared with the array in Figure 3 The dynamic range on the low illumination side is increased in the same way as the array in Figure 5 In the array, since the light received by the photodiode of the primary pixel 31a having the red filter 101 is red light, the primary pixel signal has a red component. The red color of a low-illuminance object can be distinguished in the HDR image.

[0083] exist Figure 5 In the example shown, the color of the color filter 101 other than the white color filter 101 is red, but it may be blue or other colors. In addition, the number of colors of the color filter 101 is not limited to one. For example, one of the upper left color filters 101 of two adjacent array units 102 may be a red color filter, and the other may be a blue or cyan color filter.

[0084] As described above, in the solid-state imaging element 14, Figure 3In the case where the array of the large color filter group in the solid-state imaging element 14 is a Bayer array, the ratio of the color filter 61a (101) in the large color filter group is greater than the ratio of the green color filter 81 in the large color filter group. That is, the large color filter group of the solid-state imaging element 14 is provided with a ratio of Figure 3 The larger color filter group in the solid-state imaging element 14 can achieve a wider frequency band. Therefore, the dynamic range on the low illumination side can be increased without increasing the size of the main pixel 31a by using the solid-state imaging element 14. Therefore, the imaging device 11 can capture a high-sensitivity HDR image with high resolution without increasing the size.

[0085] On the contrary, Figure 3 As shown, in the case where the array of the large color filter group is a Bayer array, in order to increase the dynamic range on the low illumination side, it is necessary to increase the size of the main pixel 31a. Therefore, the size of the solid-state imaging element 14 increases, or the number of main pixels 31a decreases to suppress the increase in the size of the solid-state imaging element 14, and the resolution of the HDR image decreases.

[0086] <2. Second Embodiment>

[0087] <Configuration Example of Imaging Device>

[0088] Figure 6 is a block diagram showing a configuration example of a second embodiment of an imaging device as an electronic device to which the present technology is applied.

[0089] exist Figure 6 In the imaging device 111, corresponding to Figure 1 The parts of the imaging device 11 in FIG. 1 are assigned the same reference numerals. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to the imaging device 11. Figure 1 The imaging device 111 is different from the imaging device 11 in that the array direction of adjacent filters in the large filter group is inclined 45 degrees to the right with respect to the row direction, and pixel interpolation is performed, and the imaging device 111 is otherwise configured in the same manner as the imaging device 11.

[0090] The imaging device 111 may include at least one processor configured to generate image data. The at least one processor may be configured to generate image data based on signals from pixels of a pixel array included in the solid-state imaging device 114. Specifically, Figure 6 The imaging device 111 in the embodiment includes a solid-state imaging element 114 and a signal processing circuit 116 instead of the solid-state imaging element 14 and the signal processing circuit 16 .

[0091] The solid-state imaging element 114 differs from the solid-state imaging element 14 in that the array direction of adjacent filters within a large filter group is tilted 45 degrees to the right with respect to the row direction, and is otherwise configured in the same manner as the solid-state imaging element 14 .

[0092] The signal processing circuit 116 supplies the pixel signal supplied from the solid-state imaging element 114 to the memory 18 for storage therein. If necessary, the signal processing circuit 116 generates a sub-pixel signal and a main pixel signal while reading the pixel signal stored in the memory 18 in the same manner as the signal processing circuit 16.

[0093] The signal processing circuit 116 (generation unit) uses the sub-pixel signal to interpolate the sub-pixel signal at the middle position between two sub-pixels adjacent in the row direction and the column direction, and the interpolated signal may include the middle sub-pixel signal. The signal processing circuit 116 uses the main pixel signal to interpolate the main pixel signal at the middle position between two main pixels adjacent in the row direction and the column direction, and the interpolated signal may include the middle main pixel signal. The signal processing circuit 116 generates an HDR image using the interpolated sub-pixel signal and the interpolated main pixel signal. The signal processing circuit 116 provides an HDR image to be displayed as a captured image on the display 17, or recorded on a recording medium (not shown).

[0094] Here, the signal processing circuit 116 generates an HDR image after interpolating the sub-pixel signal and the main pixel signal separately, but the interpolation may be performed after the HDR image is generated. In this case, the signal processing circuit 116 generates an HDR image using the sub-pixel signal and the main pixel signal of each pixel portion, similar to the signal processing circuit 16. Afterwards, the signal processing circuit 116 uses the pixel signals of two pixels adjacent in the row direction and the column direction of the HDR image to interpolate the pixel signal at the middle position between the two pixels. Then, the signal processing circuit 116 sets the interpolated HDR image as the captured image.

[0095] The configuration of the solid-state imaging element 114 is similar to that of the pixel array section. Figure 2 The configuration of the solid-state imaging element 14 in FIG. 1 is the same. Therefore, only the pixel array portion of the solid-state imaging element 114 will be described below.

[0096] <Exemplary Configuration of Pixel Array Section>

[0097] Figure 7 1 is a top view showing a configuration example of a pixel array section of the solid-state imaging element 114 . Figure 7 Another embodiment is shown where more than half of the primary pixels in a pixel array may have a clear color filter, a yellow color filter, or no color filter.

[0098] Figure 7 The pixel array section 130 is different from the pixel array section 32 in that the array direction of adjacent color filters in the large color filter group is inclined 45 degrees to the right with respect to the row direction, and the other configurations are the same as the pixel array section 32 .

[0099] Specifically, in Figure 7 In the pixel array section 130, a plurality of pixel sections 131 are arranged in a matrix. Figure 7 In FIG. 1 , only the 4×8 pixel section 131 as a part of the pixel section provided in the pixel array section 32 is shown, but the same applies to the other pixel sections 131. Figures 9 to 14 The same is true.

[0100] The pixel unit 131 is composed of a main pixel 131a and a sub-pixel 131b. When viewed from above, the color filter 161a of each main pixel 131a has a regular octagonal shape. All color filters 161a of the large color filter group are white filters. Figure 7 The angle between the array direction of adjacent color filters 161a in the large color filter group indicated by arrow A in FIG. 1 and the row direction indicated by arrow L is 45 degrees.

[0101] The color filter 161b of each sub-pixel 131b has a square shape that contacts the right central side of the regular octagon of the color filter 161a in a plan view. The array of small color filter groups is obtained by tilting the Bayer array 45 degrees to the right.

[0102] Specifically, when the small color filter group is continuously divided into array units 162 of 2 (rows) × 2 (columns) color filters 161b inclined 45 degrees to the right from the upper left color filter, the color of one color filter 161b in the first row in the array unit 162 is red (R). Both of the two color filters 161b in the second row are green (G), and the color of one color filter 161b in the third row is blue (B). That is, the small color filter group is configured by three types of color filters of red, green and blue.

[0103] <Effect of Color Filter Array>

[0104] Next, we will refer to Figures 8 to 10 describe Figure 7 The effect of the array in .

[0105] like Figure 8 As shown, when the intervals between the pixel portions 31 are both P in the row direction and the column direction, Fig. 9 The interval between adjacent primary pixels 131 a arranged side by side in the array direction indicated by the middle arrow A is P.

[0106] As described above, a main pixel signal at a middle position between two main pixels 131 a adjacent in the row direction and the column direction and a sub pixel signal at a middle position between two sub pixels 131 b adjacent in the row direction and the column direction are interpolated.

[0107] Specifically, for example, the main pixel signal at the middle position C1 between two main pixels 131a adjacent to each other in the row direction is interpolated using the main pixel signals of the two main pixels 131a. The sub-pixel signal at the middle position C2 between two sub-pixels 131b adjacent to each other in the row direction is interpolated using the sub-pixel signals of the two sub-pixels 131b.

[0108] Therefore, by using the interval P between adjacent main pixels 131a, the interval Pi in the row direction and the column direction between pixels of the HDR image generated using the interpolated main pixel signal and sub-pixel signal can be expressed by the following formula (1).

[0109] [Mathematical formula 1]

[0110]

[0111] According to formula (1), Pi is approximately 1 / 1.4 of P.

[0112] Here, in the imaging device 11, since interpolation is not performed, the intervals in the row and column directions between the pixels of the HDR image generated by the imaging device 11, that is, the intervals between the pixel portions 31 in the row and column directions, are P. Therefore, the resolution in the row and column directions between the pixels of the HDR image generated by the imaging device 111 is approximately 1.4 times the resolution in the row and column directions between the pixels of the HDR image generated by the imaging device 11.

[0113] As described above, in the imaging device 111, the array direction of the adjacent color filters 161a in the large color filter group is inclined with respect to the row direction, and the main pixel signal and the sub-pixel signal are interpolated in the row direction and the column direction. Therefore, the resolution of the HDR image can be increased compared to the imaging device 11. In addition, since the main pixel signal does not have a color component, the interpolation of the main pixel signal does not generate false colors or artifacts.

[0114] At the same time, if Fig.10 As shown, when an array of large color filter groups is obtained by tilting the Bayer array 45 degrees to the right, Fig.10 The color filters 171 of every other row and column of primary pixels 131a shown in dashed lines are red or blue filters. Fig.10When a primary pixel signal of the primary pixel 131a is generated at the intersection of every other row and column indicated by the dashed line in FIG. 1 , the white balance of the area including the position is calculated, and the green component (luminance information) is calculated. Therefore, it is difficult to correctly generate the luminance information of the primary pixel signal of a color subject or a boundary of a color subject. As a result, false colors and artifacts may be generated.

[0115] The angle of the array direction of adjacent color filters 161a in the large color filter group relative to the row direction may not be 45 degrees, as long as the angle is greater than 0 degrees and less than 90 degrees.

[0116] <Another Example of the First Color Filter Array>

[0117] Fig.11 1 is a top view showing another example of a first color filter array provided in the pixel array section 130 . Fig.11 Another embodiment is shown where more than half of the primary pixels in a pixel array may have a clear color filter, a yellow color filter, or no color filter.

[0118] exist Fig.11 In the array instance in , corresponding to Figure 7 The same reference numerals are assigned to those parts of the array instance in FIG. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to FIG. Figure 7 The parts in the array instance are different. Fig.11 The array instance in Figure 7 The array example in FIG. 1 differs in that the array of small color filter groups is not a Bayer array and is otherwise configured in the same manner as Figure 7 The array instances in are the same.

[0119] Specifically, in an array of small color filter groups, Figure 7 The green color of the Bayer array in is replaced by yellow (Ye), and the blue color is replaced by cyan (Cy). More specifically, in the case where the small color filter group is continuously divided into array units 182 of 2 (rows) × 2 (columns) color filters 181 tilted 45 degrees to the right from the upper left color filter, the color of one color filter 181 in the first row in the array unit 182 is red (R). The colors of the two color filters 181 in the second row are both yellow (Ye), and the color of one color filter 181 in the third row is cyan (Cy). That is, the small color filter group is composed of three types of color filters: red, yellow, and cyan.

[0120] As mentioned above, in Fig.11 In the array example of , the small filter group includes a yellow or cyan filter 181, which is a broadband filter. Therefore, the sensitivity of the sub-pixel 131b having the filter 181 is improved. In addition, Figure 7Compared with the case of FIG. 1 , the occurrence of false colors and artifacts caused by interpolation of sub-pixel signals can be suppressed.

[0121] In contrast, Figure 7 In the array instance of Fig. 9 The color filters 161b of the sub-pixels 131b in every other row and column indicated by the dashed lines in FIG. 1 are red or blue filters. Fig. 9 When the sub-pixel signal of the sub-pixel 131b at the intersection of every other row and column indicated by the dashed line in FIG. 1 is calculated, the white balance of the area including the position is calculated, and the green component (luminance information) is calculated. Therefore, it is difficult to accurately generate the luminance information of the sub-pixel signal of the color object or the boundary of the color object. As a result, false colors and artifacts may be generated.

[0122] In by Fig. 9 In the case where some color filters 161b of the sub-pixels 131b in every other row and column indicated by the dashed lines in FIG. 1 are made into broadband color filters to suppress the occurrence of false colors, the number of red and blue color filters in the small color filter group is reduced. Therefore, the color resolution on the high brightness side of the HDR image is reduced.

[0123] <Another Second Color Filter Array Example>

[0124] Fig.12 1 is a top view showing another example of a second color filter array provided in the pixel array section 130 . Fig.12 Another embodiment is shown where more than half of the primary pixels in a pixel array may have a clear color filter, a yellow color filter, or no color filter.

[0125] exist Fig.12 In the array instance in , corresponding to Figure 7 The same reference numerals are assigned to those parts of the array instance in FIG. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to the array instance in FIG. Figure 7 The parts in the array instance are different. Fig.12 The array instance in Figure 7 The array example in FIG. 1 differs in that some of the filters in the large filter group are red filters and the other configurations are the same as in FIG. Figure 7 The array instances in are the same.

[0126] Specifically, when the large color filter group is continuously divided into array units 202 of 2 (rows)×2 (columns) color filters 201 inclined 45 degrees to the right from the upper left color filter, the color of one color filter 201 in the first row within the array unit 202 is red (R). The colors of the other three color filters 201 are white (W).

[0127] That is, only some of the color filters 201 in the large color filter group are white filters. Fig.10 When the array of the large color filter group shown in is a Bayer array, the ratio of the white color filter 201 in the large color filter group is higher than the ratio of the green color filter 171 in the large color filter group. Specifically, Fig.12 The ratio of the white filter 201 in the large filter group in is 75% (= 3 / 4), and Fig.10 The ratio of the green filter 171 in the large filter group in FIG. 1 is greater than 50% (= 2 / 4).

[0128] As mentioned above, Fig.10 Compared to the array in Fig.12 The array in the can increase the dynamic range on the low illumination side, with Figure 7 In addition, using Fig.12 In the array, the light received by the photodiode of the primary pixel 131a having the red filter 201 is red, so that the red color of the low-light object can be distinguished in the HDR image, as shown in FIG. Figure 5 Array in .

[0129] exist Fig.12 In the example shown, the color of the color filter 201 is red, but other colors such as blue may also be used.

[0130] <Another Third Color Filter Array Example>

[0131] Fig.13 1 is a top view showing another example of a third color filter array provided in the pixel array section 130 . Fig.13 Another embodiment is shown where more than half of the primary pixels in the pixel array may have a clear filter, a yellow filter, or no filter.

[0132] exist Fig.13 In the array instance in , corresponding to Fig.12 The same reference numerals are assigned to those parts of the array instance in FIG. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to the array instance in FIG. Fig.12 The parts in the pattern instance are on different parts. Fig.13 The array instance in Fig.12 The array examples in FIG. 1 differ in that some of the filters in the large filter set are red filters or blue filters, and the other configurations are the same as Fig.12 The array instances in are the same.

[0133] Specifically, when the large color filter group is continuously divided into array units 222 of 2 (rows) × 2 (columns) color filters 221 tilted 45 degrees to the right from the upper left color filter, the color of the upper left color filter 221 of one of two adjacent array units 222 is red (R). The colors of the other three color filters 221 are white (W). The color of the upper left color filter 221 of another adjacent array unit is blue (B), and the colors of the other three color filters 221 are white (W). That is, the large color filter group is composed of three types of color filters of white, red and blue.

[0134] As described above, the ratio of the white filter 221 in the large filter group is Fig.12 Therefore, using Fig.13 , with Fig.12 Like the array in , the dynamic range on the low-light side can be improved. Fig.13 Since the large color filter group is composed of three types of color filters, white, red, and blue, the main pixel signal has these three types of color components. Therefore, the color of low-light objects can be reproduced in the HDR image.

[0135] The colors of the upper left color filter 221 of the adjacent array unit 222 are not limited to red and blue, and may be, for example, red and cyan.

[0136] <Another Fourth Color Filter Array Example>

[0137] Fig.14 1 is a top view showing another fourth color filter array example provided in the pixel array section 130 . Fig.14 Another embodiment is shown where more than half of the primary pixels in the pixel array may have a clear color filter, a yellow color filter, or no color filter.

[0138] exist Fig.14 In the array instance in , corresponding to Fig.13 The same reference numerals are assigned to those parts of the array instance in FIG. Therefore, the description of these parts is omitted as appropriate, and the description focuses on the parts related to the array instance in FIG. Fig.13 The parts in the array instance are different. Fig.14 The array instance in Fig.13 The array example in FIG. 1 is different in that the white filter of the large filter set is replaced by a yellow filter and the blue filter is replaced by a cyan filter, and the other configurations are the same as Fig.13 The array instances in are the same.

[0139] Specifically, when the large color filter group is continuously divided into array units 242 of 2 (rows) × 2 (columns) color filters 241 tilted 45 degrees to the right from the upper left color filter, the color of one upper left color filter 241 in two adjacent array units 242 is red (R). The colors of the other three color filters 241 are yellow (Ye). The color of the other upper left color filter 241 is cyan (Cy), and the colors of the other three color filters 241 are yellow (Ye). That is, the large color filter group is composed of three color filters of yellow, red, and cyan.

[0140] As mentioned above, in Fig.10 In the case where the array of the large color filter group shown is a Bayer array, the ratio of the yellow and cyan color filters 241 as broadband color filters in the large color filter group is greater than the ratio of the green color filter 171 in the large color filter group. Specifically, Fig.12 The ratio of the color filter 201 as a broadband color filter in the large color filter group in FIG. 8 is 87.5% (= 7 / 8), which is greater than Fig.10 The ratio of the green filter 171 in the large filter group in FIG. 5 is 50% (= 1 / 2).

[0141] Therefore, with Fig.12 Like the array in Fig.10 Compared to the array in Fig.14 The array in the can improve the dynamic range on the low illumination side. In addition, for Fig.14 Since the large color filter group consists of three types of filters, yellow, red, and cyan, the main pixel signal has these three types of color components. Therefore, the color of low-light objects can be reproduced in the HDR image. Since this color reproduction requires segmentation, it is similar to Fig.10 The SN ratio of the color on the low illumination side may be deteriorated compared to the array in .

[0142] The color filter 161a (201, 221, 241) of the main pixel 131a is not limited to a white or yellow color filter, but may be a green, cyan or other color filter as long as they are broadband color filters.

[0143] As described above, in the solid-state imaging element 114, Fig.10 , the ratio of the filter 161a (201, 221, 241) as a broadband filter in the large filter group is greater than the ratio of the green filter 171 in the large filter group. Therefore, like the imaging device 11, the imaging device 111 can capture HDR images with high resolution and high sensitivity without increasing the size.

[0144] In addition, in the solid-state imaging element 114, the angle between the array direction and the row direction of the adjacent color filters 161a (201, 221, 241) in the large color filter group is 45 degrees, which is greater than 0 degrees and less than 90 degrees. Therefore, compared with the solid-state imaging element 14, the solid-state imaging element 114 can improve the resolution in the row direction (horizontal direction) and the column direction (vertical direction) of the captured image through pixel interpolation.

[0145] In the solid-state imaging element 114, at least one color filter 161a (201, 221, 241) of two primary pixels 131a adjacent in the row direction and at least one color filter of two primary pixels 131a adjacent in the column direction are broadband color filters. Therefore, it is possible to prevent the occurrence of false colors and artifacts caused by interpolation of primary pixel signals.

[0146] In the first and second embodiments described above, an HDR image is generated and output using the main pixel signal and the sub pixel signal, but the main pixel signal and the sub pixel signal may be output as they are.

[0147] For example, the present technology can be applied not only to imaging devices such as digital still cameras and digital video cameras but also to various electronic devices such as mobile phones with an imaging function and other devices with an imaging function.

[0148] <3. Application to moving objects>

[0149] The technology (prior art) according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure can be implemented as a device installed on any type of mobile object such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, etc.

[0150] Fig.15 1 is a block diagram showing an example of a schematic configuration of a vehicle control system as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied. The vehicle control system described herein can be configured to control a vehicle based on image data generated using the light detection device described herein.

[0151] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. Fig.15In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown as the functional configuration of the integrated control unit 12050.

[0152] The drive system control unit 12010 controls the operation of the devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 is used as a control device to control: a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine, a drive motor, etc., a drive force transmitting mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating the braking force of the vehicle, etc.

[0153] The body system control unit 12020 controls the operation of various types of equipment configured for the body according to various programs. For example, the body system control unit 12020 is used as a control device to control the following items: a keyless entry system, a smart key system, a power window device, or various lights such as headlights, reverse lights, brake lights, turn signals, fog lights, etc. In this case, the body system control unit 12020 can receive as input radio waves transmitted from a mobile device that replaces the key or signals from various switches. The body system control unit 12020 receives these input radio waves or signals to control the door lock device, power window device, lights, etc. of the vehicle.

[0154] The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to an imaging unit 12031. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to image an image of the vehicle exterior, and receives the imaged image. Based on the received image, the vehicle exterior information detection unit 12030 may perform a process of detecting an object (such as a person, vehicle, obstacle, sign, symbol, etc. on the road), or perform a process of detecting the distance to the object.

[0155] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or can output the electrical signal as information about the measured distance. In addition, the light received by the imaging unit 12031 may be visible light, or may be invisible light such as infrared rays.

[0156] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. The in-vehicle information detection unit 12040 may be connected to a driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that photographs the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 may calculate the driver's fatigue level or the driver's concentration level, or may distinguish whether the driver is dozing off.

[0157] The microcomputer 12051 can calculate a control target value for a driving force generation device, a steering mechanism, or a braking device based on information about the inside or outside of the vehicle obtained by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control of functions intended to implement ADAS (Advanced Driver Assistance System), which includes collision avoidance or impact buffering for the vehicle, follow-up driving based on vehicle-to-vehicle distance, vehicle speed maintenance driving, an alarm for vehicle collision, an alarm for vehicle lane deviation, and the like.

[0158] In addition, the microcomputer 12051 can control the driving force generating device, the steering mechanism, and the braking device based on the information about the outside or inside of the vehicle obtained by the outside information detection unit 12030 or the inside information detection unit 12040, thereby performing collaborative control intended for automatic driving, etc. that is independent of the driver's operation.

[0159] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the exterior of the vehicle obtained by the exterior information detection unit 12030. For example, the microcomputer 12051 can control the headlights to change from high beam to low beam based on the position of the preceding vehicle or oncoming vehicle detected by the exterior information detection unit 12030, thereby performing cooperative control aimed at preventing glare by controlling the headlights.

[0160] The sound / image output unit 12052 transmits an output signal of at least one of sound and image to an output device, which can notify information to the passengers of the vehicle or the outside of the vehicle in a visual or auditory manner. Fig.15 In the example of FIG. 1206 , an audio speaker 12061, a display portion 12062, and an instrument panel 12063 are shown as output devices. The display portion 12062 may include, for example, at least one of an in-vehicle display and a head-up display.

[0161] Fig.16 12031 is a diagram showing an example of the installation position of the imaging unit 12031.

[0162] exist Fig.16 In the figure, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104 and 12105.

[0163] For example, the imaging units 12101, 12102, 12103, 12104, and 12105 may be arranged at the positions of the front nose, side mirrors, rear bumper, rear door, and upper portion of the windshield inside the vehicle 12100. The imaging unit 12101 arranged at the front nose and the imaging unit 12105 arranged at the upper portion of the windshield inside the vehicle mainly obtain images in front of the vehicle 12100. The imaging units 12102 and 12103 arranged at the side mirrors mainly obtain images of the sides of the vehicle 12100. The imaging unit 12104 arranged at the rear bumper or rear door mainly obtains images of the rear of the vehicle 12100. The imaging unit 12105 arranged at the upper portion of the windshield inside the vehicle is mainly used to detect front vehicles, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0164] Fig.16 An example of the shooting range of the imaging units 12101 to 12104 is shown. Imaging range 12111 represents the imaging range of the imaging unit 12101 arranged at the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging units 12102 and 12103 arranged at the side mirrors. Imaging range 12114 represents the imaging range of the imaging unit 12104 arranged at the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 12100 observed from above can be obtained by superimposing the image data imaged by the imaging units 12101 to 12104.

[0165] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0166] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the time change of the distance (relative speed relative to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, and thereby extract the nearest three-dimensional object as the leading vehicle, which specifically exists on the driving path of the vehicle 12100 and travels at a predetermined speed (e.g., equal to or greater than 0 km / h) in the substantially same direction as the vehicle 12100. In addition, the microcomputer 12051 can pre-set the following distance to be maintained from the leading vehicle, and perform automatic braking control (including following parking control), automatic acceleration control (including following starting control), etc. Therefore, cooperative control intended for automatic driving, etc. that does not depend on the operation of the driver, can be performed.

[0167] For example, the microcomputer 12051 can classify the three-dimensional object data about the three-dimensional object into three-dimensional object data of two-wheeled vehicles, standard-sized vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, and extract the classified three-dimensional object data for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes whether the obstacles around the vehicle 12100 are obstacles that can be visually identified by the driver of the vehicle 12100, or obstacles that are difficult for the driver of the vehicle 12100 to visually identify. Then, the microcomputer 12051 determines the collision risk, which indicates the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 outputs an alarm to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or avoidance steering via the drive system control unit 12010. Thus, the microcomputer 12051 can assist driving to avoid collisions.

[0168] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 is capable of identifying pedestrians by determining whether there are pedestrians in the imaging images of the imaging units 12101 to 12104. This pedestrian recognition is performed, for example, by the following programs: a program for extracting characteristic points in the imaging images of the imaging units 12101 to 12104 as infrared cameras, and a program for determining whether it is a pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there are pedestrians in the imaging images of the imaging units 12101 to 12104 and thus identifies the pedestrians, the sound / image output unit 12052 controls the display unit 12062 to display a square outline superimposed on the identified pedestrian for emphasizing the identified pedestrian. The sound / image output unit 12052 can also control the display unit 12062 to display an icon representing a pedestrian at a desired position.

[0169] An exemplary vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 in the above-mentioned components. Specifically, it is possible to Figure 1 The imaging device 11 and Figure 6 The imaging device 111 is applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, when the pixel portion is composed of two types of pixels with different light receiving areas, the sensitivity and dynamic range can be increased on the low illumination side without increasing the size of the pixel portion. Therefore, in the vehicle control system 12000, a small imaging unit 12031 that captures HDR images with high resolution and high sensitivity can be realized.

[0170] In this case, the color of the low-illuminance object cannot be reproduced in the captured image, or the color reproducibility is low. However, in the vehicle control system 12000 used in automatic driving, driving support systems, etc., the objects that need color recognition are high-brightness objects of light sources such as traffic lights and brake lights, and the color reproducibility of low-illuminance objects is not a big problem compared to the color reproducibility of high-brightness objects.

[0171] The embodiments of the present invention are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0172] For example, all or some of the above-mentioned embodiments may be combined together.

[0173] The effects described in this specification are merely exemplary and not restrictive, and there may be effects other than those described in this specification.

[0174] The present technology can adopt the following configurations. (1)

[0176] A solid-state imaging device comprising:

[0177] A pixel array in which a plurality of pixel portions are arranged in an array, each pixel portion being configured by a sub-pixel having a first light receiving area and a main pixel having a second light receiving area larger than the first light receiving area, wherein

[0178] A first color filter group configured by color filters of each of a plurality of sub-pixels arranged in a pixel array is configured by three types of color filters;

[0179] at least some of the color filters in the second color filter group configured by the color filters of each of the plurality of primary pixels arranged in the pixel array are broadband color filters that transmit light in a band including a green light band; and

[0180] The ratio of the broadband color filters in the second color filter group is greater than 50%. (2)

[0182] The solid-state imaging device according to (1) above, wherein

[0183] Broadband color filters are color filters that transmit light of all visible colors. (3)

[0185] The solid-state imaging device according to (1) above, wherein

[0186] The broadband color filter is a yellow color filter or a green color filter. (4)

[0188] The solid-state imaging device according to any one of (1) to (3) above, wherein

[0189] All color filters of the second color filter set are broadband color filters. (5)

[0191] The solid-state imaging device according to any one of (1) to (3) above, wherein

[0192] The second color filter group is configured by a broadband color filter and a red or blue color filter. (6)

[0194] The solid-state imaging device according to any one of (1) to (3) above, wherein

[0195] The second color filter group is configured by a broadband color filter, a red color filter, and a blue color filter. (7)

[0197] The solid-state imaging device according to any one of (1) to (3) above, wherein

[0198] The second color filter group is configured by a broadband color filter, a cyan color filter, and a red color filter. (8)

[0200] The solid-state imaging device according to any one of (1) to (7) above, wherein

[0201] The angle between the arrangement direction of adjacent color filters in the second color filter group and the row direction is greater than 0 degree and less than 90 degrees. (9)

[0203] The solid-state imaging device according to (8) above, wherein

[0204] The three color filters are a red filter, a yellow filter, and a cyan filter. (10)

[0206] An electronic device, comprising:

[0207] A solid-state imaging device, configured to include:

[0208] A pixel array in which a plurality of pixel sections are arranged in an array, each pixel section being composed of a sub-pixel having a first light receiving area and a main pixel having a second light receiving area larger than the first light receiving area,

[0209] a first color filter group, configured by a color filter of each sub-pixel in a plurality of sub-pixels arranged in a pixel array, configured by three types of color filters,

[0210] At least some of the color filters in the second color filter group, configured by the color filter of each of the plurality of primary pixels arranged in the pixel array, are broadband color filters, being color filters that transmit light in a band including a green light band, and

[0211] The ratio of broadband color filters in the second color filter group is greater than 50%; and

[0212] A generating unit generates an image by using a sub-pixel signal which is a pixel signal corresponding to light received by the sub-pixel and a main pixel signal which is a pixel signal corresponding to light received by the main pixel. (11)

[0214] The electronic device according to (10) above, wherein

[0215] The angle between the array direction and the row direction of adjacent color filters in the second color filter group is greater than 0 degrees and less than 90 degrees, and

[0216] The generation unit uses sub-pixel signals to interpolate sub-pixel signals at positions between adjacent sub-pixels in the row direction and the column direction, uses main pixel signals to interpolate main pixel signals at positions between adjacent main pixels in the row direction and the column direction, and generates an image using the interpolated sub-pixel signals and main pixel signals.

[0217] (A1)

[0218] A light detection device, comprising:

[0219] A pixel array, the pixel array comprising:

[0220] A plurality of primary pixels; and

[0221] Multiple sub-pixels with color filters,

[0222] Among them, more than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter.

[0223] (A2)

[0224] The light detection device according to (A1), wherein the plurality of sub-pixels include:

[0225] a first sub-pixel group having a color filter of a first color;

[0226] a second group of sub-pixels having a color filter of a second color; and

[0227] The third group of sub-pixels has a color filter of a third color.

[0228] (A3)

[0229] The light detecting device according to (A2), wherein the color filter of the first color includes a red color filter.

[0230] (A4)

[0231] The light detection device according to (A3), wherein:

[0232] The color filter of the second color includes a blue color filter; and

[0233] The color filter of the third color includes a green color filter.

[0234] (A5)

[0235] The light detection device according to any one of (A2) to (A4), wherein:

[0236] The color filter of the first color includes a red color filter;

[0237] The color filter of the second color includes a yellow color filter; and

[0238] The color filter of the third color includes a cyan color filter.

[0239] (A6)

[0240] The light detection device according to any one of (A1) to (A5), wherein the light receiving region of each of the plurality of main pixels includes an octagonal shape.

[0241] (A7)

[0242] The light detection device according to any one of (A1) to (A6), wherein each of the plurality of primary pixels has a transparent color filter or has no color filter.

[0243] (A8)

[0244] The light detection device according to any one of (A1) to (A7), wherein at least one main pixel among the plurality of main pixels has a red color filter.

[0245] (A9)

[0246] The light detection device according to any one of (A1) to (A8), further comprising:

[0247] A column signal line extending in a column direction;

[0248] The pixel control signal lines extending in the horizontal direction,

[0249] Among them, a plurality of main pixels and a plurality of sub-pixels are arranged in a plurality of pixel columns in a column direction and in a plurality of pixel rows in a horizontal direction.

[0250] (A10)

[0251] The light detection device according to (A9), wherein at least one of the plurality of main pixels has a red filter.

[0252] (A11)

[0253] The light detection device according to (A9), wherein:

[0254] At least one of the plurality of primary pixels has a red color filter; and

[0255] At least one main pixel among the plurality of main pixels has a blue color filter.

[0256] (A12)

[0257] The light detection device according to (A9), wherein each of the plurality of pixel rows includes at least one pixel having a transparent color filter or having no color filter.

[0258] (A13)

[0259] The light detection device according to (A12), wherein each of the plurality of pixel columns includes at least one pixel having a transparent color filter or having no color filter.

[0260] (A14)

[0261] The light detection device according to any one of (A1) to (A13), wherein:

[0262] More than half of the multiple primary pixels have a yellow color filter;

[0263] At least one of the plurality of primary pixels has a red color filter; and

[0264] At least one of the plurality of primary pixels has a cyan color filter.

[0265] (A15)

[0266] The light detection device according to any one of (A1) to (A14), further includes a processor configured to generate image data based on signals from the plurality of main pixels and signals from the plurality of sub-pixels.

[0267] (A16)

[0268] The light detection device according to (A15), wherein:

[0269] The processor is configured to use interpolation to calculate:

[0270] an intermediate primary pixel signal of a position between primary pixels of the plurality of primary pixels; and

[0271] intermediate sub-pixel signals for locations between sub-pixels in the plurality of sub-pixels; and

[0272] The processor is configured to generate image data based on signals from the plurality of main pixels, signals from the plurality of sub-pixels, intermediate main pixel signals, and intermediate sub-pixel signals.

[0273] (A17)

[0274] A vehicle comprising:

[0275] a light detection device configured to generate a signal;

[0276] a processor configured to generate image data based on the signal from the light detection device;

[0277] a vehicle control system configured to control the vehicle based on the image data,

[0278] Wherein, the light detection device comprises:

[0279] A pixel array, the pixel array comprising:

[0280] A plurality of primary pixels; and

[0281] Multiple sub-pixels with color filters,

[0282] Among them, more than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter.

[0283] (A18)

[0284] An automotive camera system, comprising:

[0285] lens,

[0286] A light detection device comprising:

[0287] A pixel array, the pixel array comprising:

[0288] A plurality of primary pixels; and

[0289] Multiple sub-pixels with color filters,

[0290] wherein more than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter; and

[0291] A processor is configured to generate image data based on signals from the plurality of main pixels and signals from the plurality of sub-pixels.

[0292] (A19)

[0293] The automotive camera system according to (A18), wherein at least one primary pixel among the plurality of primary pixels has a red color filter.

[0294] (A20)

[0295] An automotive camera system according to (A18) or (A19), wherein:

[0296] The processor is configured to use interpolation to calculate:

[0297] an intermediate primary pixel signal of a position between primary pixels of the plurality of primary pixels; and

[0298] intermediate sub-pixel signals for locations between sub-pixels in the plurality of sub-pixels; and

[0299] The processor is configured to generate image data based on signals from the plurality of main pixels, signals from the plurality of sub-pixels, intermediate main pixel signals, and intermediate sub-pixel signals.

[0300] [Reference Numbers List]

[0301] 11 Imaging device

[0302] 14 Solid-state imaging element

[0303] 16 Signal processing circuit

[0304] 31 Pixels

[0305] 31a Main pixel

[0306] 31b sub-pixel

[0307] 32 Pixel array section

[0308] 61a, 61b Color filter

[0309] 101 Color Filters

[0310] 111 Imaging Device

[0311] 114 Solid-state imaging element

[0312] 116 Signal Processing Circuit

[0313] 130 Pixel array unit

[0314] 131 Pixels

[0315] 131a Main pixel

[0316] 131b sub-pixel

[0317] 161a,161b Color filter

[0318] 181,201,221,241 color filters.

Claims

1. A light detection device, comprising: A pixel array, the pixel array comprising: A plurality of primary pixels; and Multiple sub-pixels with color filters, More than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter.

2. The light detection device according to claim 1, wherein: The plurality of sub-pixels include: A first sub-pixel group having a color filter of a first color; a second group of sub-pixels having a color filter of a second color; and The third group of sub-pixels has a color filter of a third color.

3. The light detection device according to claim 2, wherein: The color filter of the first color includes a red color filter.

4. The light detection device according to claim 3, wherein: The second color filter includes a blue filter; and The color filter of the third color includes a green color filter.

5. The light detection device according to claim 2, wherein: The color filter of the first color includes a red color filter; The second color filter includes a yellow filter; and The color filter of the third color includes a cyan color filter.

6. The light detection device according to claim 1, wherein: A light receiving region of each of the plurality of main pixels includes an octagonal shape.

7. The light detection device according to claim 1, wherein Each of the plurality of primary pixels has a transparent color filter or has no color filter.

8. The light detection device according to claim 1, wherein At least one of the plurality of primary pixels has a red color filter.

9. The light detection device according to claim 1, further comprising: A column signal line extending in a column direction; The pixel control signal lines extending in the horizontal direction, The plurality of main pixels and the plurality of sub-pixels are arranged in a plurality of pixel columns in the column direction and in a plurality of pixel rows in the horizontal direction.

10. The light detection device according to claim 9, wherein: At least one of the plurality of primary pixels has a red color filter.

11. The light detection device according to claim 9, wherein: At least one of the plurality of primary pixels has a red color filter; and At least one main pixel among the plurality of main pixels has a blue color filter.

12. The light detection device according to claim 9, wherein: Each of the plurality of pixel rows includes at least one pixel having a transparent color filter or having no color filter.

13. The light detection device according to claim 12, wherein: Each of the plurality of pixel columns includes at least one pixel having a transparent color filter or having no color filter.

14. The light detection device according to claim 1, wherein: More than half of the plurality of primary pixels have a yellow color filter; At least one of the plurality of primary pixels has a red color filter; and At least one of the plurality of primary pixels has a cyan color filter.

15. The light detection device according to claim 1, further comprising: A processor is configured to generate image data based on signals from the plurality of main pixels and signals from the plurality of sub-pixels.

16. The light detection device according to claim 15, wherein: The processor is configured to calculate using interpolation: intermediate primary pixel signals of locations between primary pixels in the plurality of primary pixels; as well as an intermediate sub-pixel signal at a position between sub-pixels in the plurality of sub-pixels; as well as The processor is configured to generate the image data based on signals from the plurality of main pixels, signals from the plurality of sub-pixels, the intermediate main pixel signal, and the intermediate sub-pixel signal.

17. A vehicle comprising: a light detection device configured to generate a signal; a processor configured to generate image data based on the signal from the light detection device; a vehicle control system configured to control the vehicle based on the image data, Wherein, the light detection device comprises: A pixel array, the pixel array comprising: A plurality of primary pixels; and Multiple sub-pixels with color filters, More than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter.

18. An automotive camera system, comprising: lens, A light detection device comprising: A pixel array, the pixel array comprising: A plurality of primary pixels; and Multiple sub-pixels with color filters, wherein more than half of the plurality of primary pixels have a transparent color filter, a yellow color filter, or no color filter; and A processor is configured to generate image data based on signals from the plurality of main pixels and signals from the plurality of sub-pixels.

19. The automotive camera system of claim 18, wherein: At least one of the plurality of primary pixels has a red color filter.

20. The automotive camera system of claim 18, wherein: The processor is configured to calculate using interpolation: intermediate primary pixel signals of locations between primary pixels of the plurality of primary pixels; and intermediate sub-pixel signals at locations between sub-pixels in the plurality of sub-pixels; as well as The processor is configured to generate the image data based on signals from the plurality of main pixels, signals from the plurality of sub-pixels, the intermediate main pixel signal, and the intermediate sub-pixel signal.

Images