Image sensor

Abstract

The present invention suppresses color shift in an image sensor in which a plurality of pixels are arranged.　This image sensor comprises a pixel array portion and a signal processing portion. In the image sensor comprising the pixel array portion and the signal processing portion, a predetermined number of pixels are arranged in the pixel array portion in a non-Bayer array. In the image sensor comprising the pixel array portion and the signal processing portion, the signal processing portion generates and outputs a saturation pattern indicating whether or not a pixel signal is saturated for each pixel.

Description

Image sensor

[0001] This technology relates to image sensors. More specifically, it relates to image sensors with multiple pixels arranged in a single array.

[0002] Image sensors have been used in vehicles and other applications for purposes such as autonomous driving and driver assistance. For example, an image sensor has been proposed that divides each pixel of a Bayer array into a pair of sub-pixels with different sensitivities and combines the signals of each sub-pixel using HDR (High-Dynamic-Range) synthesis (see, for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2017-163010

[0004] The conventional technology described above attempts to expand the dynamic range through HDR synthesis. However, with the Bayer array of the conventional technology described above, there is a risk of color shift when high-brightness visible light is incident. For example, since part of the wavelength range of the R (Red) pixel overlaps with the wavelength range of the G (Green) pixel, when high-brightness red light is incident, the R and G pixels may become saturated and shift to yellow. Particularly in automotive applications, color shift is a problem as it may cause the system to misinterpret the color of traffic lights.

[0005] This technology was developed in light of these circumstances and aims to suppress color shift in image sensors with multiple pixels arranged in a single array.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor comprising a pixel array section in which a predetermined number of pixels are arranged in an arrangement that does not correspond to a Bayer array, and a signal processing section that generates and outputs a saturation pattern indicating whether or not the pixel signal has saturated for each pixel. This results in the suppression of color shift.

[0007] Furthermore, in this first aspect, the pixels may include Mg (Magenta) pixels that receive magenta visible light, Ye (Yellow) pixels that receive yellow visible light, and Cy (Cyan) pixels that receive cyan visible light. This results in the suppression of color shift.

[0008] Furthermore, in this first aspect, the pixel may include an Rc pixel that receives red visible light with a peak wavelength of 600 nanometers, a Gc pixel that receives green visible light with a peak wavelength of 550 nanometers, and a Bc pixel that receives blue visible light with a peak wavelength of 450 nanometers. This results in properties similar to those of a cone cell.

[0009] Furthermore, in this first aspect, the above-mentioned pixels may include an Mg' pixel that receives magenta visible light, a G' pixel that receives green visible light, and a B (Blue) pixel that receives blue visible light. This results in the effect of suppressing color shift.

[0010] Furthermore, in this first aspect, the wavelength range of the color filter for the Mg' pixel may be 400 to 550 nanometers and 560 to 700 nanometers, and the wavelength range of the color filter for the G' pixel may be 460 to 610 nanometers. This results in the suppression of color shift.

[0011] Furthermore, in this first aspect, the pixels may include a B pixel that receives blue visible light, a Yo pixel that receives visible light of a color between yellow and orange, a Gp pixel that receives green visible light whose wavelength range does not overlap with that of the Yo pixel, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yo pixel. This results in the effect of suppressing color shift.

[0012] Furthermore, in this first aspect, the wavelength range of the color filter for the Gp pixel may be 500 to 570 nanometers, the wavelength range of the color filter for the Yo pixel may be 570 to 610 nanometers, and the wavelength range of the color filter for the Rp pixel may be 610 to 700 nanometers. This results in the suppression of color shift.

[0013] Furthermore, in this first aspect, the pixels may include a Bcy pixel that receives visible light of a color between blue and cyan, a Yog pixel that receives light of a color between yellow and orange, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yog pixel. This results in the suppression of color shift.

[0014] Furthermore, in this first aspect, the wavelength range of the color filter for the Bcy pixel may be 400 to 510 nanometers, the wavelength range of the color filter for the Yog pixel may be 500 to 610 nanometers, and the wavelength range of the color filter for the Rp pixel may be 610 to 700 nanometers. This results in the suppression of color shift.

[0015] Furthermore, the second aspect of this technology is an image sensor that comprises a pixel array section in which Mg' pixels that receive magenta visible light, G' pixels that receive green visible light, and B pixels that receive blue visible light are arranged. This results in the effect of suppressing color shift.

[0016] Furthermore, a third aspect of this technology is an image sensor comprising a pixel array section in which B pixels that receive blue visible light, Yo pixels that receive visible light of a color between yellow and orange, Gp pixels that receive green visible light whose wavelength range does not overlap with that of the Yo pixels, and Rp pixels that receive red visible light whose wavelength range does not overlap with that of the Yo pixels are arranged. This results in the effect of suppressing color shift.

[0017] Furthermore, the second aspect of this technology is an image sensor comprising a pixel array section in which Bcy pixels that receive visible light of a color between blue and cyan, Yog pixels that receive light of a color between yellow and orange, and Rp pixels that receive red visible light whose wavelength range does not overlap with that of the Yog pixels are arranged. This results in the suppression of color shift.

[0018] This is a block diagram showing an example configuration of a camera module in the first embodiment of this technology. This is a block diagram showing an example configuration of an image sensor in the first embodiment of this technology. This is a plan view showing an example configuration of a pixel array section in the first embodiment of this technology. This is a circuit diagram showing an example configuration of an FD (Floating Diffusion) shared block in the first embodiment of this technology. This is a block diagram showing an example configuration of a column processing section in the first embodiment of this technology. This is an example of a color tone diagram in the first embodiment of this technology. This is an example of a color tone diagram in a comparative example. This is a block diagram showing an example configuration of a signal processing section and a SoC (System on a Chip) in the first embodiment of this technology. This is a diagram showing an example of a saturation pattern and a color determination result for each saturation pattern in the first embodiment of this technology. This is a diagram showing an example of the operation of a camera module in the first embodiment of this technology. This is a plan view showing an example configuration of a pixel array section in the second embodiment of this technology. This is a diagram showing an example of the wavelength characteristics of a color filter in the second embodiment of this technology. This is an example of a color tone diagram in the second embodiment of this technology. This is a diagram showing an example of a saturation pattern and a color determination result for each saturation pattern in the second embodiment of this technology. This is a plan view showing an example configuration of the pixel array section in the third embodiment of this technology. This is a diagram showing an example of the wavelength characteristics of the color filter in the third embodiment of this technology. This is an example of a color tone diagram in the third embodiment of this technology. This is a diagram showing an example of a saturation pattern and the color determination result for each saturation pattern in the third embodiment of this technology. This is a plan view showing an example configuration of the pixel array section in the fourth embodiment of this technology. This is a graph showing an example of the wavelength characteristics of an LED (Light Emitting Diode) traffic light in the fourth embodiment of this technology. This is a graph showing an example of the wavelength characteristics of an incandescent light bulb type traffic light in the fourth embodiment of this technology. This is a diagram showing an example of saturation characteristics in the fourth embodiment and comparative example of this technology. This is a plan view showing an example configuration of the pixel array section in the fifth embodiment of this technology. This is a diagram showing an example of the wavelength range of the color filter in the fifth embodiment of this technology.This figure shows an example of saturation characteristics in a fifth embodiment and comparative example of this technology. This block diagram shows an example of a schematic configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation positions of the external information detection unit and the imaging unit.

[0019] The following describes the embodiments for implementing this technology (hereinafter referred to as "embodiments"). The description will proceed in the following order: 1. First embodiment (example using MgYeYeCy sequence) 2. Second embodiment (example using pixels with characteristics similar to cone cells) 3. Third embodiment (example using MgGGB sequence) 4. Fourth embodiment (example using RpYoGpB sequence) 5. Fifth embodiment (example using RpYogBcy sequence) 6. Application example to a mobile object

[0020] <1. First Embodiment> [Example of Camera Module Configuration] Figure 1 is a block diagram showing an example configuration of a camera module 100 in the first embodiment of this technology. This camera module 100 is a module that captures image data and outputs it to an external circuit, and comprises an optical unit 110, an image sensor 200, and an SoC 120. For example, when the camera module 100 is installed in a vehicle control system, the data from the camera module 100 is output to the ECU (Electronic Control Unit).

[0021] The optical unit 110 collects light from the subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion and outputs it to the SoC 120. The SoC 120 performs various processes on the image data, such as image recognition processing, and outputs the processed data to the outside.

[0022] [Example of Image Sensor Configuration] Figure 2 is a block diagram showing an example configuration of an image sensor 200 in the first embodiment of this technology. This image sensor 200 includes a vertical drive unit 210, a system control unit 220, a DAC (Digital to Analog Converter) 230, and a pixel array unit 240. The image sensor 200 further includes a column processing unit 250, a horizontal drive unit 260, a signal processing unit 270, and an interface 280.

[0023] Multiple pixels are arranged in a two-dimensional grid in the pixel array section 240. The vertical drive section 210 drives the rows in the pixel array section 240 sequentially to generate pixel signals.

[0024] The system control unit 220 controls the operating timing of the vertical drive unit 210, DAC 230, column processing unit 250, and horizontal drive unit 260 in synchronization with the vertical synchronization signal.

[0025] The DAC230 generates a sawtooth-shaped ramp signal through DA (Digital to Analog) conversion and supplies it to the column processing unit 250.

[0026] The column processing unit 250 has ADCs (Analog to Digital Converters) and other components arranged for each column of the pixel array unit 240. These ADCs perform analog-to-digital conversion on the pixel signals of the corresponding columns. After the analog-to-digital conversion, the column processing unit 250 performs cord-related double sampling (CDS) processing and supplies the processed digital signal to the signal processing unit 270.

[0027] The signal processing unit 270 performs various signal processing on the digital signals from the column processing unit 250. The signal processing unit 270 also generates a saturation pattern for each pixel indicating whether or not the pixel signal is saturated, adds it to the processed image data, and outputs it to the interface 280.

[0028] Interface 280 transmits and receives data to and from SoC 120. This interface 280 transmits image data with a saturation pattern added to it to SoC 120.

[0029] [Example of Pixel Array Configuration] Figure 3 is a plan view showing an example of the configuration of the pixel array section 240 in the first embodiment of this technology. The pixel array section 240 is divided into a predetermined number of FD shared blocks 241.

[0030] Each of the FD shared blocks 241 contains a 2x2 arrangement of Mg pixels that receive magenta visible light, two Ye pixels that receive yellow visible light, and a Cy pixel that receives cyan visible light. These four pixels share the FD.

[0031] Furthermore, it is possible to arrange Mg pixels, Ye pixels, and Cy pixels in each 2x2 block while not sharing the floppy disk (FD). Also, the number of pixels sharing the FD is not limited to four. For example, it is possible to arrange Mg pixels, Ye pixels, and Cy pixels in each 2x2 block while sharing the FD with eight pixels.

[0032] The arrangement of pixels (in other words, color filters) shown in the figure is called the MgYeYeCy arrangement.

[0033] Figure 4 is a circuit diagram showing one example configuration of the FD shared block 241 in the first embodiment of this technology. This FD shared block 241 comprises photodiodes 311 to 314, transfer transistors 315 to 318, reset transistor 319, FD 320, amplification transistor 321, and selection transistor 322. For example, nMOS (n-channel Metal Oxide Semiconductor) transistors are used as transistors in the FD shared block 241.

[0034] Photodiodes 311, 312, 313, and 314 generate electric charge through photoelectric conversion.

[0035] The transfer transistor 315 transfers electric charges from the photodiode 311 to the FD 320 in accordance with a transfer signal TGa from the vertical drive unit 210. The transfer transistor 316 transfers electric charges from the photodiode 312 to the FD 320 in accordance with a transfer signal TGb from the vertical drive unit 210. The transfer transistor 317 transfers electric charges from the photodiode 313 to the FD 320 in accordance with a transfer signal TGc from the vertical drive unit 210. The transfer transistor 318 transfers electric charges from the photodiode 314 to the FD 320 in accordance with a transfer signal TGd from the vertical drive unit 210.

[0036] The reset transistor 319 extracts electric charges from the FD 320 to perform initialization in accordance with a reset signal RST from the vertical drive unit 210. The FD 320 accumulates electric charges and generates a voltage corresponding to the amount of electric charges.

[0037] The amplification transistor 321 constitutes a source follower circuit, and outputs a voltage corresponding to the voltage of the FD 320 at the gate from the source. The selection transistor 322 outputs the pixel signal of the voltage from the amplification transistor 321 to the column processing unit 250 via the vertical signal line VSL in accordance with a selection signal SEL from the vertical drive unit 210.

[0038] [Configuration Example of Column Processing Unit] FIG. 5 is a block diagram showing a configuration example of the column processing unit according to the first embodiment of the present technology. In the column processing unit 250, an ADC 251 and a latch circuit 255 are arranged for each column.

[0039] The ADC 251 performs AD conversion on the pixel signal AIn from the vertical signal line VSL. The ADC 251 is, for example, a single-slope ADC, and includes a comparator 252 and a counter 253.

[0040] The comparator 252 compares the pixel signal AIn with a ramp signal RMP from the DAC 230. The comparator 252 supplies a comparison result VCO to the counter 253.

[0041] The counter 253 counts a count value over a period until the comparison result VCO is inverted. This counter 253 supplies a digital signal Dout indicating the count value to the latch circuit 255.

[0042] The latch circuit 255 holds the digital signal Dout and outputs it to the signal processing unit 270 under the control of the horizontal driving unit 260.

[0043] FIG. 6 is an example of a tone diagram according to the first embodiment of the present technology. The + direction of the horizontal axis in the figure indicates the red direction, and the - direction indicates the green direction. The + direction of the vertical axis indicates the yellow direction, and the - direction indicates the blue direction. The same applies to the subsequent tone diagrams.

[0044] Further, the black square plot indicates the hue of the image output by the image sensor 200 when red visible light from the LED traffic signal is incident. Note that since the hue behavior of the LED traffic signal is substantially the same as that of a bulb-type traffic signal, the description will be given for the LED traffic signal. The black circle plot indicates the hue of the image output by the image sensor 200 when green visible light from the LED traffic signal is incident. The black triangle plot indicates the hue of the image output by the image sensor 200 when blue visible light is incident. The same applies to the subsequent tone diagrams.

[0045] Further, the white square plot indicates the hue of the image output by the image sensor 200 when yellow visible light from the LED traffic signal is incident. The white circle plot indicates the hue of the image output by the image sensor 200 when cyan visible light is incident. The white triangle plot indicates the hue of the image output by the image sensor 200 when magenta visible light is incident. The same applies to the subsequent tone diagrams.

[0046] As exemplified in the figure, the three channels of magenta, yellow, and cyan are evenly separated on the tone diagram, so that no color shift occurs. For example, when high-luminance yellow is incident, only the Ye pixels are saturated.

[0047] Furthermore, regarding mixed colors, for example, when red visible light is incident, since red is a mixture of magenta and yellow, high brightness will cause the Mg and Ye pixels to saturate. Green is a mixture of cyan and yellow, so high brightness will cause the Cy and Ye pixels to saturate. Blue is a mixture of cyan and magenta, so high brightness will cause the Cy and Mg pixels to saturate.

[0048] As described above, even with high brightness levels for red, yellow, and green, no color shift occurs, and the combination of pixels that saturate in mixed colors (red or green) is limited. Therefore, SoC120 can accurately determine the color of a traffic light based on its saturation pattern.

[0049] Here, we will consider a typical Bayer array image sensor as a comparative example. In a Bayer array, R pixels that receive red light, G pixels that receive green light, and B pixels that receive blue light are arranged in a sequence.

[0050] Figure 7 shows an example of a color diagram in a comparative example. As illustrated in the figure, a Bayer array may experience color shift. Because a portion of the wavelength range of the red (R) pixel overlaps with the wavelength range of the green (G) pixel, for example, when high-luminance red light is incident, the R and G pixels may become saturated and shift to yellow. In this case, the red light of a traffic light may be mistakenly perceived as yellow.

[0051] [Example of Signal Processing Unit and SoC Configuration] Figure 8 is a block diagram showing an example configuration of the signal processing unit 270 and SoC 120 in the first embodiment of this technology. The signal processing unit 270 includes a saturation pattern generation unit 271 and a subsequent processing unit 272. The SoC 120 includes a color determination unit 121 and an image recognition unit 122.

[0052] The saturation pattern generation unit 271 generates a saturation pattern for each FD shared block based on the AD-converted pixel signal from the column processing unit 250. For each pixel, the saturation pattern generation unit 271 determines whether or not the pixel is saturated based on whether the value of the AD-converted pixel signal (in other words, the count value of the counter 253) is at its maximum value. The saturation pattern generation unit 271 then generates a saturation pattern as a bit sequence that arranges the determination results for each Mg pixel, Ye pixel, and Cy pixel for each FD shared block. The saturation pattern generation unit 271 then supplies each of these saturation patterns to the subsequent processing unit 272.

[0053] The subsequent processing unit 272 performs various processes on the image data, such as white balance correction and conversion from MgYeCy to RGB. The subsequent processing unit 272 adds a saturation pattern to the processed image data and transmits it to the color determination unit 121 via the interface 280.

[0054] The color determination unit 121 determines the color of each FD shared block. Based on the saturation pattern, the color determination unit 121 determines whether one or more pixels in each FD shared block are saturated. If one or more pixels are saturated, the color determination unit 121 determines the color of the entire FD shared block based on the saturation pattern and supplies the determination result to the image recognition unit 122. If there are no saturated pixels, the color determination unit 121 supplies the pixel signals within the FD block directly to the image recognition unit 122.

[0055] The image recognition unit 122 performs image recognition based on image data. When the color determination unit 121 determines the color of an FD block, all four pixels within that block are processed as being of the determined color.

[0056] Figure 9 shows a saturation pattern in the first embodiment of this technology, and an example of the color determination result for each saturation pattern.

[0057] If the Mg and Ye pixels in the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "110" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be red.

[0058] Furthermore, if only the Ye pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "010" in binary and transmits it to the SoC 120. In this case, the SoC 120 determines that the color of that block is yellow.

[0059] Furthermore, if the Ye and Cy pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "011" in binary and transmits it to the SoC 120. In this case, the SoC 120 determines the color of that block to be green.

[0060] Furthermore, if the Mg and Cy pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "101" in binary and transmits it to the SoC 120. In this case, the SoC 120 determines the color of that block to be blue.

[0061] If there are no saturated pixels within the FD shared block, a saturated pattern of "000" in binary will be output. Alternatively, no saturated pattern will be output for that block.

[0062] Furthermore, the image sensor 200 can output a saturation pattern of "100" in binary when only the Mg pixels are saturated, and a saturation pattern of "001" in binary when only the Cy pixels are saturated.

[0063] As mentioned above, the MgYeYeCy array suppresses color shift. Therefore, as illustrated in the figure, the image sensor 200 generates and outputs a saturated pattern, allowing the SoC 120 to accurately identify the colors of traffic lights and the like.

[0064] [Example of Camera Module Operation] Figure 10 shows an example of the operation of the camera module 100 in the first embodiment of this technology. This operation is started, for example, when a predetermined application for capturing image data is executed.

[0065] The image sensor 200 exposes each row using a rolling shutter method, reads out the pixel signals, and generates image data (step S901). Then, the image sensor 200 generates a saturation pattern for each FD shared block (step S902), adds it to the image data, and outputs it to the SoC 120 (step S903).

[0066] The SoC120 determines the color for each FD shared block based on the saturation pattern (step S904), and performs image recognition based on the determination result (step S905). After step S905, processing of the image data is completed.

[0067] When multiple image data are captured and processed in succession, steps S901 to S905 are repeatedly executed in synchronization with the vertical synchronization signal.

[0068] Thus, according to the first embodiment of this technology, since the pixels (color filters) are arranged in an MgYeYeCy array, color shift can be suppressed. Furthermore, because the image sensor 200 outputs a saturation pattern, the SoC 120 can accurately determine the color of traffic lights and the like.

[0069] <2. Second Embodiment> In the first embodiment described above, pixels (color filters) were arranged using an MgYeYeCy array, but the image sensor is not limited to this array. The image sensor 200 in this second embodiment differs from the first embodiment in that the arrangement of the color filters has been changed.

[0070] Figure 11 is a plan view showing one example configuration of the pixel array section 240 in a second embodiment of the present technology. The pixel array section 240 in this second embodiment differs from the first embodiment in that, for each FD shared block 241, an Rc pixel, two Gc pixels, and a Bc pixel are arranged.

[0071] Rc pixels, Gc pixels, and Bc pixels have properties similar to cone cells and are pixels that receive red, green, and blue visible light.

[0072] Figure 12 shows an example of the wavelength characteristics of a color filter in a second embodiment of the present technology. In the figure, the vertical axis represents luminance and the horizontal axis represents wavelength. The thick line shows the wavelength characteristics of the color filter for a Gc pixel, and the thick dotted line shows the wavelength characteristics of the color filter for an Rc pixel. The dashed line shows the wavelength characteristics of the color filter for a Bc pixel, and the thin dotted line shows the wavelength characteristics of yellow light from an LED traffic light.

[0073] As illustrated in the figure, the overlapping wavelength range between the color filter wavelengths of Rc pixels and Gc pixels is wider than that of typical R and G pixels. For example, the peak wavelength of the color filter for Rc pixels is 600 nanometers, and the peak wavelength of the color filter for Gc pixels is 550 nanometers. Also, the peak wavelength of the color filter for Bc pixels is 450 nanometers.

[0074] Furthermore, as indicated by the arrows, there is an overlap in the wavelength characteristics of the yellow and Gc pixel color filters.

[0075] Figure 13 is an example of a color diagram in the second embodiment of this technology. As illustrated in the figure, by arranging pixels with characteristics similar to cone cells, the separation of red, yellow, and green from LED traffic lights at saturation is improved, and color shift can be suppressed.

[0076] Figure 14 shows a saturation pattern in the second embodiment of this technology, and an example of the color determination result for each saturation pattern.

[0077] If only the Rc pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "100" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be red.

[0078] If only the Gc pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "010" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be green.

[0079] If only the Bc pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "001" in binary and sends it to the SoC 120. In this case, the SoC 120 determines that the color of that block is blue.

[0080] If the Rc and Gc pixels in the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "110" in binary and transmits it to the SoC 120. In this case, the SoC 120 determines the color of that block to be yellow.

[0081] Thus, according to the second embodiment of this technology, since pixels with characteristics similar to cone cells are arranged, color shift can be suppressed.

[0082] <3. Third Embodiment> In the first embodiment described above, pixels (color filters) were arranged using an MgYeYeCy array, but the image sensor is not limited to this array. The image sensor 200 in this third embodiment differs from the first embodiment in that the arrangement of the color filters has been changed.

[0083] Figure 15 is a plan view showing one example configuration of the pixel array section 240 in a third embodiment of the present technology. The pixel array section 240 in this third embodiment differs from the first embodiment in that, for each FD shared block 241, Mg' pixels, two G' pixels, and B pixels are arranged.

[0084] Mg', G', and B pixels receive magenta, green, and blue visible light, respectively. However, Mg' and G' pixels have different wavelength characteristics than typical Mg and G pixels.

[0085] The arrangement of pixels (color filters) shown in the figure is referred to as the MgGGB arrangement.

[0086] Figure 16 shows an example of the wavelength characteristics of a color filter in the third embodiment of this technology. In the figure, the vertical axes of a and b represent luminance, and the horizontal axis represents wavelength.

[0087] In the figure, the thick line a represents the wavelength characteristics of the color filter of the G' pixel, the dotted line represents the wavelength characteristics of the color filter of the Mg' pixel, and the dashed line represents the wavelength characteristics of the color filter of the B pixel.

[0088] The wavelength range of the color filter for the Mg' pixel is 400 to 550 nanometers and 560 to 700 nanometers. The wavelength range of the color filter for the G' pixel is 460 to 610 nanometers. The wavelength range of the color filter for the B pixel is 400 to 500 nanometers.

[0089] Furthermore, in the same figure, the thick line b represents the wavelength characteristics of yellow light from the LED traffic light, and the dotted line represents the wavelength characteristics of red light from the LED traffic light.

[0090] Figure 17 is an example of a color diagram in the third embodiment of this technology. As illustrated in the figure, the MgGGB array provides good separation of the green, yellow, and red colors of the LED traffic light when saturated, and no color shift occurs.

[0091] Figure 18 shows a saturation pattern in the third embodiment of this technology, and an example of the color determination result for each saturation pattern.

[0092] If the Mg' and G' pixels in the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "110" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be yellow.

[0093] If only the G' pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "010" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be green.

[0094] If only the Mg' pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "100" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be red.

[0095] If only the B pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "001" in binary and sends it to the SoC 120. In this case, the SoC 120 determines that the color of that block is blue.

[0096] Thus, according to the third embodiment of this technology, since the Mg' pixels, G' pixels, and B pixels are arranged, color shift can be suppressed.

[0097] <4. Fourth Embodiment> In the first embodiment described above, pixels (color filters) were arranged using an MgYeYeCy array, but the embodiment is not limited to this array. The image sensor 200 in this fourth embodiment differs from the first embodiment in that the arrangement of the color filters has been changed.

[0098] Figure 19 is a plan view showing one example configuration of the pixel array section 240 in the fourth embodiment of this technology. The pixel array section 240 in this fourth embodiment differs from the first embodiment in that Rp pixels, Yo pixels, Gp pixels, and B pixels are arranged in each FD shared block 241.

[0099] Yo pixels are pixels that receive visible light of a color between yellow and orange. Gp pixels are pixels that receive green visible light whose wavelength range does not overlap with that of Yo pixels. Rp pixels are pixels that receive red visible light whose wavelength range does not overlap with that of Yo pixels.

[0100] The arrangement of pixels (color filters) shown in the figure is called the RpYoGpB arrangement.

[0101] Figure 20 is a graph showing an example of the wavelength characteristics of an LED traffic light in the fourth embodiment of this technology. In the figure, the vertical axis represents luminance and the horizontal axis represents wavelength. The thick line in the figure shows the wavelength characteristics of green from the LED traffic light. The dashed line in the figure shows the wavelength characteristics of yellow from the LED traffic light. The dotted line in the figure shows the wavelength characteristics of red from the LED traffic light.

[0102] As illustrated in the figure, the brightness of green from an LED traffic light is higher than that of yellow, and the brightness of yellow is higher than that of red. Also, the peak wavelength of green from an LED traffic light is, for example, 507 nanometers (nm). The peak wavelength of yellow from an LED traffic light is, for example, 591 nanometers (nm). The peak wavelength of red from an LED traffic light is, for example, 628 nanometers (nm).

[0103] Furthermore, the color filter for the B pixel transmits wavelengths in the 400 to 500 nanometer (nm) range. The color filter for the Gp pixel transmits wavelengths in the 500 to 570 nanometer (nm) range. The color filter for the Yo pixel transmits wavelengths in the 570 to 610 nanometer (nm) range. The color filter for the Rp pixel transmits wavelengths in the 610 to 700 nanometer (nm) range.

[0104] On the other hand, in the comparative example of the Bayer array, the color filter of the B pixel transmits wavelengths in the 400 to 500 nanometer (nm) range. The color filter of the G pixel transmits wavelengths in the 500 to 600 nanometer (nm) range. The color filter of the R pixel transmits wavelengths in the 600 to 700 nanometer (nm) range.

[0105] Figure 21 is a graph showing an example of the wavelength characteristics of an incandescent light bulb traffic signal in the fourth embodiment of this technology. In the figure, the vertical axis represents luminance and the horizontal axis represents wavelength. The thick line in the figure shows the wavelength characteristics of green from the incandescent light bulb traffic signal. The dashed line in the figure shows the wavelength characteristics of yellow from the incandescent light bulb traffic signal. The dotted line in the figure shows the wavelength characteristics of red from the incandescent light bulb traffic signal.

[0106] As illustrated in the diagram, the brightness of the green light from an incandescent light bulb traffic signal is higher than that of the yellow or red lights.

[0107] Figure 22 shows an example of saturation characteristics in the fourth embodiment and comparative example of this technology. In figure a, "Sat" indicates that the signal is likely to saturate at the pixel when the illuminance is high. "Sec" indicates that a secondary component is obtained. "Pr" indicates that although saturation does not occur, a primary component with a higher level than the secondary component is obtained. The same applies in subsequent examples.

[0108] As illustrated in Figure a, in the fourth embodiment of the RpYoGpB array, when the red light of an LED traffic light is incident, the Rp pixels tend to saturate, and when the yellow light of an LED traffic light is incident, the Yo pixels tend to saturate. Also, when the green light of an LED traffic light is incident, the B pixels and Gp pixels tend to saturate. Furthermore, when the red light of an incandescent light-type traffic light is incident, a primary component is generated in the Rp pixels, and when the yellow light of an incandescent light-type traffic light is incident, primary components are generated in the Yo pixels and Rp pixels. When the green light of an incandescent light-type traffic light is incident, a primary component is generated in the Gp pixels.

[0109] On the other hand, in a comparative example of a Bayer array, when the red light from an LED traffic light is incident, the R and G pixels tend to saturate, and when the yellow light from an LED traffic light is incident, the R and G pixels tend to saturate. Also, when the green light from an LED traffic light is incident, the G and B pixels tend to saturate, and when the white light from a headlight is incident, the B pixels tend to saturate. Furthermore, when the red light from an incandescent light bulb is incident, a primary component is generated in the R pixel, and when the yellow light from an incandescent light bulb is incident, a primary component is generated in the G and R pixels. When the green light from an incandescent light bulb is incident, a primary component is generated in the G pixel.

[0110] As illustrated in the figure, even when light from an incandescent light bulb traffic signal is incident, the signal does not saturate in either the RpYoGpB array or the Bayer array. Therefore, in image recognition, the color of the incandescent light bulb traffic signal can be accurately identified.

[0111] However, in a Bayer array, when red or yellow light from an LED traffic light is incident, the R and G pixels become saturated, potentially causing a color shift and making color identification difficult. In contrast, in an RpYoGpB array, different pixels saturate depending on whether the LED light is red or yellow. Therefore, color shift is suppressed, and the SoC120 software performing image recognition can accurately distinguish between the red and yellow light of the LED traffic light. Furthermore, when green light from an LED traffic light is incident, the B and Gp pixels become saturated, but the pixels that saturate are different from those that saturate when yellow or red light is incident. Therefore, the software can accurately distinguish between red / yellow and green by referring to the saturation pattern.

[0112] Furthermore, in the same figure, b is a diagram showing the saturation pattern in the fourth embodiment and an example of the color determination result for each saturation pattern.

[0113] If only the B pixels within the FD shared block are saturated, the image sensor 200 generates a saturation pattern of "1000" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be blue.

[0114] If only the Gp pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "0100" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be green.

[0115] If only the Yo pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "0010" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be yellow.

[0116] If only the Rp pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "0001" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be red.

[0117] When the B pixel and Gp pixel in the FD shared block are saturated, the image sensor 200 generates a saturation pattern of "1100" in binary and transmits it to the SoC 120. In this case, the SoC 120 determines that the color of that block is green.

[0118] Further, the signal processing unit 270 in the image sensor 200 can output the saturation pattern, and also convert each signal of the RpYoGpB array into R, G and B signals by, for example, the following equation. R = a １ * Rp + a ２ * Yo ... Equation 1 G = b １ * Yo + b ２ * Gp ... Equation 2 B = c １ * Gp + c ２ * B ... Equation 3 In Equations 1 to 3, the coefficients a １ , a ２ , b １ , b ２ , c １ , c ２ are predetermined real numbers.

[0119] As described above, according to the fourth embodiment of the present technology, since the pixels are arranged in the RpYoGpB array, color shift can be suppressed when red or yellow light from an LED traffic signal enters.

[0120] <5. Fifth Embodiment> In the above-described first embodiment, the pixels (color filters) are arranged in the MgYeYeCy array, but the arrangement is not limited to this. The image sensor 200 in the fifth embodiment differs from the first embodiment in that the arrangement of the color filters is modified.

[0121] FIG. 23 is a plan view showing a configuration example of the pixel array unit 240 according to the fifth embodiment of the present technology. The pixel array unit 240 in the fifth embodiment differs from the first embodiment in that a Bcy pixel, two Yog pixels, and an Rp pixel are arranged for each FD shared block 241.

[0122] Bcy pixels are pixels that receive visible light of a color between blue and cyan. Yog pixels are pixels that receive light of a color between yellow and orange. Rp pixels are pixels that receive red visible light whose wavelength range does not overlap with that of Yog pixels.

[0123] The arrangement of pixels (color filters) shown in the figure is called the RpYogBcy arrangement.

[0124] Figure 24 shows an example of the wavelength range of a color filter in the fifth embodiment of this technology. The color filter for the Bcy pixel transmits wavelengths in the 400 to 510 nanometer (nm) range. The color filter for the Yog pixel transmits wavelengths in the 510 to 610 nanometer (nm) range. The color filter for the Rp pixel transmits wavelengths in the 610 to 700 nanometer (nm) range.

[0125] Figure 25 shows an example of saturation characteristics in a fifth embodiment and comparative example of the present technology.

[0126] As illustrated in figure a, in the fifth embodiment of the RpYogBcy array, when the red light of an LED traffic light is incident, the Rp pixels tend to saturate, and when the yellow light of an LED traffic light is incident, the Yog pixels tend to saturate. Also, when the green light of an LED traffic light is incident, the Bcy pixels tend to saturate. Furthermore, when the red light of an incandescent light bulb traffic light is incident, a primary component is generated in the Rp pixels, and when the yellow light of an incandescent light bulb traffic light is incident, primary components are generated in both the Yog pixels and the Rp pixels. When the green light of an incandescent light bulb traffic light is incident, a primary component is generated in the Yog pixels.

[0127] Thus, in the RpYogBcy array, different pixels saturate for each color of the LED traffic light. As a result, color shift is suppressed, and the software can accurately identify each color of the LED traffic light.

[0128] Furthermore, in the same figure, b is a diagram showing the saturation pattern in the fifth embodiment and an example of the color determination result for each saturation pattern.

[0129] If only the Bcy pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "100" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be green.

[0130] If only the Yog pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "010" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be yellow.

[0131] If only the Rp pixels within the FD shared block become saturated, the image sensor 200 generates a saturation pattern of "001" in binary and sends it to the SoC 120. In this case, the SoC 120 determines the color of that block to be red.

[0132] Thus, according to the fifth embodiment of this technology, since the pixels are arranged in an RpYgBcy array, color shift can be suppressed.

[0133] <6. Examples of Application to Mobile Devices> The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device, such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0134] Figure 26 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0135] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 26, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0136] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0137] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0138] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0139] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0140] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0141] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0142] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0143] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0144] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 26, the output devices include an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0145] Figure 27 shows an example of the installation position of the imaging unit 12031.

[0146] In Figure 27, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0147] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0148] Figure 27 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0149] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0150] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0151] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0152] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0153] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to, for example, the imaging unit 12031 of the configuration described above. Specifically, the image sensor 200 in Figure 2 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, color shift can be suppressed, and the software can accurately identify the color of the LED traffic light.

[0154] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0155] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0156] Furthermore, this technology can also be configured as follows: (1) An image sensor comprising: a pixel array section in which a predetermined number of pixels are arranged in an arrangement that does not correspond to a Bayer array; and a signal processing section that generates and outputs a saturation pattern indicating whether or not the pixel signal has saturated for each pixel. (2) The image sensor according to (1), wherein the pixels include an Mg pixel that receives magenta visible light, a Ye pixel that receives yellow visible light, and a Cy pixel that receives cyan visible light. (3) The image sensor according to (1), wherein the pixels include an Rc pixel that receives red visible light with a peak wavelength of 600 nanometers, a Gc pixel that receives green visible light with a peak wavelength of 550 nanometers, and a Bc pixel that receives blue visible light with a peak wavelength of 450 nanometers. (4) The image sensor according to (1), wherein the pixels include an Mg' pixel that receives magenta visible light, a G' pixel that receives green visible light, and a B pixel that receives blue visible light. (5) The image sensor according to (4), wherein the wavelength range of the color filter of the Mg' pixel is 400 to 550 nanometers and 560 to 700 nanometers, and the wavelength range of the color filter of the G' pixel is 460 to 610 nanometers. (6) The image sensor according to (1), wherein the pixels include a B pixel that receives blue visible light, a Yo pixel that receives visible light of a color between yellow and orange, a Gp pixel that receives green visible light whose wavelength range does not overlap with that of the Yo pixel, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yo pixel. (7) The image sensor according to (6), wherein the wavelength range of the color filter of the Gp pixel is 500 to 570 nanometers, the wavelength range of the color filter of the Yo pixel is 570 to 610 nanometers, and the wavelength range of the color filter of the Rp pixel is 610 to 700 nanometers. (8) The image sensor according to (1), wherein the pixels include a Bcy pixel that receives visible light of a color between blue and cyan, a Yog pixel that receives light of a color between yellow and orange, and an Rp pixel that receives visible light of red whose wavelength range does not overlap with that of the Yog pixel.(9) The image sensor according to (8), wherein the wavelength range of the color filter of the Bcy pixel is 400 to 510 nanometers, the wavelength range of the color filter of the Yog pixel is 500 to 610 nanometers, and the wavelength range of the color filter of the Rp pixel is 610 to 700 nanometers. (10) An image sensor comprising a pixel array portion arranged with Mg' pixels that receive magenta visible light, G' pixels that receive green visible light, and B pixels that receive blue visible light. (11) An image sensor comprising a pixel array portion arranged with B pixels that receive blue visible light, Yo pixels that receive visible light of a color between yellow and orange, Gp pixels that receive green visible light whose wavelength range does not overlap with that of the Yo pixels, and Rp pixels that receive red visible light whose wavelength range does not overlap with that of the Yo pixels. (12) An image sensor comprising a pixel array section having a Bcy pixel that receives visible light of a color between blue and cyan, a Yog pixel that receives light of a color between yellow and orange, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yog pixel.

[0157] 100 Camera module 110 Optical unit 120 SoC 121 Color determination unit 122 Image recognition unit 200 Image sensor 210 Vertical drive unit 220 System control unit 230 DAC 240 Pixel array unit 241 FD shared block 250 Column processing unit 251 ADC 252 Comparator 253 Counter 255 Latch circuit 260 Horizontal drive unit 270 Signal processing unit 271 Saturation pattern generation unit 272 Post-processing unit 280 Interface 311-314 Photodiode 315-318 Transfer transistor 319 Reset transistor 320 FD 321 Amplification transistor 322 Selection transistor 12031 Imaging unit

Claims

1. An image sensor comprising: a pixel array section in which a predetermined number of pixels are arranged in an arrangement that does not correspond to a Bayer array; and a signal processing section that generates and outputs a saturation pattern indicating whether or not the pixel signal has saturated for each pixel.

2. The image sensor according to claim 1, wherein the pixels include a Mg (Magenta) pixel that receives magenta visible light, a Ye (Yellow) pixel that receives yellow visible light, and a Cy (Cyan) pixel that receives cyan visible light.

3. The image sensor according to claim 1, wherein the pixels include an Rc pixel that receives red visible light with a peak wavelength of 600 nanometers, a Gc pixel that receives green visible light with a peak wavelength of 550 nanometers, and a Bc pixel that receives blue visible light with a peak wavelength of 450 nanometers.

4. The image sensor according to claim 1, wherein the pixels include an Mg' pixel that receives magenta visible light, a G' pixel that receives green visible light, and a B (Blue) pixel that receives blue visible light.

5. The image sensor according to claim 4, wherein the wavelength range of the color filter of the Mg' pixel is 400 to 550 nanometers and 560 to 700 nanometers, and the wavelength range of the color filter of the G' pixel is 460 to 610 nanometers.

6. The image sensor according to claim 1, wherein the pixels include a B pixel that receives blue visible light, a Yo pixel that receives visible light of a color between yellow and orange, a Gp pixel that receives green visible light whose wavelength range does not overlap with that of the Yo pixel, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yo pixel.

7. The image sensor according to claim 6, wherein the wavelength range of the color filter of the Gp pixel is 500 to 570 nanometers, the wavelength range of the color filter of the Yo pixel is 570 to 610 nanometers, and the wavelength range of the color filter of the Rp pixel is 610 to 700 nanometers.

8. The image sensor according to claim 1, wherein the pixels include a Bcy pixel that receives visible light of a color between blue and cyan, a Yog pixel that receives light of a color between yellow and orange, and an Rp pixel that receives red visible light whose wavelength range does not overlap with that of the Yog pixel.

9. The image sensor according to claim 8, wherein the wavelength range of the color filter of the Bcy pixel is 400 to 510 nanometers, the wavelength range of the color filter of the Yog pixel is 500 to 610 nanometers, and the wavelength range of the color filter of the Rp pixel is 610 to 700 nanometers.

10. An image sensor comprising a pixel array section arranged with Mg' pixels that receive magenta visible light, G' pixels that receive green visible light, and B pixels that receive blue visible light.

11. An image sensor comprising a pixel array section arranged with B pixels that receive blue visible light, Yo pixels that receive visible light of a color between yellow and orange, Gp pixels that receive green visible light whose wavelength range does not overlap with that of the Yo pixels, and Rp pixels that receive red visible light whose wavelength range does not overlap with that of the Yo pixels.

12. An image sensor comprising a pixel array section in which Bcy pixels that receive visible light of a color between blue and cyan, Yog pixels that receive light of a color between yellow and orange, and Rp pixels that receive red visible light whose wavelength range does not overlap with that of the Yog pixels are arranged.

Images