Polarization sensor

Abstract

The present disclosure relates to a polarization sensor capable of acquiring high-quality RGB information and simultaneously acquiring polarization information. According to the polarization sensor, a first light splitting unit which performs photoelectric conversion on light with a first wavelength of incident light and a polarization filter which allows light with a specific polarization direction of the incident light to pass through are vertically stacked; and a semiconductor substrate that photoelectrically converts light having a second wavelength of the incident light that has passed through the polarization filter. The present disclosure can be applied, for example, to a polarization sensor or the like that simultaneously acquires RGB information and polarization information.

Description

Technical Field

[0001] This disclosure relates to polarization sensors, and more specifically, to polarization sensors capable of acquiring high-quality RGB information and simultaneously acquiring polarization information. Background Technology

[0002] Conventionally, there exists an imaging sensor that can simultaneously acquire RGB images and polarization information by providing a polarizer (polarization filter) and a color filter on the imaging surface of a sensor unit that generates an image signal based on the subject light via photoelectric conversion (see, for example, Patent Document 1).

[0003] Citation List

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent No. 6673327 Summary of the Invention

[0006] The problem to be solved by the present invention

[0007] However, in a conventional configuration, the image quality of the RGB image is degraded because the light passing through the polarizer is incident on the photoelectric conversion unit.

[0008] This disclosure is made in view of the circumstances and its purpose is to enable the simultaneous acquisition of RGB information and polarization information, as well as the acquisition of high-quality RGB information.

[0009] Solution to the problem

[0010] According to one aspect of this disclosure, the polarization sensor is a sensor configured by vertically stacking the following components:

[0011] A first beam splitting unit is configured to perform photoelectric conversion on incident light having a first wavelength;

[0012] A polarization filter, through which incident light with a specific polarization direction passes; and

[0013] A semiconductor substrate configured to perform photoelectric conversion on incident light that has passed through a polarization filter and has a second wavelength.

[0014] In one aspect of this disclosure, a polarization sensor is configured by vertically stacking the following components: a first beam splitter configured to perform photoelectric conversion on incident light having a first wavelength; a polarization filter through which incident light having a specific polarization direction passes; and a semiconductor substrate configured to perform photoelectric conversion on incident light having a second wavelength after passing through the polarization filter.

[0015] A polarization sensor can be a standalone device or a module incorporated into another device. Attached Figure Description

[0016] Figure 1 This is a block diagram of a polarization sensor as a comparative example to be compared with the polarization sensor disclosed herein.

[0017] Figure 2 It is used for explanation Figure 1 A diagram showing the pixel configuration of the imaging unit of the polarization sensor in the image.

[0018] Figure 3 It is used for explanation Figure 1 A diagram showing the pixel configuration of the imaging unit of the polarization sensor in the image.

[0019] Figure 4 This is a block diagram illustrating a schematic configuration of a polarization sensor according to the present disclosure.

[0020] Figure 5 This is a diagram used to illustrate the demosaicing process applied to the original image.

[0021] Figure 6 This diagram illustrates the demosaicing process applied to the original polarized image.

[0022] Figure 7 This is a diagram illustrating a first configuration example of the imaging unit.

[0023] Figure 8 This is a diagram illustrating a first configuration example of the imaging unit.

[0024] Figure 9 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the first configuration example.

[0025] Figure 10 This is a table comparing sensitivity and resolution between a first configuration example and a comparative example of the imaging unit.

[0026] Figure 11 This is a diagram illustrating a second configuration example of the imaging unit.

[0027] Figure 12 This is a diagram illustrating a second configuration example of the imaging unit.

[0028] Figure 13 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the second configuration example.

[0029] Figure 14 This is a table comparing sensitivity and resolution between a second configuration example and a comparative example of the imaging unit.

[0030] Figure 15 This is a diagram illustrating a third configuration example of the imaging unit.

[0031] Figure 16 This is a diagram illustrating a third configuration example of the imaging unit.

[0032] Figure 17 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the third configuration example.

[0033] Figure 18 This is a table comparing sensitivity and resolution between the third configuration example and the comparative example of the imaging unit.

[0034] Figure 19 This is a diagram illustrating a fourth configuration example of the imaging unit.

[0035] Figure 20 This is a diagram illustrating a fourth configuration example of the imaging unit.

[0036] Figure 21 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the fourth configuration example.

[0037] Figure 22 This is a table comparing sensitivity and resolution between the fourth configuration example and the comparative example of the imaging unit.

[0038] Figure 23 This is a diagram illustrating a fifth configuration example of the imaging unit.

[0039] Figure 24 This is a diagram illustrating a fifth configuration example of the imaging unit.

[0040] Figure 25 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the fifth configuration example.

[0041] Figure 26 This is a table comparing sensitivity and resolution between the fifth configuration example and the comparative example of the imaging unit.

[0042] Figure 27 This is a diagram illustrating a sixth configuration example of the imaging unit.

[0043] Figure 28 This is a diagram illustrating a sixth configuration example of the imaging unit.

[0044] Figure 29 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the sixth configuration example.

[0045] Figure 30This is a table comparing sensitivity and resolution between the sixth configuration example and the comparative example of the imaging unit.

[0046] Figure 31 This is a diagram illustrating a seventh configuration example of the imaging unit.

[0047] Figure 32 This is a diagram illustrating a seventh configuration example of the imaging unit.

[0048] Figure 33 This is a block diagram illustrating a configuration example of the signal processing unit and the imaging unit according to the seventh configuration example.

[0049] Figure 34 This is a table comparing sensitivity and resolution between the seventh configuration example and the comparative example of the imaging unit.

[0050] Figure 35 It is a diagram summarizing the features of the comparison examples and the first to third configuration examples.

[0051] Figure 36 This is a diagram summarizing the features of the fourth through sixth configuration examples.

[0052] Figure 37 This is a diagram summarizing the features of the seventh configuration example. Detailed Implementation

[0053] In the following description, a mode (hereinafter referred to as an implementation) for carrying out the technology of this disclosure will be described with reference to the accompanying drawings. The description will be given in the following order.

[0054] 1. Configuration example of the polarization sensor used as a comparative example in this disclosure

[0055] 2. Block diagram of the polarization sensor disclosed herein

[0056] 3. Example of the first configuration of the imaging unit

[0057] 4. Example of a second configuration for the imaging unit

[0058] 5. Example of a third configuration for the imaging unit

[0059] 6. Example of a fourth configuration for the imaging unit

[0060] 7. Example of the fifth configuration of the imaging unit

[0061] 8. Sixth configuration example of the imaging unit

[0062] 9. Seventh Configuration Example of Imaging Unit

[0063] 10. Summary of the first to seventh configuration examples of the imaging unit

[0064] Note that in the accompanying drawings referenced in the following description, identical or similar parts are indicated by identical or similar reference numerals, and their descriptions will not be repeated as appropriate. The drawings are schematic, and the relationships between thickness and planar dimensions, the ratio of thickness per layer, etc., differ from actual dimensions. Furthermore, in some cases, the drawings may include parts with different dimensional relationships and ratios.

[0065] Furthermore, in the following description, the definitions of directions such as up and down are used only for ease of explanation and do not limit the technical concept of this disclosure. For example, when the object is rotated 90° for observation, the top and bottom become the left and right sides, and when the object is rotated 180° for observation, the top and bottom are reversed.

[0066] <1. Configuration example of the polarization sensor used as a comparative example in this disclosure>

[0067] First, in order to compare with the polarization sensor of this disclosure, a polarization sensor will be described as a comparison example.

[0068] Figure 1 This is a block diagram of a polarization sensor as a comparative example to be compared with the polarization sensor disclosed herein.

[0069] The polarization sensor 10 includes at least an imaging unit 21 and an RGB / polarization information generation unit 22.

[0070] Imaging unit 21 includes a pixel array unit in which multiple pixels are arranged in a matrix, generates imaging information based on the amount of incident light, and outputs the imaging information to RGB / polarization information generation unit 22. Each pixel in the pixel array unit includes a photoelectric conversion unit, which performs photoelectric conversion on the incident light to generate a pixel signal corresponding to the amount of received light.

[0071] Reference Figure 2 and Figure 3 Describes the pixel configuration of imaging unit 21.

[0072] In the cross-sectional view, the pixels of the imaging unit 21 are configured by vertically stacking a color filter 31, a polarization filter (polarizer) 32, and a semiconductor substrate 33 in sequence from the light incident surface side. The semiconductor substrate 33 includes a silicon substrate using, for example, silicon (Si) as the semiconductor. A photodiode (PD) 41 is formed on the semiconductor substrate 33 as a photoelectric conversion unit.

[0073] like Figure 2As shown, the color filter 31 is arranged in a square Bayer array, in which the color filters for red (R), green (G), blue (B) are arranged in a 2×2 four-pixel unit, where the arrangement unit for one color is four pixels in a 2×2 array. The color filter array can be a Bayer array in which the color filters for G, B, R, and G are arranged in four pixels in a 2×2 array.

[0074] like Figure 2 As shown, the polarization filter 32 is configured by arranging polarizers with polarization directions (polarization angles) of 0 degrees, 45 degrees, 90 degrees and 135 degrees in four pixels of 2×2, and repeatedly arranging these polarizers as repeating units in the row and column directions of the pixel array unit.

[0075] In the imaging unit 21 configured as described above, for example, in Figure 3 In the pixel shown on the left, which has a color filter 31 with R, a pixel signal is generated by photoelectric conversion of light with an R color component having a predetermined deflection angle (polarization component) of 0 degrees, 45 degrees, 90 degrees, or 135 degrees as polarization angle. Additionally, for example, in... Figure 3 In the pixel shown on the right, which has a color filter 31 with G, a pixel signal is generated by photoelectric conversion of light with a G color component having a predetermined deflection angle (polarization component) of 0 degrees, 45 degrees, 90 degrees, or 135 degrees as polarization angle. Although not shown, in the pixel having a color filter 31 with B, a pixel signal is generated by photoelectric conversion of light with a B color component having a predetermined deflection angle (polarization component) of 0 degrees, 45 degrees, 90 degrees, or 135 degrees as polarization angle.

[0076] therefore, Figure 1 The imaging unit 21 outputs the original image as imaging information to the RGB / polarization information generation unit 22. In the original image, the pixel signals with any one of polarization components of 0 degrees, 45 degrees, 90 degrees and 135 degrees and any one of color components of R, G and B are arranged into a mosaic pattern.

[0077] The RGB / polarization information generation unit 22 performs demosaic processing on the raw image provided by the imaging unit 21 to generate a 3ch (channel) RGB image of R, G, and B and outputs it as RGB information. Furthermore, the RGB / polarization information generation unit 22 performs demosaic processing on the raw image provided by the imaging unit 21 to generate a total of 12ch (channel) color polarization image and outputs it as polarization information. This color polarization image includes 3ch RGB images of R, G, and B for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

[0078] In polarization sensor 10, photodiode 41 performs photoelectric conversion on light passing through polarization filter 32, which degrades the sensitivity of RGB information. Furthermore, R, G, and B are arranged in units of four pixels (2×2), but these four pixels have different polarization components. Therefore, the four pixels need to be combined, and the resolution is also degraded. Thus, in the polarization sensor 10 of the comparative example, the image quality of the RGB image is degraded.

[0079] <2. Block diagram of the polarization sensor disclosed herein>

[0080] Figure 4 This is a block diagram illustrating a schematic configuration of a polarization sensor according to the present disclosure.

[0081] Figure 4 The polarization sensor 100 includes an imaging unit 111, a signal processing unit 112, and an output circuit 113. The imaging unit 111 includes a pixel array unit 122, a vertical driving unit 123, an AD conversion unit 124, a horizontal driving unit 125, a timing control unit 126, etc., in which multiple pixels 121 are arranged in a matrix. The signal processing unit 112 includes a signal processing circuit 131 and a memory 132.

[0082] Pixel 121 includes a photoelectric conversion unit, a polarization filter, and multiple pixel transistors. The multiple pixel transistors include, for example, MOS transistors, as well as transfer transistors, amplification transistors, selection transistors, and reset transistors. In pixel 121, under the control of the vertical drive unit 123, a pixel signal corresponding to the amount of received light is generated and output to the AD conversion unit 124.

[0083] The vertical driving unit 123 includes, for example, a shift register and drives the pixels 121 row by row by applying driving pulses to each pixel 121 via pixel driving wiring (not shown). The vertical driving unit 123 sequentially selects and scans each pixel 121 of the pixel array unit 122 in the vertical direction row by row, and provides pixel signals based on signal charge to the AD conversion unit 124 via vertical signal lines (not shown) typically arranged in columns, the signal charge being generated according to the amount of incident light in the photoelectric conversion unit of each pixel 121.

[0084] The AD conversion unit 124 performs correlated double sampling (CDS) processing and AD conversion processing on the pixel signals output from each pixel 121 of a row of the pixel array unit 122 to remove pixel-specific fixed pattern noise.

[0085] The horizontal drive unit 125 includes, for example, a shift register and sequentially outputs horizontal scan pulses so that the signal processing circuit 131 sequentially outputs (digital) pixel signals, which are obtained by performing AD conversion on each pixel in a predetermined row and are held in the AD conversion unit 124.

[0086] The timing control unit 126 receives input clock and indication of operating mode, and outputs data such as internal information of the polarization sensor 100. Based on the vertical synchronization signal, horizontal synchronization signal, and master clock, the timing control unit 126 generates clock signals or control signals that serve as references for the operation of the vertical drive unit 123, AD conversion unit 124, horizontal drive unit 125, etc. Then, the timing control unit 126 outputs the generated clock signals and control signals to the vertical drive unit 123, AD conversion unit 124, horizontal drive unit 125, etc.

[0087] The signal processing circuit 131 performs various digital signal processing operations on the pixel signals provided from the AD conversion unit 124, such as black level adjustment and column change correction. In addition, the signal processing circuit 131 also performs de-mosaic processing on the original RGB image in which the polarization components at 0 degrees, 45 degrees, 90 degrees and 135 degrees or the R, G and B color components are mosaic-patterned.

[0088] Figure 5 An example of demosaicing is shown for an original RGB image in which the R, G, and B color components form a mosaic pattern.

[0089] A 3ch RGB image of R, G, and B is generated by performing demosaic processing on the 1ch RGB raw image generated by imaging unit 111, in which color filters are arranged in a Bayer array or a tetrahedral Bayer array. A Bayer array is an array in which color filters of G, B, R, and G are arranged in 2×2 units of four pixels forming repeating units in the matrix direction. A tetrahedral Bayer array is an array in which color filters of G, B, R, and G are arranged in a Bayer array unit of four pixels in 2×2 units, wherein the arrangement unit for one color is four pixels in 2×2 units.

[0090] Figure 6 An example of demosaicing is shown for a polarized original image in which the polarization components at 0 degrees, 45 degrees, 90 degrees, and 135 degrees are mosaic-patterned.

[0091] By performing demosaicing on the 1ch polarized raw image generated by imaging unit 111, 4ch polarized images of 0°, 45°, 90°, and 135° are generated, in which the polarization components are arranged in a Bayer array or a tetrahedral Bayer array. A Bayer array is an array in which the polarization components of 0°, 45°, 90°, and 135° are arranged in 2×2 units of four pixels forming repeating units in the matrix direction. A tetrahedral Bayer array is an array in which the polarization components of 0°, 45°, 90°, and 135° are arranged in a 2×2 unit of four pixels using a Bayer array, where one polarization component is arranged in a 2×2 unit of four pixels.

[0092] Details of the demosaic process are disclosed, for example, in Japanese Patent Nos. 6750633 and 7070423. The signal processing circuit 131 can perform the demosaic process by employing the techniques disclosed above or other known techniques.

[0093] The memory 132 stores the parameters and signals required for processing by the signal processing circuit 131 as needed.

[0094] The output circuit 113 buffers the signals sequentially output from the signal processing circuit 131 and outputs the signals to external circuits such as a subsequent stage image signal processor (ISP).

[0095] In the polarization sensor 100 configured as described above, the imaging unit 111 may have a multi-pixel configuration capable of suppressing image quality degradation of RGB images, acquiring high-quality RGB information, and simultaneously acquiring polarization information. The multi-pixel structures that can be acquired by the imaging unit 111 will be described sequentially below.

[0096] <3. Example of the first configuration of the imaging unit>

[0097] Figure 7 This is a diagram illustrating a first configuration example of the imaging unit 111.

[0098] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the first configuration example are configured by vertically stacking a color filter 151, an organic photoelectric conversion film 152, a polarization filter (polarizer) 153, and a semiconductor substrate 154 in sequence from the light incident surface side. The semiconductor substrate 154 includes a silicon substrate using, for example, silicon (Si) as the semiconductor. A photodiode (PD) 161 is formed on the semiconductor substrate 154 as a photoelectric conversion unit.

[0099] The color filter 151 is arranged in a square Bayer array, in which the color filters for red (R), green (G), blue (B) are arranged in a 2×2 four-pixel unit, wherein the arrangement unit for one color is four pixels in a 2×2 array. The color filter array can be a Bayer array instead of a square Bayer array.

[0100] The organic photoelectric conversion film 152 is a beam-splitting and photoelectric conversion unit that absorbs and photoelectrically converts light in the visible light region to generate signal charge (charge). Transparent electrodes for reading out the signal charge generated in the organic photoelectric conversion film 152 are formed on the upper and lower surfaces of the organic photoelectric conversion film 152, but these transparent electrodes are not shown. Since the color filter 151 is disposed on the light-incident surface side of the organic photoelectric conversion film 152, the organic photoelectric conversion film 152 photoelectrically converts light corresponding to the color (R, G, or B) of the color filter 151 to generate charge.

[0101] The polarization filter 153 is configured by arranging polarizers with polarization directions (polarization angles) of 0 degrees, 45 degrees, 90 degrees and 135 degrees in four pixels of 2×2, and by repeatedly arranging these polarizers as repeating units in the row and column directions of the pixel array unit.

[0102] The photodiode 161 is a photoelectric conversion unit that performs photoelectric conversion on light in the invisible light region, i.e. infrared light (IR), that passes through the polarization filter 153 to generate signal charge (charge).

[0103] In the imaging unit 111 configured as described above according to the first configuration example, for example, Figure 8 As shown on the left, in pixel 121 where an R color filter 151 is disposed, an organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the R color component as first-layer imaging information. Light with the G and B color components is absorbed by the R color filter 151, and only infrared light in the invisible light region is incident on the polarization filter 153. A photodiode 161 generates a pixel signal obtained by photoelectric conversion of infrared light passing through the polarization filter 153 as second-layer imaging information. This polarization filter has a polarization direction in any one of the deflection directions (hereinafter referred to as a specific polarization direction) of 0 degrees, 45 degrees, 90 degrees, or 135 degrees.

[0104] In addition, for example, such as Figure 8As shown on the right, in pixel 121 where a color filter 151 for color G is disposed, an organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer of imaging information. Light with the R and B color components is absorbed by the color filter 151 for color G, and only infrared light in the invisible light region is incident on the polarization filter 153. A photodiode 161 generates a pixel signal obtained by photoelectric conversion of infrared light passing through the polarization filter 153 as the second layer of imaging information, which has a polarization direction in a specific deflection direction.

[0105] Figure 9 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the first configuration example.

[0106] The signal processing unit 112 includes: an RGB generation unit 171 configured to generate RGB information; and a polarization information generation unit 172 configured to generate polarization information.

[0107] Imaging unit 111 generates an RGB original image as the first layer of imaging information, which is a mosaic pattern of a square Bayer array according to color filter 151, and outputs the RGB original image to RGB generation unit 171. RGB generation unit 171 performs demosaic processing on the RGB original image of the first layer of imaging information to generate a 3ch RGB image of R, G, and B and outputs it as RGB information.

[0108] Furthermore, the imaging unit 111 generates an IR polarization original image with a mosaic pattern in which the polarization components are 0 degrees, 45 degrees, 90 degrees, or 135 degrees as the second layer of imaging information, and the imaging unit 111 outputs the IR polarization original image to the polarization information generation unit 172. The polarization information generation unit 172 performs demosaic processing on the polarization original image of the second layer of imaging information to generate a 4ch monochromatic polarization image with polarization components of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, and the polarization information generation unit 172 outputs this monochromatic polarization image as polarization information.

[0109] Figure 10 This is a table comparing sensitivity and resolution between a first configuration example and a comparative example of imaging unit 111.

[0110] In the comparison results between the first to seventh configuration examples and the comparative examples described below, the estimate of the light utilization efficiency of the organic photoelectric conversion film 152 with respect to sensitivity is shown.

[0111] In the first configuration example, light is received before passing through polarization filter 153, while in the comparative example, light is received after passing through polarization filter 32. Therefore, regarding RGB information, the first configuration example can receive two or more times the amount of light, and the sensitivity is two or more times greater. The RGB resolution in the first configuration example is approximately four times greater because the smallest pixel unit in the first configuration example is one pixel, while the smallest pixel unit in the comparative example is four pixels. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0112] Regarding polarization information, similar to the comparative example, light passing through polarization filter 153 is received in the first configuration example, and therefore the sensitivity is equal. The resolution in the comparative example is in units of four pixels, while the resolution in the first configuration example is in units of one pixel, and therefore the resolution in the first configuration example is approximately four times greater. The light component (wavelength) used to generate the polarization information is infrared (IR) light in both examples.

[0113] Therefore, the polarization sensor 100, which includes the imaging unit 111 according to the first configuration example, can improve resolution and sensitivity compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained.

[0114] <4. Example of a second configuration for the imaging unit>

[0115] Figure 11 This is a diagram illustrating a second configuration example of the imaging unit 111.

[0116] In the descriptions of the second to seventh configuration examples below, the same reference numerals will be used for the parts that are the same as those described above, and redundant descriptions will be omitted as appropriate.

[0117] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the second configuration example are configured by vertically stacking an IR cutoff filter 181, an organic photoelectric conversion film 152, a polarization filter 153, and a semiconductor substrate 154 in sequence from the light incident surface side. The second configuration example differs from the first configuration example in that an IR cutoff filter 181 is provided instead of a color filter 151, but they are otherwise similar.

[0118] The IR cutoff filter (IRCF) 181 absorbs light in the invisible region, i.e., infrared light (IR), and transmits light in the visible region so that the light is incident on the organic photoelectric conversion film 152.

[0119] The organic photoelectric conversion film 152 is a beam splitting unit and photoelectric conversion unit that absorbs and photoelectrically converts any color component of light (R, G, or B) in the visible light region to generate signal charge. As shown in the plan view, the array (color array) of color components absorbed by the organic photoelectric conversion film 152 is arranged, for example, in a tetragonal Bayer array. The color array of the organic photoelectric conversion film 152 can be a Bayer array. As an organic photoelectric conversion film for photoelectric conversion of G light, for example, organic photoelectric conversion materials containing rhodamine dyes, merocyanine dyes, quinacridones, etc. can be used. As an organic photoelectric conversion film for photoelectric conversion of R light, organic photoelectric conversion materials containing phthalocyanine dyes can be used. As an organic photoelectric conversion film for photoelectric conversion of B light, organic photoelectric conversion materials containing coumarin dyes, tri-8-hydroxyquinoline Al(Alq3), merocyanine dyes, etc. can be used. The organic photoelectric conversion film 152 transmits light that is not absorbed by the organic photoelectric conversion film 152 and directs this light onto the polarization filter 153.

[0120] The polarization filter 153 and photodiode 161 are configured the same as in the first configuration example.

[0121] In the imaging unit 111 configured as described above according to the second configuration example, for example, Figure 12 As shown on the left, in pixel 121, an organic photoelectric conversion film 152 is disposed therein to photoelectricly convert light of the R color component. The organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light of the R color component as the first layer of imaging information. Light of the G and B color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light of the Cy (cyan) color component (which is the complementary color of the R color component) with a specific deflection direction as the second layer of imaging information. Complementary colors are colors at opposite positions on the color wheel, and are also called opposite colors. When two colors with opposite color relationships are mixed, the color becomes achromatic.

[0122] In addition, for example, such as Figure 12As shown on the right, in pixel 121, an organic photoelectric conversion film 152 is disposed therein for photoelectric conversion of light with the G color component. The organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer of imaging information. Light with the R and B color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the Mg (magenta) color component (which is the complementary color of the G color component) having a polarization direction with a specific deflection direction as the second layer of imaging information.

[0123] Although not shown, in pixel 121, where an organic photoelectric conversion film 152 for photoelectric conversion of light with the B color component is disposed, the organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the B color component as first-layer imaging information. Light with the R and G color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the Ye (yellow) color component (which is the complementary color of the B color component) having a polarization direction with a specific deflection direction as second-layer imaging information.

[0124] Figure 13 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the second configuration example.

[0125] The signal processing unit 112 includes: an RGB generation unit 201 configured to generate RGB information; a polarization information generation unit 202 configured to generate polarization information; and a WB gain calculation unit 203.

[0126] Imaging unit 111 generates an RGB original image with a mosaic pattern corresponding to the organic photoelectric conversion film 152 of the tetrahedral Bayer array as the first layer of imaging information, and outputs the RGB original image to RGB generation unit 201. RGB generation unit 201 performs demosaic processing on the RGB original image of the first layer of imaging information to generate a 3ch RGB image of R, G, and B and outputs it as RGB information.

[0127] Furthermore, the imaging unit 111 generates an RGB complementary color polarization original image as the second layer of imaging information, consisting of a mosaic pattern of a square Bayer array with color components of cyan, magenta, and yellow, and a Bayer array with polarization components of 0 degrees, 45 degrees, 90 degrees, or 135 degrees. The imaging unit 111 then outputs this RGB complementary color polarization original image to the polarization information generation unit 202. The polarization information generation unit 202 performs demosaic processing on the RGB complementary color polarization original image of the second layer of imaging information to generate a total of 12 channels of color polarization image (including 3 channels of RGB complementary color images for Cy, Mg, and Ye for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees), and outputs this color polarization image as polarization information.

[0128] The WB gain calculation unit 203 calculates the white balance gain, which is a pixel value adjustment parameter used to correct the pixel values ​​of the captured image to the original colors of the subject. The WB gain calculation unit 203 calculates the white balance gain using an RGB image and a color polarized image. The RGB image is RGB information provided by the RGB generation unit 201, and the color polarized image is polarization information provided by the polarization information generation unit 202. For example, the WB gain calculation unit 203 uses the G pixel value of the captured image as a reference and, without changing the G pixel value, calculates a white balance gain kR multiplied by the R pixel value of the captured image and a white balance gain kB multiplied by the B pixel value of the captured image. Each element of the white balance gain (kR, kB) corresponds to the multiplication parameter of the pixel value (R, G, B) for each color in the captured image.

[0129] WB gain calculation unit 203 assumes that the "color components of the complementary colors of R, G, and B" after white balance adjustment are equal to the "color components of the polarization degree of the complementary colors of R, G, and B", and calculates the white balance gain k. R The complementary color component of R is cyan (Cy), the complementary color component of G is magenta (Mg), and the complementary color component of B is yellow (Ye).

[0130] The color components of the complementary colors of R, G and B after white balance adjustment are represented by the following formula (1).

[0131] [Mathematical Expression 1]

[0132] The color components of R's complementary color:

[0133] The color components of G's complementary color:

[0134] The color components of B's ​​complementary color:

[0135] in

[0136] I inv，R The luminance of the complementary color of R

[0137] I inv，G The luminance of the complementary color of G

[0138] I inv，B The luminance of the complementary color of B

[0139] The average value of the brightness of complementary colors.

[0140] I MAX Maximum brightness

[0141] Furthermore, the color component of the polarization degree d of the complementary colors of R, G and B is represented by the following equation (2).

[0142] [Mathematical Expression 2]

[0143] The color components of the polarization degree d of the complementary color of R:

[0144]

[0145] The color components of the polarization degree d of the complementary color of G:

[0146]

[0147] The color components of the polarization degree d of the complementary color of B:

[0148]

[0149] in

[0150] d inv，R =d Cy The color component (cy) of the polarization degree d of the complementary color of R.

[0151] d inv，G =d Mg The color component (Mg) of the polarization degree d of the complementary color of G.

[0152] d inv，B =d Ye The color component (Ye) of the polarization degree d of the complementary color of B.

[0153] The average value of the color components of the polarization degree d of complementary colors

[0154]

[0155] When R, G and B are expressed together as a single equation (where the color components of the complementary colors of R, G and B after white balance adjustment in equation (1) are taken as the left part, and the color components of the polarization degree of the complementary colors of R, G and B in equation (2) are taken as the right part), the following equation (3) is obtained.

[0156] [Mathematical Expression 3]

[0157]

[0158] I inv，R I inv，G and I inv，B It has the following relationship:

[0159] [Mathematical Expression 4]

[0160]

[0161] I inv，B =I MAX -k B I B

[0162] k R White balance gain of R

[0163] k B White balance gain of B

[0164] Therefore, equation (4) is obtained by rearranging equation (3), and equation (5) is obtained by transforming equation (4).

[0165] [Mathematical Expression 5]

[0166]

[0167] Since the unknowns in equation (5) are the two white balance gains kR and kB of R and B, and there are three equations about R, G and B, the simultaneous equations can be solved and the white balance gains kR and kB can be calculated.

[0168] "Through white balance gain (k R I R I G k B I B The white balance adjustment image (R, G, B) of the opposite colors (Cy, Mg, Ye) = the color components of the polarization degree d of the complementary colors of R, G, and B (d inv，R =d Cy d inv，G =d Mg d inv，B =d Ye )”

[0169] The reason for establishing this relationship is that the phases of diffusely polarized light and specularly polarized light generated by reflection from the subject are offset from each other, and furthermore, when comparing the intensity of specularly polarized light with the intensity of diffusely polarized light, the intensity of specularly polarized light is greater, that is, the specular polarization degree > diffuse polarization degree. According to the second configuration example of imaging unit 111, since the polarization information obtained as the second layer of imaging information is an RGB complementary color image, the complementary colors (opposite colors) of RGB can be directly optically acquired, and the step of calculating the complementary colors (opposite colors) of RGB through arithmetic operations can be omitted.

[0170] Figure 14 This is a table comparing sensitivity and resolution between a second configuration example and a comparative example of imaging unit 111.

[0171] In the second configuration example, light passing through polarization filter 153 is received, while in the comparative example, light passing through polarization filter 32 is received. Therefore, regarding RGB information, the second configuration example can receive two or more times the amount of light, and the sensitivity is two or more times greater. The RGB resolution in the second configuration example is approximately four times greater because the smallest pixel unit in the second configuration example is one pixel, while the smallest pixel unit in the comparative example is four pixels. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0172] In the second configuration example, light of two colors passing through polarization filter 153 is received. Therefore, regarding polarization information, the amount of light received in the second configuration example is two or more times greater, and the sensitivity is two or more times greater. The resolution is the same. The light components (wavelengths) used to generate polarization information are RGB in the comparative example, but are the complementary colors of RGB in the second configuration example.

[0173] Therefore, the polarization sensor 100, which includes the imaging unit 111 according to the second configuration example, can improve resolution and sensitivity compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained.

[0174] <5. Example of a third configuration for the imaging unit>

[0175] Figure 15 This is a diagram illustrating a third configuration example of the imaging unit 111.

[0176] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the third configuration example are configured by vertically stacking an organic photoelectric conversion film 152, a polarization filter 153 and a semiconductor substrate 154 in sequence from the light incident surface side.

[0177] The configuration of the organic photoelectric conversion film 152, polarization filter 153, and photodiode 161 is similar to that in the second configuration example. The organic photoelectric conversion film 152 is a beam splitter and photoelectric conversion unit that absorbs and photoelectrically converts only any color component of light (R, G, or B) in the visible light region to generate a signal charge. The color components of light not absorbed by the organic photoelectric conversion film 152 are incident on the polarization filter 153. The color array of the organic photoelectric conversion film 152 is arranged, for example, in a tetrahedral Bayer array as shown in the plan view.

[0178] The third configuration example differs from the second configuration example in that the IR cutoff filter 181 is not set.

[0179] In the imaging unit 111 configured as described above according to the third configuration example, for example, Figure 16 As shown on the left, in pixel 121, an organic photoelectric conversion film 152 is disposed therein for photoelectric conversion of light with the R color component. The organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the R color component as the first layer of imaging information. Light with the G, B, and IR color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the complementary color Cy of the R color component and the color component of infrared light (Cy+IR) having a polarization direction with a specific deflection direction as the second layer of imaging information.

[0180] Additionally, for example, such as Figure 16 As shown on the right, in pixel 121, an organic photoelectric conversion film 152 is disposed therein for photoelectric conversion of light with the G color component. The organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer of imaging information. Light with the R, B, and IR color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the complementary color Mg of the G color component having a specific polarization direction and the infrared color component (Mg+IR) as the second layer of imaging information.

[0181] Although not shown, in pixel 121, where an organic photoelectric conversion film 152 for photoelectric conversion of light with the B color component is disposed, the organic photoelectric conversion film 152 outputs a pixel signal obtained by photoelectric conversion of light with the B color component as first-layer imaging information. Light with the R, G, and IR color components passes through the organic photoelectric conversion film 152, passes through the polarization filter 153, and is incident on the semiconductor substrate 154. Photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the complementary color Ye of the B color component and the color component of infrared light (Ye+IR) having a polarization direction with a specific deflection direction as second-layer imaging information.

[0182] Figure 17 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the third configuration example.

[0183] The signal processing unit 112 includes: an RGB generation unit 211 configured to generate RGB information; and a polarization information generation unit 212 configured to generate polarization information.

[0184] Imaging unit 111 generates an RGB original image with a mosaic pattern corresponding to the organic photoelectric conversion film 152 of the tetrahedral Bayer array as the first layer of imaging information, and outputs the RGB original image to RGB generation unit 211. RGB generation unit 211 performs demosaic processing on the RGB original image of the first layer of imaging information to generate a 3ch RGB image of R, G, and B and outputs it as RGB information.

[0185] Furthermore, the imaging unit 111 generates an RGB complementary color IR polarization original image as the second layer of imaging information, which has a mosaic pattern of a square Bayer array with cyan, magenta, and yellow color components and Bayer arrays with polarization components of 0°, 45°, 90°, or 135°. This RGB complementary color IR polarization original image is then output to the polarization information generation unit 212. The polarization information generation unit 212 performs demosaicing processing on the RGB complementary color IR polarization original image of the second layer of imaging information to generate a total of 12ch color polarization images (including 3ch RGB complementary color IR images of Cy+IR, Mg+IR, and Ye+IR for each of the 0°, 45°, 90°, and 135° polarization components), and outputs this color polarization image as polarization information. Figure 17 In this context, Cy', Mg', and Ye' represent infrared light and satisfy Cy' = Cy + IR, Mg' = Mg + IR, and Ye' = Ye + IR.

[0186] Figure 18This is a table comparing sensitivity and resolution between a third configuration example and a comparative example of imaging unit 111.

[0187] In the third configuration example, light is received before passing through polarization filter 153, while in the comparative example, light is received after passing through polarization filter 32. Therefore, regarding RGB information, the third configuration example can receive two or more times the amount of light, and the sensitivity is two or more times greater. The RGB resolution in the third configuration example is approximately four times greater because the smallest pixel unit in the third configuration example is one pixel, while the smallest pixel unit in the comparative example is four pixels. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0188] In the third configuration example, light of two colors passing through polarization filter 153 is received. Therefore, regarding polarization information, in the third configuration example, the amount of light received can be two or more times greater, and the sensitivity is two or more times greater. The resolution is equal. The light component (wavelength) used to generate polarization information is RGB in the comparative example, but in the third configuration example, it is a mixture of complementary colors of RGB and infrared light.

[0189] Therefore, the polarization sensor 100, including the imaging unit 111 according to the third configuration example, can improve resolution and sensitivity compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained. In the third configuration example, light with arbitrary color components of R, G, B, and IR is used as a signal, and light can be used as a signal without waste.

[0190] <6. Example of the fourth configuration of the imaging unit>

[0191] Figure 19 This is a diagram illustrating a fourth configuration example of the imaging unit 111.

[0192] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the fourth configuration example are configured by sequentially and vertically stacking an IR cutoff filter 181, a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, a polarization filter 153, and a semiconductor substrate 154 from the light incident surface side. The fourth configuration is... Figure 11 The difference in the second configuration example shown is that a second organic photoelectric conversion film 152B is also disposed between the first organic photoelectric conversion film 152A and the polarization filter 153.

[0193] The first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B are organic photoelectric conversion films 152 having different color components (wavelengths) to be photoelectric converted. In this embodiment, the first organic photoelectric conversion film 152A absorbs and photoelectrically converts only the G color component light in the visible light region to generate signal charge, while the second organic photoelectric conversion film 152B absorbs and photoelectrically converts only the R color component light in the visible light region to generate signal charge. The B color component light that is not absorbed by the first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B is incident on the polarization filter 153. The first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B are formed on the entire surface (all pixels) of the pixel array unit 122.

[0194] In the imaging unit 111 configured as described above according to the fourth configuration example, such as Figure 20 As shown, the first organic photoelectric conversion film 152A outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer imaging information. The second organic photoelectric conversion film 152B outputs a pixel signal obtained by photoelectric conversion of light with the R color component as the second layer imaging information. Light with the B color component is transmitted through the first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B, and also through the polarization filter 153, and is incident on the semiconductor substrate 154. The photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the B color component having a polarization direction with a specific deflection direction as the third layer imaging information. Infrared light (IR) in the invisible light region is absorbed by the IR cutoff filter 181.

[0195] Figure 21 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the fourth configuration example.

[0196] The signal processing unit 112 includes: an RGB / polarization information generation unit 261 configured to generate polarization information; and an RGB synthesis unit 262 configured to generate RGB information.

[0197] Imaging unit 111 generates a G image, including the G pixel signal generated by the first organic photoelectric conversion film 152A, as the first layer of imaging information, and outputs the G image to RGB synthesis unit 262. Imaging unit 111 generates an R image, including the R pixel signal generated by the second organic photoelectric conversion film 152B, as the second layer of imaging information, and outputs the R image to RGB synthesis unit 262. Imaging unit 111 generates a B-polarized raw image, including the B pixel signal generated by photodiode 161 and having a polarization direction in a specific deflection direction, as the third layer of imaging information, and outputs the B-polarized raw image to RGB / polarization information generation unit 261.

[0198] The RGB / polarization information generation unit 261 performs demosaic processing on the original B-polarized image of the third-layer imaging information to generate a B-image excluding polarization components and 4-ch B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 261 outputs the B-image excluding polarization components to the RGB synthesis unit 262, and outputs the 4-ch B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0199] The RGB synthesis unit 262 combines the G and R images from the imaging unit 111 with the B image from the RGB / polarization information generation unit 261 to generate a 3ch RGB image of R, G and B and outputs it as RGB information.

[0200] Figure 22 This is a table comparing the sensitivity and resolution between the fourth configuration example and the comparative example of the imaging unit 111.

[0201] In the fourth configuration example, light passing through polarization filter 153 (G and R) is received, while in the comparative example, light passing through polarization filter 32 is received. Therefore, regarding RGB information, the fourth configuration example can receive two or more times the amount of light, and the sensitivity is two or more times greater. In the fourth configuration example, pixel signals of two colors (R, G, and B) are obtained from all pixels instead of a mosaic array, and thus the RGB resolution is approximately 16 times greater. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0202] Regarding polarization information, the sensitivity is equal, and the resolution is approximately four times greater because polarization information of the same color (R) can be obtained in all pixels. The light component (wavelength) used to generate the polarization information is RGB in the comparative example, but in the fourth configuration example, it is the color component of one of R, G, and B, namely the B color component that is not absorbed by the first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B.

[0203] Therefore, the polarization sensor 100, which includes the imaging unit 111 according to the fourth configuration example, can improve resolution and sensitivity compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained.

[0204] <7. Fifth Configuration Example of Imaging Unit>

[0205] Figure 23 This is a diagram illustrating a fifth configuration example of the imaging unit 111.

[0206] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the fifth configuration example are configured by vertically stacking an IR cutoff filter 181, a first organic photoelectric conversion film 152A, a polarization filter 153, a second organic photoelectric conversion film 152B, and a semiconductor substrate 154 in sequence from the light incident surface side. This is consistent with the configuration described above. Figure 19 The fourth configuration example differs in the placement of the second organic photoelectric conversion film 152B. Figure 19 In the fourth configuration example, the second organic photoelectric conversion film 152B is disposed between the first organic photoelectric conversion film 152A and the polarization filter 153. However, in the fifth configuration example, the second organic photoelectric conversion film 152B is disposed between the polarization filter 153 and the semiconductor substrate 154. Other configurations are the same as those in the fourth configuration example.

[0207] In the imaging unit 111 configured as described above according to the fifth configuration example, such as Figure 24 As shown, the first organic photoelectric conversion film 152A outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer of imaging information. Light with the R and B color components passes through the first organic photoelectric conversion film 152A, passes through the polarization filter 153, and is incident on the second organic photoelectric conversion film 152B. The second organic photoelectric conversion film 152B outputs a pixel signal obtained by photoelectric conversion of light with the R color component having a polarization direction with a specific deflection direction as the second layer of imaging information. Light with the B color component is transmitted through the second organic photoelectric conversion film 152B and is incident on the photodiode 161. The photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the B color component having a polarization direction with a specific deflection direction as the third layer of imaging information. Infrared light (IR) in the invisible light region is absorbed by the IR cutoff filter 181.

[0208] Figure 25 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the fifth configuration example.

[0209] The signal processing unit 112 includes: RGB / polarization information generation units 281 and 282, which are configured to generate polarization information; and RGB synthesis unit 283, which is configured to generate RGB information.

[0210] Imaging unit 111 outputs a G image, including the G pixel signal generated by the first organic photoelectric conversion film 152A, as the first layer of imaging information to RGB synthesis unit 283. Imaging unit 111 outputs an R-polarized raw image, including the R pixel signal, as the second layer of imaging information to RGB / polarization information generation unit 281. This R pixel signal is generated by the second organic photoelectric conversion film 152B and has a polarization direction in a specific deflection direction. Imaging unit 111 outputs a B-polarized raw image, including the B pixel signal, as the third layer of imaging information to RGB / polarization information generation unit 282. This B pixel signal is generated by the photodiode 161 and has a polarization direction in a specific deflection direction.

[0211] The RGB / polarization information generation unit 281 performs demosaic processing on the R-polarized raw image of the second layer imaging information to generate an R-image excluding polarization components and 4ch R-polarization images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 281 outputs the R-image excluding polarization components to the RGB synthesis unit 283, and outputs the 4ch R-polarization images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0212] The RGB / polarization information generation unit 282 performs demosaic processing on the original B-polarized image of the third-layer imaging information to generate a B-image excluding polarization components and 4-ch B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 282 outputs the B-image excluding polarization components to the RGB synthesis unit 283, and outputs the B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0213] The RGB synthesis unit 283 combines the G image from the imaging unit 111, the R image from the RGB / polarization information generation unit 281, and the B image from the RGB / polarization information generation unit 282 to generate a 3ch RGB image of R, G, and B and outputs it as RGB information.

[0214] Figure 26 This is a table comparing the sensitivity and resolution between the fifth configuration example and the comparative example of the imaging unit 111.

[0215] In the fifth configuration example, light G is received before polarization filter 153, while in the comparative example, light is received after polarization filter 32. Therefore, regarding RGB information, the fifth configuration example can receive two or more times the amount of light, and the sensitivity is two or more times greater. In the fifth configuration example, pixel signals of two colors among R, G, and B are obtained from all pixels instead of a mosaic array, and thus the RGB resolution is approximately 16 times greater. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0216] Regarding polarization information, the sensitivity is equal, but it can be improved by combining the polarization information of R and B. Since the same color (R, B) polarization information can be obtained in all pixels, the resolution is approximately four times greater. The light components (wavelengths) used to generate the polarization information are RGB in the comparative example, but in the fourth configuration example, they are two colors among R, G, and B, namely the G and B color components that are not absorbed by the first organic photoelectric conversion film 152A, and the number of light components that enable the acquisition of polarization information can be increased.

[0217] Therefore, the polarization sensor 100, which includes the imaging unit 111 according to the fifth configuration example, can improve resolution and sensitivity compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained.

[0218] <8. Sixth Configuration Example of Imaging Unit>

[0219] Figure 27 This is a diagram illustrating a sixth configuration example of the imaging unit 111.

[0220] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the sixth configuration example are configured by sequentially and vertically stacking an IR cutoff filter 181, a polarization filter 153, a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, and a semiconductor substrate 154 from the light incident surface side. Figure 23 The fifth configuration example differs in the placement of polarization filter 153. Figure 23 In the fifth configuration example, the polarization filter 153 is disposed between the first organic photoelectric conversion film 152A and the second organic photoelectric conversion film 152B. However, in the sixth configuration example, the polarization filter 153 is disposed between the IR cutoff filter 181 and the first organic photoelectric conversion film 152A. Other configurations are similar to those in the fifth configuration example.

[0221] In the imaging unit 111 configured as described above according to the sixth configuration example, such as Figure 28As shown, the first organic photoelectric conversion film 152A outputs a pixel signal obtained by photoelectric conversion of light with the G color component having a polarization direction in a specific deflection direction, which has passed through the polarization filter 153, as the first layer of imaging information. Light with R and B color components passes through the first organic photoelectric conversion film 152A and is incident on the second organic photoelectric conversion film 152B. The second organic photoelectric conversion film 152B outputs a pixel signal obtained by photoelectric conversion of light with the R color component having a polarization direction in a specific deflection direction, which has passed through the polarization filter 153, as the second layer of imaging information. Light with the B color component is transmitted through the second organic photoelectric conversion film 152B and is incident on the photodiode 161. The photodiode 161 generates a pixel signal obtained by photoelectric conversion of light with the B color component having a polarization direction in a specific deflection direction as the third layer of imaging information. Infrared light (IR) in the invisible light region is absorbed by the IR cutoff filter 181.

[0222] Figure 29 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the sixth configuration example.

[0223] The signal processing unit 112 includes: RGB / polarization information generation units 301, 302 and 303, which are configured to generate polarization information; and RGB synthesis unit 304, which is configured to generate RGB information.

[0224] Imaging unit 111 outputs a G-polarized raw image as the first layer of imaging information to RGB / polarization information generation unit 301. This G-polarized raw image includes G-pixel signals generated by the first organic photoelectric conversion film 152A and having a polarization direction with a specific deflection direction. Imaging unit 111 outputs an R-polarized raw image as the second layer of imaging information to RGB / polarization information generation unit 302. This R-polarized raw image includes R-pixel signals generated by the second organic photoelectric conversion film 152B and having a polarization direction with a specific deflection direction. Imaging unit 111 outputs a B-polarized raw image as the third layer of imaging information to RGB / polarization information generation unit 303. This B-polarized raw image includes B-pixel signals generated by the photodiode 161 and having a polarization direction with a specific deflection direction.

[0225] The RGB / polarization information generation unit 301 performs demosaic processing on the original G-polarized image of the first layer of imaging information to generate a G image excluding polarization components and 4-ch G-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 301 outputs the G image excluding polarization components to the RGB synthesis unit 304, and outputs the 4-ch G-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0226] The RGB / polarization information generation unit 302 performs demosaic processing on the R-polarized raw image of the second layer imaging information to generate an R-image excluding polarization components and 4ch R-polarization images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 302 outputs the R-image excluding polarization components to the RGB synthesis unit 304, and outputs the R-polarization images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0227] The RGB / polarization information generation unit 303 performs demosaic processing on the original B-polarized image of the third-layer imaging information to generate a B-image excluding polarization components and 4-ch B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The RGB / polarization information generation unit 303 outputs the B-image excluding polarization components to the RGB synthesis unit 304, and outputs the 4-ch B-polarized images for each polarization component at 0 degrees, 45 degrees, 90 degrees, and 135 degrees as polarization information.

[0228] The RGB synthesis unit 304 combines the G image from the RGB / polarization information generation unit 301, the R image from the RGB / polarization information generation unit 302, and the B image from the RGB / polarization information generation unit 303 to generate a 3chRGB image of R, G, and B and outputs it as RGB information.

[0229] Figure 30 This is a table comparing the sensitivity and resolution between the sixth configuration example and the comparative example of the imaging unit 111.

[0230] Regarding RGB information, similar to the comparative example, light passing through polarization filter 153 is received in the sixth configuration example, and therefore the sensitivity is equal. In the sixth configuration example, pixel signals for specific polarization components of the three colors R, G, and B are obtained in all pixels, and therefore the RGB resolution is approximately 4 times greater. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0231] Regarding polarization information, the sensitivity is equal, but it can be improved by combining the polarization information of R and B. Since the same color (G, R, B) polarization information can be obtained in all pixels, the resolution is approximately 4 times greater. Similar to the comparative example, the light components (wavelengths) used to generate the polarization information are RGB.

[0232] Therefore, according to the polarization sensor 100 including the imaging unit 111 according to the fifth configuration example, the resolution of both RGB information and polarization information can be improved compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be obtained.

[0233] <9. Seventh Configuration Example of Imaging Unit>

[0234] Figure 31 This is a diagram illustrating a seventh configuration example of the imaging unit 111.

[0235] In the cross-sectional view, the pixels 121 of the imaging unit 111 according to the seventh configuration example are configured by sequentially and vertically stacking a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, a third organic photoelectric conversion film 152C, a polarization filter 153, and a semiconductor substrate 154 from the light incident surface side. (This is consistent with the description above.) Figure 27 The sixth configuration example differs in that the IR cutoff filter 181 is omitted, a third organic photoelectric conversion film 152C is added below the second organic photoelectric conversion film 152B, and a polarization filter 153 is placed between the third organic photoelectric conversion film 152C and the semiconductor substrate 154. The third organic photoelectric conversion film 152C outputs a pixel signal obtained by photoelectric conversion of light with the B color component as the third layer imaging information. The photodiode 161 generates a pixel signal obtained by photoelectric conversion of infrared light (IR) that passes through the polarization filter 153, is light in the invisible light region, and has a polarization direction in a specific polarization direction as the fourth layer imaging information. Other configurations are similar to those in the fifth configuration example.

[0236] In the imaging unit 111 configured as described above according to the seventh configuration example, such as Figure 32As shown, the first organic photoelectric conversion film 152A outputs a pixel signal obtained by photoelectric conversion of light with the G color component as the first layer imaging information. Light with IR, R, and B color components passes through the first organic photoelectric conversion film 152A and is incident on the second organic photoelectric conversion film 152B. The second organic photoelectric conversion film 152B outputs a pixel signal obtained by photoelectric conversion of light with the R color component as the second layer imaging information. Light with IR and B color components passes through the second organic photoelectric conversion film 152B and is incident on the third organic photoelectric conversion film 152C. The third organic photoelectric conversion film 152C outputs a pixel signal obtained by photoelectric conversion of light with the B color component as the third layer imaging information. Light with the IR color component is transmitted through the third organic photoelectric conversion film 152C and is incident on the photodiode 161. The photodiode 161 generates a pixel signal obtained by photoelectric conversion of infrared light (IR) with a polarization direction that has passed through the polarization filter 153 and has a specific deflection direction as the fourth layer imaging information.

[0237] Figure 33 This is a block diagram illustrating a configuration example of the signal processing unit 112 when the imaging unit 111 is configured using the seventh configuration example.

[0238] The signal processing unit 112 includes: an RGB synthesis unit 331 configured to generate RGB information; and a polarization information generation unit 332 configured to generate polarization information.

[0239] Imaging unit 111 outputs a G image, including the G pixel signal generated by the first organic photoelectric conversion film 152A, as the first layer of imaging information to RGB synthesis unit 331. Imaging unit 111 outputs an R image, including the R pixel signal generated by the second organic photoelectric conversion film 152B, as the second layer of imaging information to RGB synthesis unit 331. Imaging unit 111 outputs a B image, including the B pixel signal generated by the third organic photoelectric conversion film 152C, as the third layer of imaging information to RGB synthesis unit 331. Imaging unit 111 outputs an IR polarization raw image, including the IR pixel signal generated by photodiode 161 and having a polarization direction with a specific deflection direction, as the fourth layer of imaging information to polarization information generation unit 332.

[0240] The RGB synthesis unit 331 combines the G image, R image and B image from the imaging unit 111 to generate a 3ch RGB image of R, G and B and outputs it as RGB information.

[0241] The polarization information generation unit 332 performs demosaic processing on the IR polarization raw image of the fourth layer imaging information to generate a 4ch monochromatic polarization image with polarization components of 0 degrees, 45 degrees, 90 degrees and 135 degrees, and outputs the monochromatic polarization image as polarization information.

[0242] Figure 34 This is a table comparing the sensitivity and resolution between the seventh configuration example and the comparative example of imaging unit 111.

[0243] In the seventh configuration example, light of the R, G, and B colors is received before polarization filter 153, while in the comparative example, light is received after polarization filter 32. Therefore, regarding RGB information, the seventh configuration example receives approximately twice the amount of light, and the sensitivity is approximately twice as high. In the sixth configuration example, pixel signals for all three colors (R, G, and B) are obtained in all pixels, and thus the RGB resolution is approximately 16 times higher. The light components (wavelengths) used to generate the RGB information are RGB in both examples.

[0244] Regarding polarization information, the sensitivity is equal. Since IR polarization information can be obtained in all pixels, the resolution is approximately four times greater. The light component (wavelength) used to generate the polarization information is infrared (IR) light.

[0245] Therefore, according to the polarization sensor 100 including the imaging unit 111 according to the seventh configuration example, the resolution and sensitivity can be improved compared to the polarization sensor 10 of the comparative example. RGB information and polarization information can be acquired simultaneously in the same pixel, and high-quality RGB information can be acquired. Both RGB information and polarization information can be improved in terms of resolution. In the seventh configuration example, light with arbitrary color components of R, G, B, and IR is used as a signal, and light can be used as a signal without waste.

[0246] <10. Summary of the first to seventh configuration examples of the imaging unit>

[0247] Figure 35 to Figure 37 It is a diagram summarizing the features of the first configuration example to the seventh configuration example and the comparative example of the imaging unit 111.

[0248] Figure 35 It is a diagram summarizing the features of the comparison examples and the first to third configuration examples.

[0249] In the cross-sectional view, the pixel structure of the polarization sensor 10 of the comparative example has a configuration obtained by sequentially and vertically stacking a color filter 31, a polarization filter 32, and a semiconductor substrate 33 on which a photodiode 41 is formed, in the form of a Bayer array or a tetrahedral Bayer array from the light incident surface side.

[0250] Because photodiode 41 performs photoelectric conversion on the light passing through polarization filter 32, the sensitivity of RGB information is degraded. Furthermore, since four adjacent pixels of the same color (R, G, B) have different polarization components, the four pixels need to be combined, which also degrades the resolution. Therefore, in the polarization sensor 10 of the comparative example, the image quality of the RGB image is degraded.

[0251] The pixel structure of the polarization sensor 100 in the first configuration example is configured by vertically stacking a color filter 151, an organic photoelectric conversion film 152, a polarization filter 153, and a semiconductor substrate 154 on which a photodiode 161 is formed, in a Bayer array or a tetragonal Bayer array, from the light incident surface side. In the same pixel, the polarization sensor 100 of the first configuration example generates an RGB raw image having a tetragonal Bayer array or a Bayer array as first-layer imaging information, and uses infrared light (IR) to generate an IR polarization raw image having polarization components of a tetragonal Bayer array or a Bayer array as second-layer imaging information.

[0252] The polarization sensor 100 in the first configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, it can improve resolution and sensitivity. According to the first configuration example, it is easy to use because IR polarization information can be obtained as polarization information.

[0253] The pixel structure of the polarization sensor 100 in the second configuration example is configured by sequentially and vertically stacking an IR cutoff filter 181, an organic photoelectric conversion film 152 in the form of a Bayer array or a tetragonal Bayer array, a polarization filter 153, and a semiconductor substrate 154 on which a photodiode 161 is formed, from the light incident surface side. In the same pixel, the polarization sensor 100 of the second configuration example generates an RGB original image having an RGB array in the form of a tetragonal Bayer array or a Bayer array as first layer imaging information, and generates an RGB complementary color polarization original image as second layer imaging information, in which the arrays of RGB complementary colors and polarization components are in the form of a tetragonal Bayer array or a Bayer array.

[0254] The polarization sensor 100 in the second configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, resolution and sensitivity can be improved. According to the second configuration example, since the polarization information of the RGB complementary colors can be directly obtained, light white balance processing can be performed.

[0255] The pixel structure of the polarization sensor 100 in the third configuration example is configured by sequentially and vertically stacking an organic photoelectric conversion film 152, a polarization filter 153, and a semiconductor substrate 154 on which a photodiode 161 is formed, arranged in a Bayer array or a tetragonal Bayer array, from the light incident surface side. In the same pixel, the polarization sensor 100 of the third configuration example generates an RGB original image having an RGB array arranged in a tetragonal Bayer array or a Bayer array as the first layer of imaging information, and generates an RGB complementary color IR polarization original image as the second layer of imaging information, in which the array of RGB complementary colors including infrared light and polarization components is arranged in a tetragonal Bayer array or a Bayer array.

[0256] The polarization sensor 100 in the third configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, it can improve resolution and sensitivity. According to the third configuration example, with simple configuration, light can be used as a signal without waste.

[0257] Figure 36 This is a diagram summarizing the features of the fourth through sixth configuration examples.

[0258] The pixel structure of the polarization sensor 100 in the fourth configuration example is configured by vertically stacking an IR cutoff filter 181, a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, a polarization filter 153, and a semiconductor substrate 154 on which a photodiode 161 is formed, sequentially from the light incident surface side. In the same pixel, the polarization sensor 100 of the fourth configuration example generates an image of a first color in RGB, such as a G image (which is an image of G), as first-layer imaging information; generates an image of a second color in RGB, such as an R image (which is an image of R), as second-layer imaging information; and generates a polarized original image (where the array of polarization components is a tetragonal Bayer array or a Bayer array) of a third color in RGB, such as a B-polarized original image, as third-layer imaging information. The RGB color components generated in the first to third layers of imaging information can be freely replaced.

[0259] The polarization sensor 100 in the fourth configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, it can improve resolution and sensitivity. According to the fourth configuration example, two colors in RGB can be acquired at full resolution, and polarization information of other colors can also be acquired.

[0260] The pixel structure of the polarization sensor 100 in the fifth configuration example is configured by vertically stacking an IR cutoff filter 181, a first organic photoelectric conversion film 152A, a polarization filter 153, a second organic photoelectric conversion film 152B, and a semiconductor substrate 154 on which a photodiode 161 is formed, sequentially from the light incident surface side. In the same pixel, the polarization sensor 100 of the fifth configuration example generates an image of a first color in RGB, such as a G image (which is an image of G), as first-layer imaging information; generates a polarized original image of a second color in RGB (where the array of polarization components is a tetragonal Bayer array or a Bayer array), such as an R-polarized original image, as second-layer imaging information; and generates a polarized original image of a third color in RGB (where the array of polarization components is a tetragonal Bayer array or a Bayer array), such as a B-polarized original image, as third-layer imaging information. The RGB color components generated in the first to third layers of imaging information can be freely replaced.

[0261] The polarization sensor 100 in the fifth configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, it can improve resolution and sensitivity. According to the fifth configuration example, one color among RGB can be acquired at full resolution, and the polarization information of the other two colors can also be acquired.

[0262] The pixel structure of the polarization sensor 100 in the sixth configuration example is configured by vertically stacking an IR cutoff filter 181, a polarization filter 153, a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, and a semiconductor substrate 154 on which a photodiode 161 is formed, sequentially from the light incident surface side. In the same pixel, the polarization sensor 100 of the sixth configuration example generates a polarization original image (where the array of polarization components is a tetragonal Bayer array or a Bayer array), e.g., a G polarization original image of the first color in the RGB image (where the array of polarization components is a tetragonal Bayer array or a Bayer array), e.g., an R polarization original image of the second color in the RGB image (where the array of polarization components is a tetragonal Bayer array or a Bayer array), e.g., an R polarization original image of the third color in the RGB image (where the array of polarization components is a tetragonal Bayer array or a Bayer array), e.g., a B ...) is freely replaceable.

[0263] The polarization sensor 100 in the sixth configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared to the polarization sensor 10 in the comparative example, the resolution is improved. According to the sixth configuration example, polarization information can be acquired in all RGB colors. Compared to the comparative example, the resolution is improved in both RGB information and polarization information.

[0264] Figure 37 This is a diagram summarizing the features of the seventh configuration example.

[0265] The pixel structure of the polarization sensor 100 in the seventh configuration example is configured by vertically stacking a first organic photoelectric conversion film 152A, a second organic photoelectric conversion film 152B, a third organic photoelectric conversion film 152C, a polarization filter 153, and a semiconductor substrate 154 on which a photodiode 161 is formed, sequentially from the light incident surface side. In the same pixel, the polarization sensor 100 of the seventh configuration example generates an image of a first color in RGB, such as a G image (which is an image of G), as first layer imaging information; generates an image of a second color in RGB, such as an R image (which is an image of R), as second layer imaging information; and generates an image of a third color in RGB, such as a B image (which is an image of B), as third layer imaging information. The RGB color components generated in the first to third layer imaging information can be freely replaced. Furthermore, in the same pixel, the polarization sensor 100 of the seventh configuration example uses infrared light (IR) to generate an IR polarized raw image having polarization components arranged in a tetrahedral Bayer array or a Bayer array as fourth layer imaging information.

[0266] The polarization sensor 100 in the seventh configuration example can simultaneously acquire RGB information and polarization information of the same pixel, and can acquire high-quality RGB information. Compared with the polarization sensor 10 in the comparative example, resolution and sensitivity can be improved. According to the seventh configuration example, all colors of RGB can be acquired at full resolution, and polarization information of other colors can also be acquired. Since IR polarization information can be obtained as polarization information, it is easy to use. Light can be used as a signal without waste.

[0267] The imaging unit 111 common to the polarization sensor 100 in the first to seventh configuration examples is configured by vertically stacking an organic photoelectric conversion film 152 (first beam-splitting unit), a polarization filter 153, and a semiconductor substrate 154. The organic photoelectric conversion film is configured to perform photoelectric conversion only on incident light with a first wavelength. Light with a specific polarization direction in the incident light passes through the polarization filter. The semiconductor substrate is configured to perform photoelectric conversion on incident light with a second wavelength. The vertical position of the polarization filter 153 differs in the first to seventh configuration examples. The first to third configuration examples include one layer of organic photoelectric conversion film 152, and the fourth to sixth configuration examples include two layers of organic photoelectric conversion film 152 (first organic photoelectric conversion film 152A and second organic photoelectric conversion film 152B). The seventh configuration example includes three layers of organic photoelectric conversion film 152 (first organic photoelectric conversion film 152A to third organic photoelectric conversion film 152C).

[0268] The implementation of this disclosure is not limited to the above-described implementation, and various modifications can be made without departing from the technical scope of this disclosure.

[0269] Note that the effects described in this specification are illustrative and not restrictive, and effects other than those described in this specification may exist.

[0270] Note that the technology disclosed herein may have the following configurations. (1)

[0272] A polarization sensor configured by vertically stacking the following components:

[0273] A first beam splitting unit is configured to perform photoelectric conversion on incident light having a first wavelength.

[0274] A polarization filter, wherein light of a specific polarization direction in the incident light passes through the polarization filter; and

[0275] A semiconductor substrate configured to perform photoelectric conversion on incident light that has passed through the polarization filter and has a second wavelength. (2)

[0277] The polarization sensor described in (1) above further includes:

[0278] A color filter, wherein the color filter is located on the light incident surface side of the first beam splitter, wherein

[0279] The first beam splitter performs photoelectric conversion on light having the first wavelength corresponding to the color of the color filter, and

[0280] The semiconductor substrate performs photoelectric conversion on infrared light, which is light having the second wavelength, passing through the polarization filter. (3)

[0282] According to the polarization sensor described in (1) or (2) above, wherein

[0283] The first beam splitting unit generates an RGB raw image as the first layer of imaging information, and

[0284] The semiconductor substrate generates an IR polarization raw image as the second layer of imaging information. (4)

[0286] The polarization sensor described in (1) above further includes:

[0287] An infrared light cutoff filter is located on the light incident surface side of the first beam splitter and is configured to absorb infrared light, wherein...

[0288] The first beam splitter absorbs and photoelectrically converts the incident light having the first wavelength, and transmits light having the second wavelength.

[0289] The semiconductor substrate performs photoelectric conversion on the incident light that has passed through the polarization filter and has the second wavelength. (5)

[0291] According to the polarization sensor described in (4) above, wherein

[0292] The light with the second wavelength is the complementary color of the first wavelength. (6)

[0294] According to the polarization sensor described in (4) or (5) above, wherein

[0295] The first beam splitting unit generates an RGB raw image as the first layer of imaging information, and

[0296] The semiconductor substrate generates an RGB complementary color polarized original image as the second layer of imaging information. (7)

[0298] The polarization sensor according to any one of (4) to (6) above further includes:

[0299] An RGB generation unit is configured to generate an RGB image from the original RGB image and output the RGB image as RGB information.

[0300] A polarization information generation unit is configured to generate RGB complementary color images for each polarization component from the original RGB complementary color polarization image, and output the RGB complementary color images as polarization information; and

[0301] A white balance gain calculation unit is configured to calculate white balance gain using the RGB information and the polarization information. (8)

[0303] According to the polarization sensor described in (1) above, wherein

[0304] The first beam splitter absorbs and photoelectrically converts the incident light having the first wavelength, and transmits light having the second wavelength.

[0305] The semiconductor substrate performs photoelectric conversion on the incident light that has passed through the polarization filter and has the second wavelength. (9)

[0307] According to the polarization sensor described in (8) above, wherein

[0308] The light with the second wavelength is light of the complementary color of the first wavelength and infrared light. (10)

[0310] According to the polarization sensor described in (1), (8) or (9) above, wherein

[0311] The first beam splitting unit generates an RGB raw image as the first layer of imaging information, and

[0312] The semiconductor substrate generates a polarized original image of RGB complementary colors and infrared light as the second layer of imaging information. (11)

[0314] The polarization sensor described in (1) above further includes:

[0315] An infrared light cutoff filter, located on the light incident surface side of the first beam splitter and configured to absorb infrared light; and

[0316] A second beam splitter is located between the first beam splitter and the polarization filter, and is configured to absorb and photoelectrically convert light having a third wavelength. (12)

[0318] According to the polarization sensor described in (11) above, wherein

[0319] The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information.

[0320] The second beam splitting unit generates an image of the second color in RGB as the second layer of imaging information, and

[0321] The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information. (13)

[0323] The polarization sensor described in (1) above further includes:

[0324] An infrared light cutoff filter, located on the light incident surface side of the first beam splitter and configured to absorb infrared light; and

[0325] The second beam splitter is located between the polarization filter and the semiconductor substrate and is configured to absorb and photoelectrically convert light having a third wavelength. (14)

[0327] According to the polarization sensor described in (13) above, wherein

[0328] The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information.

[0329] The second beam splitting unit generates a polarized original image of the second color in RGB as the second layer imaging information, and

[0330] The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information. (15)

[0332] The polarization sensor described in (1) above further includes:

[0333] An infrared light cutoff filter, located on the light incident surface side of the first beam splitter and configured to absorb infrared light; and

[0334] A second beam splitter is located between the first beam splitter and the semiconductor substrate, and is configured to absorb and photoelectrically convert light having a third wavelength, wherein...

[0335] The polarization filter is disposed between the infrared cutoff filter and the first beam splitter. (16)

[0337] According to the polarization sensor described in (15) above, wherein

[0338] The first beam splitting unit generates a polarized original image of the first color in RGB as the first layer of imaging information.

[0339] The second beam splitting unit generates a polarized original image of the second color in RGB as the second layer imaging information, and

[0340] The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information. (17)

[0342] The polarization sensor described in (1) above further includes:

[0343] A second beam-splitting unit, configured to absorb and photoelectrically convert light having a third wavelength; and

[0344] The third beam splitter is configured to absorb and photoelectrically convert light having a fourth wavelength, wherein...

[0345] The polarization sensor is configured by vertically stacking the first beam splitter, the second beam splitter, the third beam splitter, the polarization filter, and the semiconductor substrate in sequence from the light incident surface side. (18)

[0347] According to the polarization sensor described in (17) above, wherein

[0348] The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information.

[0349] The second beam splitter generates an image of the second color in the RGB spectrum as the second layer of imaging information.

[0350] The third beam splitting unit generates an image of the third color in RGB as the third layer imaging information, and

[0351] The semiconductor substrate generates an IR polarization raw image as the fourth layer of imaging information. (19)

[0353] According to any one of (1) to (18) above, the polarization sensor wherein

[0354] The first spectral splitting unit includes an organic photoelectric conversion film.

[0355] Reference Symbol List

[0356] 100 polarization sensor

[0357] 111 Imaging Units

[0358] 112 Signal Processing Unit

[0359] 121 pixels

[0360] 122 pixel array unit

[0361] 151 Color Filter

[0362] 152 Organic photoelectric conversion film

[0363] 152A First Organic Photoelectric Conversion Film

[0364] 152B Second Organic Photoelectric Conversion Film

[0365] 152C Third Organic Photoelectric Conversion Film

[0366] 153 Polarization Filter

[0367] 154 Semiconductor substrate

[0368] 161 photodiode

[0369] 171 RGB generation units

[0370] 172 Polarization Information Generation Unit

[0371] 181 IR cutoff filter

[0372] 201 RGB generation units

[0373] 202 Polarization Information Generation Unit

[0374] 203 WB Gain Calculation Unit

[0375] 211 RGB generation units

[0376] 212 Polarization Information Generation Unit

[0377] 261 Polarization Information Generation Unit

[0378] 262RGB synthesis unit

[0379] 281 Polarization Information Generation Unit

[0380] 282 Polarization Information Generation Unit

[0381] 283RGB synthesis unit

[0382] 301 Polarization Information Generation Unit

[0383] 302 Polarization Information Generation Unit

[0384] 303 Polarization Information Generation Unit

[0385] 304RGB Synthesis Unit

[0386] 331RGB Synthesis Unit

[0387] 332 Polarization Information Generation Unit

Claims

1. A polarization sensor configured by vertically stacking the following components: A first beam splitting unit is configured to perform photoelectric conversion on incident light having a first wavelength. A polarization filter, wherein light with a specific polarization direction in the incident light passes through the polarization filter; as well as A semiconductor substrate configured to perform photoelectric conversion on incident light that has passed through the polarization filter and has a second wavelength.

2. The polarization sensor according to claim 1, further comprising: A color filter, wherein the color filter is located on the light incident surface side of the first beam splitter, wherein The first beam splitter performs photoelectric conversion on light having the first wavelength corresponding to the color of the color filter, and The semiconductor substrate performs photoelectric conversion on infrared light, which is light having the second wavelength, passing through the polarization filter.

3. The polarization sensor according to claim 1, wherein... The first beam splitting unit generates a raw red-green-blue (RGB) image as the first layer of imaging information, and The semiconductor substrate generates an infrared (IR) polarized raw image as the second layer of imaging information.

4. The polarization sensor according to claim 1, further comprising: An infrared light cutoff filter is located on the light incident surface side of the first beam splitter and is configured to absorb infrared light, wherein... The first beam splitter absorbs and photoelectrically converts the incident light having the first wavelength, and transmits light having the second wavelength. The semiconductor substrate performs photoelectric conversion on the incident light that has passed through the polarization filter and has the second wavelength.

5. The polarization sensor according to claim 4, wherein... The light with the second wavelength is the complementary color of the first wavelength.

6. The polarization sensor according to claim 1, wherein... The first beam splitting unit generates an RGB raw image as the first layer of imaging information, and The semiconductor substrate generates an RGB complementary color polarized original image as the second layer of imaging information.

7. The polarization sensor according to claim 6, further comprising: An RGB generation unit is configured to generate an RGB image from the original RGB image and output the RGB image as RGB information. A polarization information generation unit is configured to generate an RGB complementary color image for each polarization component from the original RGB complementary color polarization image, and output the RGB complementary color image as polarization information. as well as A white balance gain calculation unit is configured to calculate white balance gain using the RGB information and the polarization information.

8. The polarization sensor according to claim 1, wherein The first beam splitter absorbs and photoelectrically converts the incident light having the first wavelength, and transmits light having the second wavelength. The semiconductor substrate performs photoelectric conversion on the incident light that has passed through the polarization filter and has the second wavelength.

9. The polarization sensor according to claim 8, wherein The light with the second wavelength is light of the complementary color of the first wavelength and infrared light.

10. The polarization sensor according to claim 1, wherein... The first beam splitting unit generates an RGB raw image as the first layer of imaging information, and The semiconductor substrate generates a polarized original image of RGB complementary colors and infrared light as the second layer of imaging information.

11. The polarization sensor according to claim 1, further comprising: An infrared light cutoff filter is located on the light incident surface side of the first beam splitter and is configured to absorb infrared light. as well as A second beam splitter is located between the first beam splitter and the polarization filter, and is configured to absorb and photoelectrically convert light having a third wavelength.

12. The polarization sensor according to claim 11, wherein... The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information. The second beam splitting unit generates an image of the second color in RGB as the second layer of imaging information, and The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information.

13. The polarization sensor according to claim 1, further comprising: An infrared light cutoff filter is located on the light incident surface side of the first beam splitter and is configured to absorb infrared light. as well as The second beam splitter is located between the polarization filter and the semiconductor substrate and is configured to absorb and photoelectrically convert light having a third wavelength.

14. The polarization sensor according to claim 13, wherein The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information. The second beam splitting unit generates a polarized original image of the second color in RGB as the second layer imaging information, and The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information.

15. The polarization sensor according to claim 1, further comprising: An infrared light cutoff filter is located on the light incident surface side of the first beam splitter and is configured to absorb infrared light. as well as A second beam splitter is located between the first beam splitter and the semiconductor substrate, and is configured to absorb and photoelectrically convert light having a third wavelength, wherein... The polarization filter is disposed between the infrared cutoff filter and the first beam splitter.

16. The polarization sensor according to claim 15, wherein The first beam splitting unit generates a polarized original image of the first color in RGB as the first layer of imaging information. The second beam splitting unit generates a polarized original image of the second color in RGB as the second layer imaging information, and The semiconductor substrate generates a polarized original image of the third color in RGB as the third layer imaging information.

17. The polarization sensor according to claim 1, further comprising: The second beam splitter is configured to absorb and photoelectrically convert light having a third wavelength; as well as The third beam splitter is configured to absorb and photoelectrically convert light having a fourth wavelength, wherein... The polarization sensor is configured by vertically stacking the first beam splitter, the second beam splitter, the third beam splitter, the polarization filter, and the semiconductor substrate in sequence from the light incident surface side.

18. The polarization sensor according to claim 17, wherein The first beam splitting unit generates an image of the first color in RGB as the first layer of imaging information. The second beam splitter generates an image of the second color in the RGB spectrum as the second layer of imaging information. The third beam splitting unit generates an image of the third color in RGB as the third layer imaging information, and The semiconductor substrate generates an IR polarization raw image as the fourth layer of imaging information.

19. The polarization sensor according to claim 1, wherein The first spectral splitting unit includes an organic photoelectric conversion film.

Images