Electronic apparatus, solid-state imaging element, and signal processing device

Abstract

[Problem] The present invention makes it possible to generate a high-quality captured image even when remosaic processing is performed outside a solid-state imaging device. [Solution] An electronic apparatus is provided with: a solid-state imaging element; and a signal processing device for performing remosaic processing on an output signal of the solid-state imaging element. The solid-state imaging element includes: a plurality of pixels arrayed in two-dimensional directions; a plurality of on-chip lenses provided for individual pixel groups each including two or more of the pixels; and a phase difference / sensitivity difference characteristic removal unit for removing high-frequency components generated due to a phase difference and a sensitivity difference for the same color between two or more pixel signals of the same color, output for each of the pixel groups.

Description

Electronic device, solid-state image sensor and signal processing device

[0001] The present disclosure relates to an electronic device, a solid-state imaging device, and a signal processing device.

[0002] An image sensor with a multi-OCL structure has been proposed in which a color filter of the same color and one on-chip lens (OCL) are arranged across multiple pixels (see, for example, Patent Document 1).

[0003] By adopting a multi-OCL structure, sensitivity can be improved by adding together multiple pixels in the dark, and resolution can be improved by performing re-mosaic processing in the bright.

[0004] On the other hand, for image sensors built into mobile electronic devices, low power consumption and miniaturization are important, and there is a demand to miniaturize image sensors by performing some of the signal processing, such as re-mosaic processing, which is generally performed inside the image sensor, on a chip separate from the image sensor.

[0005] Japanese Patent Application Laid-Open No. 2022-114386

[0006] However, re-mosaic processing is an important function for image sensors, and particularly in image sensors with a multi-OCL structure, it is necessary to perform re-mosaic processing that is suited to the pixel arrangement of the pixel group to which light from the on-chip lens is incident. Therefore, even if general-purpose re-mosaic processing is performed on a chip provided outside the image sensor, a high-quality captured image cannot be obtained.

[0007] Therefore, the present disclosure provides an electronic device, a solid-state imaging device, and a signal processing device that can generate high-quality captured images even when re-mosaic processing is performed outside the solid-state imaging device.

[0008] In order to solve the above-described problems, the present disclosure provides an electronic device including: a solid-state imaging element; and a signal processing device that performs re-mosaic processing on output signals of the solid-state imaging element, wherein the solid-state imaging element has: a plurality of pixels arranged in a two-dimensional direction; a plurality of on-chip lenses provided for each pixel group including two or more of the pixels; and a phase difference / sensitivity difference characteristic removal unit that removes high-frequency components that occur due to a phase difference and a sensitivity difference between two or more pixel signals of the same color that are output for each pixel group.

[0009] The phase difference sensitivity difference characteristic removal unit may remove artifacts that occur when the on-chip lens is provided for each of the pixel groups.

[0010] The pixel group onto which the light that has passed through one of the on-chip lenses is incident may include the two or more pixels of the same color, and the phase difference / sensitivity difference characteristic removal unit may remove artifacts caused by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color that are included in the pixel group onto which the light that has passed through the on-chip lens is incident.

[0011] The solid-state imaging element may include: a first substrate having the plurality of pixels and the plurality of on-chip lenses provided for each of the pixel groups; and a second substrate stacked on the first substrate and having the phase difference sensitivity difference characteristic removal unit.

[0012] The pixel group may be formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate may output pixel signals of the predetermined pixel array having a phase difference and a sensitivity difference comparable to those in a case where the on-chip lens is provided for each pixel, after removing high-frequency components caused by a phase difference and a sensitivity difference.

[0013] The second substrate may have: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each of the pixel groups output from the first substrate; and a second processing unit that generates a pixel signal of the predetermined pixel array from the intermediate signal, with high-frequency components caused by phase differences and sensitivity differences removed.

[0014] The first processing unit may have at least one of: a color balance calculation unit that calculates a gain for adjusting the color balance of each pixel included in the local region based on the two or more pixel signals for each pixel group; a direction determination unit that determines the gradient of pixel values ​​at the position of the target pixel; and a flatness determination unit that determines whether the pixel value of the target pixel is flat or not based on the gradient of pixel values ​​determined by the direction determination unit, and the first processing unit may generate the intermediate signal that includes an output signal of at least one of the color balance calculation unit, the direction determination unit, or the flatness determination unit.

[0015] The second processing unit may include: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each of the pixel groups, based on the intermediate signals generated by the first processing unit; a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components, based on the intermediate signals generated by the first processing unit; and a phase difference and sensitivity difference characteristic removal unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase differences and sensitivity differences have been removed, based on the phase difference components, sensitivity differences, and the low-frequency components.

[0016] The first processing unit or the second processing unit may have a defect detection unit that detects defective pixels, and the phase difference / sensitivity difference characteristic removal unit may generate a pixel signal of the predetermined pixel array from which high frequency components caused by the phase difference and sensitivity difference have been removed, based on the phase difference and sensitivity difference components, the low frequency components, and the defective pixels detected by the defect detection unit.

[0017] The signal processing device may output pixel signals of a predetermined reference array after performing the re-mosaic processing.

[0018] The reference array may be a Bayer array, and the array of the pixel group may be an array other than the Bayer array.

[0019] The signal processing device may be built into a third substrate that is arranged separately from the first substrate and the second substrate that are stacked.

[0020] The signal processing device may perform the re-mosaic processing that is performed when the on-chip lens is provided for each pixel.

[0021] According to the present disclosure, there is provided a solid-state imaging device comprising: a first substrate having a plurality of pixels arranged in a two-dimensional direction and a plurality of on-chip lenses provided for each pixel group including two or more of the pixels; and a second substrate stacked on the first substrate, which outputs signals from the plurality of pixel signals output from the first substrate, from which high-frequency components caused by phase differences and sensitivity differences contained in the signals have been removed, without performing re-mosaic processing.

[0022] The pixel group onto which light that has passed through one of the on-chip lenses is incident may include the two or more pixels of the same color, and the second substrate may remove artifacts caused by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color that are included in the pixel group onto which light that has passed through the on-chip lens is incident.

[0023] The pixel group may be formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate may output pixel signals of the predetermined pixel array having a phase difference and a sensitivity difference comparable to those in a case where the on-chip lens is provided for each pixel, after removing high-frequency components caused by a phase difference and a sensitivity difference.

[0024] The second substrate may have: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each of the pixel groups output from the first substrate; and a second processing unit that generates a pixel signal of the predetermined pixel array from the intermediate signal, with high-frequency components caused by phase differences and sensitivity differences removed.

[0025] the first processing unit may have at least one of: a color balance calculation unit that calculates a gain for adjusting the color balance of each pixel included in the local region, based on the two or more pixel signals for each pixel group; a direction determination unit that determines a gradient of pixel values ​​at a position of a pixel of interest; and a flatness determination unit that determines whether the pixel value of the pixel of interest is flat or not, based on the gradient of pixel values ​​determined by the direction determination unit; and the first processing unit may generate the intermediate signal including at least one output signal of the color balance calculation unit, the direction determination unit, or the flatness determination unit; and the second processing unit may have: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each pixel group, based on the intermediate signal generated by the first processing unit; a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components, based on the intermediate signal generated by the first processing unit; and a phase difference and sensitivity difference characteristic removal unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase difference and sensitivity difference have been removed, based on the phase difference and sensitivity difference components and the low-frequency components.

[0026] According to the present disclosure, there is provided a signal processing device including: a receiving unit that receives pixel signals output from a solid-state imaging element, from which artifacts caused by providing an on-chip lens for each pixel group including two or more pixels have been removed; a re-mosaic processing unit that performs re-mosaic processing based on the signals received by the receiving unit; and an output unit that outputs the signals that have been subjected to the re-mosaic processing.

[0027] The receiving unit may receive the pixel signals of a predetermined pixel array having a phase difference and a sensitivity difference similar to those in a case where the on-chip lens is provided for each pixel, and the re-mosaic processing unit may perform the re-mosaic processing that is performed in a case where the on-chip lens is provided for each pixel.

[0028] 1 is a block diagram showing a schematic configuration of a solid-state imaging element according to the present disclosure. FIG. 2 is a circuit diagram showing a circuit configuration of a pixel. FIG. 3 is a schematic perspective view of a solid-state imaging element with a two-layer structure. FIG. 4 is a schematic perspective view of a solid-state imaging element with a three-layer structure. FIG. 4 is a cross-sectional view showing an example of the cross-sectional structure of a solid-state imaging element according to the present disclosure. FIG. 5 is a more detailed cross-sectional view of a pixel sensor substrate. FIG. 6 is a diagram showing an example of a pixel layout and an on-chip lens arrangement according to the present disclosure. FIG. 7 is a diagram showing a pixel layout and an on-chip lens arrangement according to a first modified example. FIG. 8 is a diagram showing a pixel layout and an on-chip lens arrangement according to a second modified example. FIG. 9 is a diagram showing a pixel layout and an on-chip lens arrangement according to a third modified example. FIG. 10 is a diagram showing a pixel layout and an on-chip lens arrangement according to a fourth modified example. FIG. 11 is a diagram showing a variation in pixel values ​​of multiple pixels 32 of the same color that receive light incident through the same on-chip lens. FIG. 12 is a diagram showing an example of an image captured by a solid-state imaging element with a multi-OCL configuration. FIG. 13 is a block diagram showing a schematic configuration of an electronic device including a solid-state imaging element according to an embodiment. FIG. 14 is a block diagram showing the internal configurations of a first processing unit, a second processing unit, and a signal processing device in more detail. The present invention relates to a vehicle control system, a vehicle exterior information detection unit, and an imaging unit.

[0029] Hereinafter, embodiments of an electronic device, a solid-state imaging device, and a signal processing device will be described with reference to the drawings. The following description will focus on the main components of the electronic device, the solid-state imaging device, and the signal processing device, but the electronic device, the solid-state imaging device, and the signal processing device may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0030] 1 is a block diagram showing a schematic configuration of a solid-state imaging device 11 according to the present disclosure. As shown in FIG. 1 , the solid-state imaging device 11 according to the present disclosure includes a pixel array unit 1, a vertical drive circuit 2, a column signal processing circuit 3, a horizontal drive circuit 4, an output circuit 5, and a control circuit 6.

[0031] The pixel array unit 1 has a plurality of pixels 32 arranged in a first direction X and a second direction Y. A detailed configuration of the pixels 32 will be described later. In this specification, the first direction X may be referred to as the row direction or horizontal direction, and the second direction Y may be referred to as the column direction or vertical direction, but the first direction X and the second direction Y may be any specific direction as long as they intersect with each other.

[0032] The pixel array unit 1 has a plurality of row selection lines 9 arranged for each pixel group (pixel row) including two or more pixels 32 arranged in a first direction X, and a plurality of vertical signal lines VSL arranged for each pixel group (pixel column) including two or more pixels 32 arranged in a second direction Y. The plurality of row selection lines 9 extend in the first direction X and are arranged in the second direction Y. The plurality of vertical signal lines VSL extend in the second direction Y and are arranged in the first direction X. The plurality of pixels 32 output pixel signals corresponding to charges generated by photoelectric conversion to the corresponding vertical signal lines VSL.

[0033] A plurality of row selection lines 9 are connected to the vertical drive circuit 2. The vertical drive circuit 2 sequentially drives the plurality of row selection lines 9 using, for example, a shift register, so that pixel signals of a plurality of pixels 32 are output in parallel to a plurality of vertical signal lines VSL for each pixel row in the pixel array unit 1.

[0034] A plurality of vertical signal lines VSL are connected to the column signal processing circuit 3. The column signal processing circuit 3 has a plurality of AD converters 10 that perform analog-to-digital conversion (hereinafter referred to as AD conversion) of a plurality of pixel signals transmitted through the plurality of vertical signal lines VSL. In addition, the column signal processing circuit 3 may perform CDS (Correlated Double Sampling) processing, which detects the difference between the signal level of the reset level of each pixel 32 and the pixel signal level corresponding to the charge generated by photoelectric conversion.

[0035] The horizontal drive circuit 4 sequentially outputs horizontal scanning pulses with different phases to sequentially select the plurality of AD converters 10 in the column signal processing circuit 3. As a result, the digital pixel signals AD-converted by the plurality of AD converters 10 are sequentially output and input to the output circuit 5.

[0036] The output circuit 5 performs various digital signal processing on the digital pixel signals and outputs the processed signals. The specific content of the digital signal processing performed by the output circuit 5 is arbitrary, and may include, for example, black level adjustment or column variation correction, and may also include the CDS processing described above.

[0037] The control circuit 6 controls the operation timing of the vertical drive circuit 2, the column signal processing circuit 3, and the horizontal drive circuit 4. The control circuit 6 also supplies a reference signal used for AD conversion to the column signal processing circuit 3. Note that a circuit that generates the reference signal may be provided separately from the control circuit 6.

[0038] FIG. 2 is a circuit diagram showing the circuit configuration of the pixel 32 (hereinafter referred to as the pixel circuit 31).

[0039] As shown in FIG. 2 , the pixel circuit 31 according to the present disclosure includes a photoelectric conversion element 33 , a transfer transistor 34 , an amplification transistor 35 , a selection transistor 36 , a reset transistor 37 , and a discharge transistor 38 .

[0040] The photoelectric conversion element 33 is, for example, a photodiode 51, which accumulates charge corresponding to the amount of incident light. The transfer transistor 34 switches whether to transfer the charge accumulated in the photodiode 51 to a floating diffusion region (hereinafter referred to as FD) based on a transfer signal TRG input to its gate. The amplification transistor 35 and the selection transistor 36 form a source follower circuit. The gate of the amplification transistor 35 is connected to the FD, the drain of the amplification transistor 35 is connected to a power supply voltage node VDD, and the source of the amplification transistor 35 is connected to the drain of the selection transistor 36. The selection transistor 36 switches whether to output a pixel signal corresponding to the voltage level of the FD to the vertical signal line VSL based on a selection signal SEL input to its gate. The reset transistor 37 switches whether to initialize the voltage level of the FD to a reset level based on a reset signal RST input to its gate. The discharge transistor 38 switches whether or not to discharge the charges (for example, electrons) accumulated in the photodiode 51 to the power supply voltage node VDD based on an OFG signal input to the gate thereof.

[0041] 2 shows an example of the pixel circuit 31, and various modifications are possible. For example, by adding another transfer transistor 34 and arranging a memory between the two transfer transistors 34, it is possible to configure the pixel circuit 31 compatible with the global shutter system.

[0042] The solid-state imaging device 11 according to the present disclosure can be formed on a single substrate, or can be formed by dividing it into multiple substrates that are stacked together. Fig. 3A is a schematic perspective view of the solid-state imaging device 11 having a two-layer structure, and Fig. 3B is a schematic perspective view of the solid-state imaging device 11 having a three-layer structure.

[0043] 3A includes a stacked first substrate 21 and a second substrate 22. A pixel array unit 1 is disposed on the first substrate 21. The pixel array unit 1 includes a plurality of pixels 32 arranged in a first direction X and a second direction Y, and each pixel 32 includes a pixel circuit 31.

[0044] A logic circuit 24 is disposed on the second substrate 22. The logic circuit 24 includes the vertical drive circuit 2, column signal processing circuit 3, horizontal drive circuit 4, output circuit 5, and control circuit 6 shown in FIG. 1 . The first substrate 21 and the second substrate 22 are formed, for example, in plurals on a semiconductor wafer, and then diced into individual pieces and stacked. For this reason, the first substrate 21 can be called the first chip, and the second substrate 22 can be called the second chip.

[0045] The solid-state imaging device 11 of FIG. 3B includes a first substrate 21, a second substrate 22, and a third substrate 25, which are stacked together. In the solid-state imaging device 11 of FIG. 3B, the pixel array section 1 is divided and arranged on the first substrate 21 and the second substrate 22. Specifically, a first portion 23a of a pixel circuit 31 including a photodiode in each pixel 32 is arranged on the first substrate 21. A second portion 23b of each pixel circuit 31 is arranged on the second substrate 22. A logic circuit 24 is arranged on the third substrate 25. The circuits and the like arranged on each of the stacked substrates are arbitrary, and various modifications are possible. For example, a portion of the logic circuit 24 may be arranged on the first substrate 21 of FIG. 3A and the second substrate 22 of FIG. 3B.

[0046] The first substrate 21 and the second substrate 22 in Fig. 3A are joined by, for example, a copper-copper connection (CCC), vias, or bumps to transmit signals. Similarly, the first substrate 21 and the second substrate 22, and the second substrate 22 and the third substrate 25 in Fig. 3B are joined by, for example, a copper-copper connection (CCC), vias, or bumps to transmit signals.

[0047] (Cross-sectional structure of solid-state imaging element 11 with two-layer structure) Fig. 4 is a cross-sectional view showing an example of the cross-sectional structure of the solid-state imaging element 11 according to the present disclosure. Fig. 4 shows an example of the solid-state imaging element 11 according to the present disclosure having a two-layer structure as shown in Fig. 3A. Hereinafter, of the stacked first substrate 21 and second substrate 22, the first substrate 21 arranged on the light incident surface side may be referred to as the pixel sensor substrate 21, and the second substrate 22 arranged on the opposite side to the light incident surface may be referred to as the logic substrate 22.

[0048] The logic substrate 22 has a multilayer wiring layer 82 disposed on the upper side (on the pixel sensor substrate 21 side) of a semiconductor substrate 81 (hereinafter referred to as silicon substrate 81) made of, for example, silicon (Si). The logic circuit 24 shown in FIGS. 3A and 3B is formed on this multilayer wiring layer 82.

[0049] The multilayer wiring layer 82 has a plurality of wiring layers 83, including a top wiring layer 83a closest to the pixel sensor substrate 21, an intermediate wiring layer 83b, and a bottom wiring layer 83c closest to the silicon substrate 81, and an interlayer insulating film 84 formed between each wiring layer 83.

[0050] The plurality of wiring layers 83 are formed using, for example, copper (Cu), aluminum (Al), tungsten (W), etc. The interlayer insulating film 84 is formed, for example, using a silicon oxide film, a silicon nitride film, etc. All of the plurality of wiring layers 83 and the interlayer insulating film 84 may be formed using the same material, or two or more materials may be used depending on the layer.

[0051] A silicon through-hole 85 is formed in the silicon substrate 81, penetrating the silicon substrate 81. A connection conductor 87 is embedded in the inner wall of the silicon through-hole 85 via an insulating film 86, thereby forming a through silicon via (TSV) 88. The insulating film 86 is made of, for example, SiO ２ The insulating film can be formed of a film or a SiN film.

[0052] 4, an insulating film 86 and a connecting conductor 87 are formed along the inner wall surface, and the inside of the silicon through hole 85 is hollow, but depending on the inner diameter, the inside of the silicon through hole 85 may be entirely filled with the connecting conductor 87. In other words, the inside of the through hole may be either filled with a conductor or partially hollow. This also applies to a through chip via (TCV) 105 (described later) and the like.

[0053] The connection conductor 87 of the silicon through electrode 88 is connected to a rewiring 90 formed on the lower surface side of the silicon substrate 81. The rewiring 90 is connected to the solder ball 11e. The connection conductor 87 and the rewiring 90 can be made of, for example, copper (Cu), tungsten (W), polysilicon, or the like.

[0054] A solder mask (solder resist) 91 is formed on the lower surface of the silicon substrate 81 so as to cover the rewiring 90 and the insulating film 86 except for the area where the solder balls 11e are to be formed.

[0055] On the other hand, the pixel sensor substrate 21 includes a multilayer wiring layer 102 disposed on the underside (the logic substrate 22 side) of a semiconductor substrate 101 (hereinafter referred to as the silicon substrate 101) made of, for example, silicon (Si). The pixel circuits 31 of the first substrate 21 shown in FIGS. 3A and 3B are formed on this multilayer wiring layer 102.

[0056] The multilayer wiring layer 102 has a plurality of wiring layers 103, including a top wiring layer 103a closest to the silicon substrate 101, an intermediate wiring layer 103b, and a bottom wiring layer 103c closest to the logic substrate 22, and an interlayer insulating film 104 formed between each wiring layer 103.

[0057] The materials used for the plurality of wiring layers 103 and the interlayer insulating film 104 can be the same as the materials for the above-described wiring layer 83 and interlayer insulating film 84. In addition, like the above-described wiring layer 83 and interlayer insulating film 84, the plurality of wiring layers 103 and interlayer insulating film 104 may be formed using one or more materials.

[0058] In the example of Figure 4, the multilayer wiring layer 102 of the pixel sensor substrate 21 is composed of three wiring layers 103, and the multilayer wiring layer 82 of the logic substrate 22 is composed of four wiring layers 83, but the total number of wiring layers is not limited to this and can be formed with any number of layers.

[0059] A photodiode 51 formed by a PN junction is formed in the silicon substrate 101 for each pixel 32. Although not shown, a transfer transistor and the like are also formed in the multilayer wiring layer 102 and the silicon substrate 101.

[0060] At predetermined positions on the silicon substrate 101 where the color filter 11c and the on-chip lens 11d are not formed, a silicon through electrode 109 connected to the wiring layer 103a of the pixel sensor substrate 21 and a chip through electrode 105 connected to the wiring layer 83a of the logic substrate 22 are formed.

[0061] The through-chip electrode 105 and the through-silicon electrode 109 are connected by a connection wiring 106 formed on the upper surface of the silicon substrate 101. An insulating film 107 is formed between the through-silicon electrode 109 and the silicon substrate 101 and between the through-silicon electrode 105 and the silicon substrate 101. Furthermore, a color filter 11c and an on-chip lens 11d are formed on the upper surface of the silicon substrate 101 with a planarizing film (insulating film) 108 interposed therebetween.

[0062] 2 has a laminated structure in which the multilayer wiring layer 102 side of the logic substrate 22 is bonded to the multilayer wiring layer 82 side of the pixel sensor substrate 21. In Fig. 4, the bonded surface between the multilayer wiring layer 82 side of the logic substrate 22 and the multilayer wiring layer 102 side of the pixel sensor substrate 21 is indicated by a dashed line.

[0063] In addition, in the solid-state imaging element 11, the wiring layer 103 of the pixel sensor substrate 21 and the wiring layer 83 of the logic substrate 22 are connected by two through-silicon electrodes, namely, the through-silicon electrode 109 and the chip through-electrode 105, and the wiring layer 83 of the logic substrate 22 and the solder ball (rear electrode) 11e are connected by the through-silicon electrode 88 and the rewiring 90. This allows the solid-state imaging element 11 to be made as small as possible. Furthermore, by forming a cavity-less structure between the solid-state imaging element 11 and the glass substrate 12 and bonding them together with adhesive 13, the height can also be reduced.

[0064] Therefore, the solid-state imaging device 11 shown in FIG. 1 can realize a more compact semiconductor device (semiconductor package).

[0065] (Detailed Cross-Sectional Structure of Pixel Sensor Substrate) Fig. 5 is a more detailed cross-sectional view of the pixel sensor substrate. As shown in Fig. 5, the pixel sensor substrate 21 of this embodiment further includes a silicon substrate 101 having a plurality of pixels 32, a multilayer wiring layer 102 arranged on the front surface side of the silicon substrate 101, and an insulating film 220 having a fixed charge (hereinafter referred to as a fixed charge film), an insulating film 221, a light-shielding film 225, a planarizing film 108, a color filter 11c, and an on-chip lens 11d, which are arranged in this order on the back surface side of the silicon substrate 101.

[0066] The silicon substrate 101 has a thickness of, for example, 1 μm or more and 6 μm or less. A plurality of pixels 32, each having a photodiode 51 and a plurality of pixel transistors, are formed in a two-dimensional matrix in the pixel region of the silicon substrate 101. Adjacent photodiodes 51 are electrically isolated from each other by inter-pixel isolation portions 219.

[0067] The photodiode 51 has p-type regions 223, 224 formed on the front and back sides of the silicon substrate 101, and an n-type region 222 formed therebetween. A pn ​​junction is formed between the p-type regions 223, 224 and the n-type region 222. In the photodiode 51, signal charges corresponding to the amount of incident light are generated and accumulated in the n-type region 222. Furthermore, electrons that cause dark current generated at the interface of the silicon substrate 101 are absorbed by holes, which are majority carriers in the p-type regions 223, 224 formed on the front and back sides of the silicon substrate 101, thereby suppressing the dark current. Furthermore, each photodiode 51 is electrically isolated by a p-type region 218 and an inter-pixel isolation portion 219 formed in the p-type region 218.

[0068] 5, the FD 55 has an n+ type region formed by ion-implanting a high concentration of n-type impurities into a p- well layer 229 formed on the surface side of the silicon substrate 101. In addition, a transfer gate electrode 216 which is the gate electrode of the transfer transistor 34 is formed on the surface side of the silicon substrate 101 between the photodiode 51 and the FD 55 via a gate insulating film 217.

[0069] The inter-pixel isolation portion 219 has a trench isolation structure. For example, the inter-pixel isolation portion 219 includes a trench 239 formed in the depth direction from the back surface side of the silicon substrate 101, a fixed charge film 220 formed so as to cover the inner surface of the trench 239, and an insulating film 221 buried in the trench 239 via the fixed charge film 220. The inter-pixel isolation portion 219 is formed by being dug into a p-type region 218 formed in the silicon substrate 101. The inter-pixel isolation portion 219 is formed, for example, in a lattice shape so as to surround the pixel 32. Furthermore, when a pixel transistor is formed between two adjacent photodiodes 51, the inter-pixel isolation portion 219 is arranged so as to overlap the FD 55 and the source / drain regions of the pixel transistor in a planar view.

[0070] The inter-pixel isolation portion 219 is formed to a depth that reaches the p-well layer 29 in which the pixel transistors are formed, but does not reach the FD 55 or the source / drain regions. That is, the inter-pixel isolation portion 219 is formed from the back surface (top surface in FIG. 5 ) of the silicon substrate 101 to a position midway in the thickness direction of the silicon substrate 101.

[0071] The trench 239 constituting the inter-pixel isolation portion 219 opens to the rear surface side of the silicon substrate 101 and has a bottom surface within the silicon substrate 101. The trench 239 is formed to a depth that does not reach the surface of the silicon substrate 101. For example, when the silicon substrate 101 has a thickness of 1 μm or more and 6 μm or less, the trench 239 is formed to a depth of 0.25 μm or more and 5.0 μm or less from the rear surface of the silicon substrate 101.

[0072] 5 shows a case where the inter-pixel isolation portion 219 is formed to a depth that reaches the p-well layer 229, but it does not necessarily have to be deep enough to reach the p-well layer 229. For example, the inter-pixel isolation portion 219 may be formed so as not to reach the p-well layer 229, but to remain within the p-type region 218. Even in a configuration in which the inter-pixel isolation portion 219 does not reach the p-well layer 229, the effect of insulating isolation can be obtained.

[0073] The fixed charge film 220 formed in the trench 239 is formed on the inner peripheral surface and bottom surface of the trench 239, and also on the entire back surface of the silicon substrate 101. In the following description, the inner peripheral surface and bottom surface of the trench 239 will be collectively referred to as the "inner wall surface." The fixed charge film 220 is preferably made of a material that can generate fixed charges and strengthen pinning by being deposited on a substrate such as silicon, and a high refractive index material film or a high dielectric film having a negative charge can be used.

[0074] Specific examples of the material of the fixed charge film 220 include oxides or nitrides containing at least one element selected from hafnium (Hf), aluminum (Al), zirconium (Zr), tantalum (Ta), and titanium (Ti). Examples of methods for forming the fixed charge film 220 include chemical vapor deposition (hereinafter referred to as CVD), sputtering, and atomic layer deposition (hereinafter referred to as ALD). The ALD method can be used to deposit SiO 2 , which reduces the interface state density during film formation. ２ A film having a thickness of about 1 nm can be simultaneously formed. Other materials for the fixed charge film 220 include oxides or nitrides containing at least one element selected from the group consisting of lanthanum (La), praseodymium (Pr), cerium (Ce), neodymium (Nd), promethium (Pm), samarium (Sm), europium (Eu), gadolinium (Gd), terbium (Tb), dysprosium (Dy), holmium (Ho), thulium (Tm), ytterbium (Yb), lutetium (Lu), and yttrium (Y). The fixed charge film can also be formed of a hafnium oxynitride film or an aluminum oxynitride film.

[0075] The material of the fixed charge film 220 may contain silicon (Si) or nitrogen (N) as long as the insulating properties are not impaired. The concentration is determined appropriately within the range in which the insulating properties of the film are not impaired. By adding silicon (Si) or nitrogen (N) in this way, it is possible to improve the heat resistance of the film and its ability to block ion implantation during the process.

[0076] In this embodiment, a fixed charge film 220 having a negative charge is formed on the inner wall surface of the trench 239 and the back surface of the silicon substrate 101, and therefore an inversion layer is formed on the surface in contact with the fixed charge film 220. As a result, the silicon interface is pinned by the inversion layer, and the generation of dark current is suppressed. Furthermore, when the trench 239 is formed in the silicon substrate 101, physical damage may occur on the sidewalls and bottom surface of the trench 239, which may cause depinning around the trench 239. To address this problem, in this embodiment, depinning is prevented by forming a fixed charge film 220 having a large amount of fixed charge on the sidewalls and bottom surface of the trench 239.

[0077] The insulating film 221 is buried in the trench 239 in which the fixed charge film 220 is formed, and is also formed on the entire back surface of the silicon substrate 101. The insulating film 221 is preferably formed from a material having a refractive index different from that of the fixed charge film 220, and examples of materials that can be used for the insulating film 221 include silicon oxide, silicon nitride, silicon oxynitride, and resin. Also, the insulating film 221 can be made of a material that has no positive fixed charge or has only a small amount of positive fixed charge.

[0078] Furthermore, by burying the trenches 239 in the insulating film 221, the photodiodes 51 that constitute each pixel 32 are separated by the insulating film 221. This makes it difficult for signal charges to leak into adjacent pixels 32, so that when signal charges exceeding the saturation charge amount (Qs) are generated, it is possible to reduce leakage of the overflowing signal charges into adjacent photodiodes 51. This makes it possible to suppress electronic color mixing.

[0079] Furthermore, the two-layer structure of the fixed charge film 220 and the insulating film 221 formed on the back surface, which is the incident surface side of the silicon substrate 101, functions as an anti-reflection film due to the difference in refractive index between them, thereby preventing reflection of light incident from the back surface side of the silicon substrate 101.

[0080] The light-shielding film 225 is formed in a desired region on the insulating film 221 formed on the back surface of the silicon substrate 101, and in the pixel region, is formed in a lattice pattern so as to open the photodiodes 51. That is, the light-shielding film 225 is formed in a position corresponding to the inter-pixel isolation portion 219. The light-shielding film 225 is formed in a position overlapping the inter-pixel isolation portion 219 in a planar view. The material constituting the light-shielding film 225 may be any material that blocks light, and may be, for example, tungsten (W), aluminum (Al), or copper (Cu).

[0081] The planarization film 108 is formed on the entire surface of the insulating film 221 including the light-shielding film 225, thereby flattening the rear surface of the silicon substrate 101. The planarization film 108 may be made of an organic material such as a resin.

[0082] The color filters 11c are formed on the upper surface of the planarization film 108, and are formed to correspond to, for example, R (red), G (green), and B (blue) for each pixel 32. Light of a desired wavelength is transmitted through the color filters 11c, and the transmitted light is incident on the photodiodes 51 in the silicon substrate 101.

[0083] The on-chip lens 11d is formed on the upper surface of the color filter 11c. The on-chip lens 11d condenses the irradiated light, and the condensed light is efficiently incident on each photodiode 51 via the color filter 11c.

[0084] In the pixel sensor substrate 21 having the above configuration, light is irradiated from the back surface (top surface in FIG. 5 ) of the silicon substrate 101, and the light that has passed through the on-chip lens 11 d and the color filter 11 c is photoelectrically converted by the photodiode 51 to generate a signal charge. The signal charge generated by the photodiode 51 is then output as a pixel signal by a vertical signal line VSL41 (see FIG. 4 , for example) that is configured as a part of the multilayer wiring layer 102 via a pixel transistor formed on the front surface side of the silicon substrate 101.

[0085] (Pixel Layout and OCL) Fig. 6 is a diagram showing an example of a pixel layout and an arrangement of on-chip lenses 11d according to the present disclosure. As shown in Fig. 6, one on-chip lens 11d is arranged for each pixel group including 2 x 2 pixels 32 of the same color arranged adjacently in the first direction X and the second direction Y. Adjacent pixel groups in the first direction X and the second direction Y are of different colors, with two green pixel groups, one red pixel group, and one blue pixel group. In this way, each pixel group is arranged in a Bayer array.

[0086] 5, an inter-pixel separator 219 is disposed in the boundary region of the pixels 32. By disposing the inter-pixel separator 219 in each of the boundary regions of the same-color pixels 32 and the boundary region of the different-color pixels 32, color mixing is suppressed. Furthermore, by providing one on-chip lens 11d for each pixel group including multiple pixels 32 of the same color, sensitivity can be improved by adding the pixel values ​​of the multiple pixels 32 of the same color in the dark, and resolution can be improved by performing re-mosaic processing that changes the arrangement of the multiple pixels 32 for each color in the bright.

[0087] 6 is merely one example of the pixel layout and the arrangement of the on-chip lenses 11d, and various modifications are possible. Representative first to fourth modifications will be described below in order.

[0088] 7A is a diagram showing a pixel layout and the arrangement of on-chip lenses 11d according to a first modified example. In Fig. 7A, one on-chip lens 11d is arranged for each pixel group including 2 x 1 pixels of the same color. Five green pixel groups are arranged adjacent to each other in the first direction X and the second direction Y, and four red and blue pixel groups are arranged adjacent to each other in the first direction X and the second direction Y. The planar shape of the on-chip lens 11d is elliptical.

[0089] 7B is a diagram showing a pixel layout and the arrangement of on-chip lenses 11d according to a second modified example. In Fig. 7B, one on-chip lens 11d is arranged for each pixel group including 3 x 3 pixels of the same color. Adjacent pixel groups in the first direction X and the second direction Y have different colors, with two green pixel groups and one red and one blue pixel group arranged.

[0090] 7C is a diagram showing a pixel layout and the arrangement of on-chip lenses 11d according to a third modified example. In Fig. 7C, one on-chip lens 11d is arranged for each pixel group including 4 x 4 pixels of the same color. Adjacent pixel groups in the first direction X and the second direction Y have different colors, with two green pixel groups, one red pixel group, and one blue pixel group arranged.

[0091] 7D is a diagram showing a pixel layout and the arrangement of on-chip lenses 11d according to a fourth modified example. In FIG. 7D, one on-chip lens 11d is arranged for each pixel group including 2×2 pixels of the same color. Furthermore, in the fourth modified example, two pixel groups of 2×2 pixels of the same color are arranged in the first direction X and the second direction Y. A plurality of pixel regions are arranged in the first direction X and the second direction Y, with each pixel region including a 2×2 pixel group of the same color in FIG. 7D being a unit. Pixel regions adjacent to each other in the first direction X and the second direction Y have different colors, with two green pixel regions and one red and one blue pixel region being arranged.

[0092] In a solid-state imaging device 11 with a multi-OCL configuration in which an on-chip lens 11d is arranged for each pixel group including a plurality of pixels 32 of the same color, the pixel values ​​of the plurality of pixels 32 that receive light incident through one on-chip lens 11d are not necessarily the same, and variations occur. Fig. 8A is a diagram schematically showing variations in the pixel values ​​of the plurality of pixels 32 of the same color that receive light incident through the same on-chip lens 11d. The pixel values ​​of each pixel 32 vary for each pixel group, and the manner of variation is not necessarily uniform.

[0093] 8B is a diagram showing an example of an image captured by the solid-state imaging element 11 with a multi-OCL configuration. As shown in Fig. 8B, when the pixel values ​​of the multiple pixels 32 that receive light incident through one on-chip lens 11d fluctuate as shown in Fig. 8A, streaks that do not exist in the original image appear in the captured image, degrading image quality. Such degradation in image quality is sometimes called artifacts.

[0094] The solid-state imaging element 11 according to this embodiment is characterized by taking measures to prevent the degradation of image quality that is specific to the multi-OCL configuration as shown in FIG. 8B , and more specifically, by removing artifacts that occur when an on-chip lens 11 d is provided for each pixel group.

[0095] 9 is a block diagram showing a schematic configuration of an electronic device 41 including a solid-state imaging element 11 according to an embodiment. As shown in FIG. 9 , the electronic device 41 according to an embodiment includes the solid-state imaging element 11 and a signal processing device 42. The solid-state imaging element 11 has, for example, a two-layer structure, including a first substrate 21 and a second substrate 22 that are stacked together. A pixel array unit 1 is disposed on the first substrate 21, and a logic circuit 24 is disposed on the second substrate 22. The logic circuit 24 includes a first processing unit 43 and a second processing unit 44. Note that at least a portion of the first processing unit 43 and the second processing unit 44 may be disposed on the first substrate 21.

[0096] The first processing unit 43 generates an intermediate signal based on two or more pixel signals for each pixel group. The first processing unit 43 can be called a pre-processing unit because it performs pre-processing for the second processing unit 44. The second processing unit 44 generates a pixel signal of a predetermined pixel array based on the intermediate signal, from which high-frequency components caused by phase differences and sensitivity differences have been removed.

[0097] After removing high-frequency components resulting from phase differences and sensitivity differences, the solid-state imaging element 11 outputs pixel signals of a predetermined pixel array equivalent to that obtained when an on-chip lens 11d is provided for each pixel 32. The phase differences and sensitivity differences between adjacent pixels 32 of the same color differ between the case where an on-chip lens 11d is provided for each pixel 32 and the case where an on-chip lens 11d is provided for each pixel group. Therefore, "equivalent to the case where an on-chip lens 11d is provided for each pixel 32" means that the phase differences and sensitivity differences are comparable to those obtained when an on-chip lens 11d is provided for each pixel 32. In other words, the solid-state imaging element 11 according to this embodiment outputs pixel signals of a predetermined pixel array having phase differences and sensitivity differences comparable to those obtained when an on-chip lens 11d is provided for each pixel 32. The predetermined pixel array refers to the array of each pixel 32 constituting the pixel group of the solid-state imaging element 11. For example, if the pixel group of the solid-state imaging element 11 is arranged as shown in FIG. 6 , the solid-state imaging element 11 outputs pixel signals of a pixel row having two green pixels 32 and one red and one blue pixel 32.

[0098] The signal processing device 42 includes a receiving unit 58 , a re-mosaic processing unit 59 , and an output unit 60 .

[0099] The receiving unit 58 receives pixel signals output from the solid-state imaging device 11, from which artifacts generated by providing the on-chip lens 11d for each pixel group including two or more pixels 32 have been removed. More specifically, the receiving unit 58 receives pixel signals of a predetermined pixel array having phase differences and sensitivity differences comparable to those in the case where the on-chip lens 11d is provided for each pixel 32.

[0100] The re-mosaic processing unit 59 performs re-mosaic processing on pixel signals output from the solid-state imaging element 11. The re-mosaic processing performed by the signal processing device 42 is a general-purpose re-mosaic processing that is performed when an on-chip lens 11d is provided for each pixel 32. The signal processing device 42 is, for example, an application processor (hereinafter, may be referred to as AP).

[0101] The output unit 60 outputs the signal that has been subjected to the re-mosaic processing.

[0102] As described above, since no re-mosaic processing is performed inside the solid-state imaging element 11 according to an embodiment, the configuration of the solid-state imaging element 11 can be simplified. Furthermore, the solid-state imaging element 11 according to an embodiment performs processing to remove artifacts that occur when the on-chip lens 11d is provided for each pixel group, and outputs pixel signals equivalent to those obtained when the on-chip lens 11d is provided for each pixel 32. This allows the signal processing device 42 to perform general-purpose re-mosaic processing, thereby simplifying the configuration of the signal processing device 42. In other words, even if the signal processing device 42 performs general-purpose re-mosaic processing, the electronic device 41 according to this embodiment can generate high-quality captured images because the solid-state imaging element 11 performs artifact removal processing.

[0103] Fig. 10 is a block diagram showing in more detail the internal configurations of the first processing unit 43, the second processing unit 44, and the signal processing device 42. Note that Fig. 10 omits the receiving unit 58 and the output unit 60 in the signal processing device 42.

[0104] The first processing unit 43 has at least one of an LCB (Local Color Balance) calculation unit 50, a direction determination unit 52, and a flatness determination unit 53. Although Fig. 9 and Fig. 10 show an example in which the first processing unit 43 is provided with the LCB calculation unit 50, the direction determination unit 52, and the flatness determination unit 53, it is not necessarily required to have all three of these units.

[0105] The LCB calculation unit 50 calculates the color balance of each pixel 32 included in a local region based on two or more pixel signals for each pixel group, and calculates and outputs an LCB gain for adjusting the color balance. The local region is an area of ​​a size (e.g., 5 × 5 pixels, 6 × 6 pixels, etc.) for determining the gradient of pixel values. More specifically, the LCB calculation unit 50 performs color balance adjustment (e.g., white balance adjustment) based on the pixel values ​​of each color included in the local region, and calculates a red gain lcb_gain_r relative to green and a blue gain lcb_gain_b relative to green. The LCB calculation unit 50 supplies the two or more pixel signals for each pixel group and the gains lcb_gain_r and lcb_gain_b to the direction determination unit 52. In this specification, the gains lcb_gain_r and lcb_gain_b may be referred to as LCB gains.

[0106] The direction determination unit 52 determines the gradient of pixel values ​​at the position of the pixel of interest. More specifically, the direction determination unit 52 detects the gradient of pixel values ​​at the position of the pixel of interest for each direction based on two or more pixel signals for each pixel group and the LCB gain, and determines the direction in which the detected gradient is smallest. The direction determination unit 52 determines the gradient between pixels of the same color and the gradient between pixels of different colors.

[0107] The flatness determining unit 53 determines whether the pixel value of the pixel of interest is flat or not based on the gradient of the pixel value determined by the direction determining unit 52 .

[0108] The LCB gain calculated by the LCB calculation unit 50, the signal indicating the determination result of the direction determination unit 52, and the signal indicating the determination result of the flatness determination unit 53 are collectively called intermediate signals.

[0109] The second processing unit 44 includes a defect detection unit 54 , a phase difference sensitivity difference component extraction unit 55 , a DC component extraction unit 56 , and a phase difference sensitivity difference characteristic removal unit 57 .

[0110] The defect detection section 54 detects the defective pixels 32. The defect detection section 54 may be provided in the first processing section 43 instead of in the second processing section 44.

[0111] The phase difference and sensitivity difference component extraction unit 55 extracts phase difference and sensitivity difference components of two or more pixel signals for each pixel group based on the intermediate signal output from the first processing unit 43. The phase difference and sensitivity difference components extracted by the phase difference and sensitivity difference component extraction unit 55 are high-frequency components.

[0112] The DC component extraction unit 56 extracts a DC component based on the intermediate signal output from the first processing unit 43. More precisely, the DC component is a low-frequency component that has passed through a low-pass filter. In this specification, the low-frequency component is referred to as the DC component for convenience.

[0113] The phase difference sensitivity difference characteristic removal unit 57 generates a pixel signal of a predetermined pixel array from which high-frequency components caused by phase differences and sensitivity differences have been removed, based on a signal indicating the detection result of the defective pixel 32, the phase difference and sensitivity difference components extracted by the phase difference sensitivity difference component extraction unit 55, and the DC component extracted by the DC component extraction unit 56. The phase difference sensitivity difference characteristic removal unit 57 removes the high-frequency components caused by phase differences and sensitivity differences. More specifically, the phase difference sensitivity difference characteristic removal unit 57 generates a pixel signal of a predetermined pixel array having phase differences and sensitivity differences of the same degree as when an on-chip lens 11d is provided for each pixel 32.

[0114] In this way, the second processing unit 44 removes high-frequency components caused by the phase difference and sensitivity difference contained in the pixel signal of a predetermined pixel array output from the solid-state imaging device 11 having the on-chip lens 11d for each pixel group, and generates a pixel signal of the predetermined pixel array having a phase difference and sensitivity difference equivalent to that in the case where an on-chip lens 11d is provided for each pixel 32.

[0115] The signal processing device 42 includes a re-mosaic processing unit 59. The re-mosaic processing unit 59 performs general-purpose re-mosaic processing that is performed when an on-chip lens 11d is provided for each pixel 32. The re-mosaic processing unit 59 includes, for example, a direction determination unit 61, a flatness determination unit 62, a high-frequency G plane calculation unit 63, a DC component extraction unit 64, and a correlation interpolation processing unit 65.

[0116] The direction determining unit 61 determines the gradient of the pixel value at the position of the pixel of interest.

[0117] The flatness determination unit 53 determines whether the pixel value of the pixel of interest is flat or not based on the gradient of the pixel value determined by the direction determination unit 61 .

[0118] The high-frequency G plane calculation unit 63 performs interpolation processing over the entire image based on a signal indicating the determination result of the direction determination unit 61 and a signal indicating the determination result of the flatness determination unit 62, and generates an all-over G pixel signal. The all-over G pixel signal is a signal of high-frequency components.

[0119] The DC component extraction unit 64 extracts a DC component based on a signal indicating the determination result of the direction determination unit 61 and a signal indicating the determination result of the flatness determination unit 62. More precisely, the DC component is a low-frequency component that has passed through a low-pass filter.

[0120] The correlation interpolation processing unit 65 generates pixel signals of a reference array based on the full G pixel signals calculated by the high frequency G plane calculation unit 63 and the DC components extracted by the DC component extraction unit 64. The reference array is, for example, a Bayer array.

[0121] 11 is a block diagram showing a schematic configuration of an electronic device 41a including a solid-state imaging device 11a according to a comparative example. The electronic device 41a according to the comparative example includes a solid-state imaging device 11 and a signal processing device 42, similar to FIG. 9.

[0122] The solid-state imaging device 11a according to the comparative example has a first processing unit 43 and a second processing unit 44a, as shown in Fig. 10. The first processing unit 43 in Fig. 11 performs the same processing as the first processing unit 43 in Fig. 9, whereas the second processing unit 44a in Fig. 10 performs a processing operation different from that of the second processing unit 44a in Fig. 9.

[0123] 11 includes a re-mosaic processing unit 59a in addition to the phase difference sensitivity difference characteristic removal unit 57. The re-mosaic processing unit 59a provided in the second processing unit 44a in Fig. 11 is based on the premise that an on-chip lens 11d is provided for each pixel group, and performs re-mosaic processing optimized for the pixel array that constitutes the pixel group.

[0124] The solid-state imaging device 11a according to the comparative example performs re-mosaic processing optimized for the case where an on-chip lens 11d is provided for each pixel group, and therefore does not need to perform re-mosaic processing in the signal processing device 42 at the subsequent stage.

[0125] However, the solid-state imaging device 11a according to the comparative example has a more complex internal configuration, a larger chip size, and higher power consumption than the solid-state imaging device 11 according to this embodiment.

[0126] As described above, the solid-state imaging device 11 according to one embodiment performs a phase difference sensitivity difference characteristic removal process to remove artifacts caused by providing the on-chip lens 11d for each pixel group, but does not perform a re-mosaic process. The solid-state imaging device 11 also outputs pixel signals of a predetermined pixel array equivalent to a case where an on-chip lens 11d is provided for each pixel 32. Therefore, the signal processing device 42 downstream of the solid-state imaging device 11 can generate a high-quality captured image by performing a general-purpose re-mosaic process similar to a case where an on-chip lens 11d is provided for each pixel 32.

[0127] According to this embodiment, the configuration of the solid-state imaging element 11 can be simplified, thereby realizing miniaturization, low power consumption, and low cost of the solid-state imaging element 11, and providing a solid-state imaging element 11 suitable for mobile applications, for example.

[0128] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0129] FIG. 12 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0130] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 12, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0131] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0132] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0133] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0134] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0135] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0136] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0137] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0138] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0139] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 12, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0140] FIG. 13 is a diagram showing an example of the installation position of the imaging unit 12031.

[0141] In FIG. 13, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0142] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0143] 13 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0144] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0145] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0146] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0147] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0148] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the image capturing unit 12031 among the components described above. By applying the technology according to the present disclosure to the image capturing unit 12031, a high-quality captured image can be obtained.

[0149] The present technology may have the following configurations: (1) An electronic device including: a solid-state imaging element; and a signal processing device that performs re-mosaic processing on output signals of the solid-state imaging element, wherein the solid-state imaging element includes: a plurality of pixels arranged in a two-dimensional direction; a plurality of on-chip lenses provided for each pixel group including two or more of the pixels; and a phase difference sensitivity difference characteristic removal unit that removes high-frequency components caused by a phase difference and a sensitivity difference between two or more pixel signals of the same color output from each pixel group. (2) The electronic device described in (1), wherein the phase difference sensitivity difference characteristic removal unit removes artifacts caused by providing the on-chip lens for each pixel group. (3) The pixel group to which light that has passed through one on-chip lens is incident includes the two or more pixels of the same color, and the phase difference sensitivity difference characteristic removal unit removes artifacts caused by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color included in the pixel group to which light that has passed through the on-chip lens is incident. (4) The electronic device according to any one of (1) to (3), wherein the solid-state imaging element includes: a first substrate having the plurality of pixels and the plurality of on-chip lenses provided for each of the pixel groups; and a second substrate stacked on the first substrate and having the phase difference and sensitivity difference characteristic removal unit. (5) The electronic device according to (4), wherein the pixel groups are formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate removes high-frequency components caused by phase difference and sensitivity difference and then outputs pixel signals of the predetermined pixel array having phase difference and sensitivity difference comparable to those in a case where the on-chip lens is provided for each pixel. (6) The electronic device according to (5), wherein the second substrate includes: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each of the pixel groups output from the first substrate; and a second processing unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase difference and sensitivity difference have been removed based on the intermediate signal.(7) The electronic device described in (6), wherein the first processing unit has at least one of: a color balance calculation unit that calculates a gain to adjust the color balance of each pixel included in the local region based on the two or more pixel signals for each pixel group; a direction determination unit that determines the gradient of pixel values ​​at the position of the target pixel; and a flatness determination unit that determines whether the pixel value of the target pixel is flat or not based on the gradient of pixel values ​​determined by the direction determination unit; and the first processing unit generates the intermediate signal including at least one output signal of the color balance calculation unit, the direction determination unit, or the flatness determination unit. (8) The electronic device according to (6) or (7), wherein the second processing unit includes: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each of the pixel groups based on the intermediate signals generated by the first processing unit, a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on the intermediate signals generated by the first processing unit, and a phase difference and sensitivity difference characteristic removal unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase differences and sensitivity differences have been removed based on the phase difference and sensitivity difference components and the low-frequency components. (9) The electronic device according to (8), wherein the first processing unit or the second processing unit includes a defect detection unit that detects defective pixels, and the phase difference and sensitivity difference characteristic removal unit generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase differences and sensitivity differences have been removed based on the phase difference and sensitivity difference components, the low-frequency components, and the defective pixels detected by the defect detection unit. (10) The electronic device according to any one of (1) to (9), wherein the signal processing device outputs pixel signals of a predetermined reference array after performing the re-mosaic processing. (11) The electronic device according to (10), wherein the reference array is a Bayer array, and the array of the pixel group is an array other than the Bayer array. (12) The electronic device according to any one of (4) to (9), wherein the signal processing device is built into a third substrate that is arranged separately from the stacked first substrate and the second substrate. (13) The electronic device according to any one of (1) to (12), wherein the signal processing device performs the re-mosaic processing that is performed when the on-chip lens is provided for each pixel.(14) A solid-state imaging device comprising: a first substrate having a plurality of pixels arranged in a two-dimensional direction and a plurality of on-chip lenses provided for each pixel group including two or more of the pixels; and a second substrate stacked on the first substrate and outputting signals from the first substrate, in which high-frequency components generated by a phase difference and a sensitivity difference included in the plurality of pixel signals output from the first substrate have been removed, without performing re-mosaic processing. (15) The pixel group to which light that has passed through one of the on-chip lenses is incident includes the two or more pixels of the same color, and the second substrate removes artifacts generated by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color included in the pixel group to which light that has passed through the on-chip lens is incident. (16) The solid-state imaging device according to (14) or (15), wherein the pixel group is formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate outputs pixel signals of the predetermined pixel array having a phase difference similar to that in a case where the on-chip lens is provided for each pixel after removing high-frequency components caused by a phase difference and a sensitivity difference. (17) The solid-state imaging device according to (16), wherein the second substrate has: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each pixel group output from the first substrate; and a second processing unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by a phase difference and a sensitivity difference have been removed based on the intermediate signal.(18) The first processing unit has at least one of: a color balance calculation unit that calculates a gain for adjusting the color balance of each pixel included in the local region, based on the two or more pixel signals for each pixel group; a direction determination unit that determines a gradient of pixel values ​​at a position of a pixel of interest; and a flatness determination unit that determines whether the pixel value of the pixel of interest is flat or not, based on the gradient of pixel values ​​determined by the direction determination unit; and the first processing unit generates the intermediate signal including at least one output signal of the color balance calculation unit, the direction determination unit, or the flatness determination unit; and the second processing unit has: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each pixel group, based on the intermediate signal generated by the first processing unit; a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components, based on the intermediate signal generated by the first processing unit; and a phase difference and sensitivity difference characteristic removal unit that generates pixel signals of the predetermined pixel array from which high-frequency components caused by phase difference and sensitivity difference have been removed, based on the phase difference and sensitivity difference components and the low-frequency components. (17) The solid-state imaging element according to (19). (19) A signal processing device comprising: a receiving unit that receives pixel signals output from the solid-state imaging element, from which artifacts caused by providing an on-chip lens for each pixel group including two or more pixels have been removed, a re-mosaic processing unit that performs re-mosaic processing based on the signals received by the receiving unit, and an output unit that outputs the signals after the re-mosaic processing. (20) The signal processing device according to (19), wherein the receiving unit receives the pixel signals of a predetermined pixel array having approximately the same phase difference and sensitivity difference as in a case where the on-chip lens is provided for each pixel, and the re-mosaic processing unit performs the re-mosaic processing that is performed when the on-chip lens is provided for each pixel.

[0150] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0151] 1 pixel array section, 2 vertical drive circuit, 3 column signal processing circuit, 4 horizontal drive circuit, 5 output circuit, 6 control circuit, 9 row selection line, 10 converter, 11, 11a solid-state imaging element, 11c color filter, 11d on-chip lens, 11e solder ball, 12 glass substrate, 13 adhesive, 21 first substrate (pixel sensor substrate), 22 second substrate (logic substrate), 23a first portion, 23b second portion, 24 logic circuit, 25 third substrate, 29 well layer, 31 pixel circuit, 32 pixel, 33 photoelectric conversion element, 34 transfer transistor, 35 amplification transistor, 36 selection transistor, 37 reset transistor, 38 discharge transistor, 41 electronic device, 41a electronic device, 42 signal processing device, 43 first processing unit, 44 second processing unit, 44a second processing unit, 50 LCB calculation unit, 51 photodiode, 52 DIRECTION DETERMINATION SECTION, 53: FLATNESS DETERMINATION SECTION, 54: DEFECT DETECTION SECTION, 55: PHASE DIFFERENCE COMPONENT EXTRACTION SECTION, 56: DC COMPONENT EXTRACTION SECTION, 57: PHASE DIFFERENCE COMPONENT EXTRACTION SECTION, 58: RECEIVING SECTION, 59: RESEMASACING SECTION, 59a: RESEMASACING SECTION, 60: OUTPUT SECTION, 61: DIRECTION DETERMINATION SECTION, 62: FLATNESS DETERMINATION SECTION, 63: PLANE CALCULATION SECTION, 64: COMPONENT EXTRACTION SECTION, 65: CORRELATION INTERPOLATION SECTION, 81: SILICON SUBSTRATE, 82: MULTI-LAYER WIRING LAYER, 83: WIRED WIRING LAYER, 83a: WIRED WIRING LAYER, 83b: INTERMEDIATE WIRING LAYER, 83c: LOWER WIRING LAYER, 84: INTER-LAYER INSULATOR, 85: SILICON THROUGH-HOLE, 86: WIRED WIRING LAYER, 87: CONNECTING CONDUCTOR, 88: SILICON THROUGH-VIA, 90: REWIRED WIRING, 91: SOLDER MASK (SOLDER RESIST), 101: SILICON SUBSTRATE, 102: MULTI-LAYER WIRING LAYER, 103: WIRED WIRING LAYER, 103a: WIRED WIRING LAYER, 103a: Top wiring layer, 103b intermediate wiring layer, 103c bottom wiring layer, 104 interlayer insulating film, 105 chip through electrode, 106 connection wiring, 107 insulating film, 108 planarizing film, 109 silicon through electrode, 216 transfer gate electrode, 217 gate insulating film, 219 inter-pixel isolation portion, 220 fixed charge film, 221 insulating film, 225 light-shielding film, 229 well layer, 239 trench

Claims

1. An electronic device comprising: a solid-state imaging element; and a signal processing device that performs re-mosaic processing on an output signal of the solid-state imaging element, wherein the solid-state imaging element has a plurality of pixels arranged in a two-dimensional direction, a plurality of on-chip lenses provided for each pixel group including two or more of the pixels, and a phase difference and sensitivity difference characteristic removal section that removes high-frequency components generated due to the phase difference and sensitivity difference between two or more pixel signals of the same color that are output for each pixel group.

2. The electronic device according to claim 1, wherein the phase difference sensitivity difference characteristic elimination unit eliminates artifacts that occur when the on-chip lens is provided for each of the pixel groups.

3. The electronic device according to claim 2, wherein the pixel group into which light that has passed through one of the on-chip lenses is incident includes the two or more pixels of the same color, and the phase difference and sensitivity difference characteristic removal unit removes artifacts caused by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color included in the pixel group into which the light that has passed through the on-chip lens is incident.

4. The electronic device according to claim 1, wherein the solid-state imaging element comprises: a first substrate having the plurality of pixels and the plurality of on-chip lenses provided for each of the pixel groups; and a second substrate laminated on the first substrate and having the phase difference sensitivity difference characteristic removal section.

5. The electronic device according to claim 4, wherein the pixel group is formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate outputs pixel signals of the predetermined pixel array having the same degree of phase difference and sensitivity difference as in the case where the on-chip lens is provided for each pixel, after removing high frequency components caused by phase difference and sensitivity difference.

6. The electronic device described in claim 5, wherein the second substrate has: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each of the pixel groups output from the first substrate; and a second processing unit that generates a pixel signal for the specified pixel array from the intermediate signal, with high-frequency components caused by phase difference and sensitivity difference removed.

7. The electronic device described in claim 6, wherein the first processing unit has at least one of: a color balance calculation unit that calculates a gain to adjust the color balance of each pixel included in the local region based on the two or more pixel signals for each pixel group; a direction determination unit that determines the gradient of the pixel value at the position of the target pixel; and a flatness determination unit that determines whether the pixel value of the target pixel is flat or not based on the gradient of the pixel value determined by the direction determination unit, and the first processing unit generates the intermediate signal including at least one output signal of the color balance calculation unit, the direction determination unit, or the flatness determination unit.

8. The electronic device of claim 6, wherein the second processing unit comprises: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each pixel group based on the intermediate signal generated by the first processing unit; a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on the intermediate signal generated by the first processing unit; and a phase difference and sensitivity difference characteristic removal unit that generates a pixel signal of the specified pixel array from which effects of phase difference and sensitivity difference have been removed based on the phase difference and sensitivity difference components and the low-frequency components.

9. The electronic device according to claim 8, wherein the first processing unit or the second processing unit has a defect detection unit that detects defective pixels, and the phase difference and sensitivity difference characteristic removal unit generates a pixel signal of the specified pixel array from which high frequency components caused by the phase difference and sensitivity difference have been removed, based on the phase difference and sensitivity difference components, the low frequency components, and the defective pixels detected by the defect detection unit.

10. The electronic device according to claim 1, wherein the signal processing device outputs pixel signals in a predetermined reference array after performing the re-mosaic processing.

11. The electronic device according to claim 10, wherein the reference array is a Bayer array, and the array of the pixel group is an array other than the Bayer array.

12. The electronic device according to claim 4, wherein the signal processing device is built into a third substrate that is arranged separately from the first substrate and the second substrate that are stacked.

13. The electronic device according to claim 1, wherein the signal processing device performs the re-mosaic processing that is performed when the on-chip lens is provided for each pixel.

14. A solid-state imaging device comprising: a first substrate having a plurality of pixels arranged in a two-dimensional direction and a plurality of on-chip lenses provided for each pixel group including two or more of the pixels; and a second substrate stacked on the first substrate and outputting a signal from the first substrate, from which high-frequency components generated by phase differences and sensitivity differences contained in the plurality of pixel signals output from the first substrate have been removed, without performing re-mosaic processing.

15. The solid-state imaging element according to claim 14, wherein the pixel group onto which light that has passed through one of the on-chip lenses is incident includes the two or more pixels of the same color, and the second substrate removes artifacts caused by a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color included in the pixel group onto which the light that has passed through the on-chip lens is incident.

16. The solid-state imaging element according to claim 14, wherein the pixel group is formed by arranging the two or more pixels of the same color in a predetermined pixel array, and the second substrate outputs pixel signals of the predetermined pixel array having a phase difference and a sensitivity difference equivalent to those in the case where the on-chip lens is provided for each pixel, after removing high-frequency components caused by a phase difference and a sensitivity difference.

17. The solid-state imaging element described in claim 16, wherein the second substrate has: a first processing unit that generates an intermediate signal based on the two or more pixel signals for each pixel group output from the first substrate; and a second processing unit that generates a pixel signal for the specified pixel array from which high-frequency components caused by phase differences and sensitivity differences have been removed based on the intermediate signal.

18. The first processing unit has at least one of: a color balance calculation unit that calculates a gain to adjust the color balance of each pixel included in the local region based on the two or more pixel signals for each pixel group; a direction determination unit that determines a gradient of pixel values ​​at a position of a pixel of interest; and a flatness determination unit that determines whether the pixel value of the pixel of interest is flat or not based on the gradient of pixel values ​​determined by the direction determination unit; the first processing unit generates the intermediate signal including at least one output signal of the color balance calculation unit, the direction determination unit, or the flatness determination unit; and the second processing unit has: a phase difference and sensitivity difference component extraction unit that extracts phase difference and sensitivity difference components of the two or more pixel signals for each pixel group based on the intermediate signal generated by the first processing unit; a low-frequency component extraction unit that extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on the intermediate signal generated by the first processing unit; and a phase difference and sensitivity difference characteristic removal unit that generates a pixel signal of the predetermined pixel array from which high-frequency components caused by phase difference and sensitivity difference have been removed based on the phase difference and sensitivity difference components and the low-frequency components. The solid-state imaging device according to claim 17.

19. A signal processing device comprising: a receiving unit that receives pixel signals output from a solid-state imaging element and from which artifacts caused by providing an on-chip lens for each pixel group including two or more pixels have been removed; a re-mosaic processing unit that performs re-mosaic processing based on the signals received by the receiving unit; and an output unit that outputs the signals after the re-mosaic processing.

20. The signal processing device according to claim 19, wherein the receiving unit receives the pixel signals of a predetermined pixel array having a phase difference and a sensitivity difference similar to those in the case where the on-chip lens is provided for each pixel, and the re-mosaic processing unit performs the re-mosaic processing that is performed in the case where the on-chip lens is provided for each pixel.

Images