Electronic device, solid-state imaging element, and signal processing device

Abstract

[Problem] The present invention makes it possible to generate a high-quality captured image even when re-mosaic processing is performed outside a solid-state imaging device. [Solution] An electronic device provided with: a solid-state imaging element; and a signal processing means for performing a re-mosaic process on an output signal of the solid-state imaging element. The solid-state imaging element includes: a plurality of pixels arranged in a two-dimensional direction; a plurality of on-chip lenses disposed for each pixel group, each pixel group including two or more pixels; and a phase difference / sensitivity difference characteristic removal unit for removing a high-frequency component generated due to a phase difference and a sensitivity difference for the same color between two or more pixel signals of the same color output for each pixel group.

Description

Technical Field

[0001] This disclosure relates to electronic devices, solid-state imaging elements, and signal processing apparatus. Background Technology

[0002] Image sensors with an on-lens multiple layer (OCL) structure have been proposed, in which color filters of the same color and an OCL are arranged across multiple pixels (see, for example, Patent Document 1).

[0003] By employing a multi-OCL structure, sensitivity can be improved in darkness by adding multiple pixels together, and resolution can be improved in bright light by performing heavy mosaic processing.

[0004] Meanwhile, in image sensors built into electronic devices for mobile applications, low power consumption and miniaturization are considered important, and miniaturization is required by performing some of the signal processing (such as heavy mosaic processing) that is usually performed inside the image sensor in a separate chip.

[0005] Reference List

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2022-114386 Summary of the Invention

[0008] The technical problem to be solved by the present invention

[0009] However, heavy mosaic processing is an important function of image sensors, and specifically, in image sensors with multiple OCL structures, it is necessary to perform heavy mosaic processing on the pixel array suitable for the pixel group to which light from the on-chip lens is incident, and even when general heavy mosaic processing is performed on the chip set outside the image sensor, high-quality captured images cannot be obtained.

[0010] Therefore, this disclosure provides an electronic device, a solid-state imaging element, and a signal processing apparatus that can generate high-quality captured images even when heavy mosaic processing is performed outside the solid-state imaging device.

[0011] Technical solutions to technical problems

[0012] To address the aforementioned problems, according to this disclosure, an electronic device is provided, comprising: Solid-state imaging elements; and A signal processing device performs re-mosaic processing on the output signal of a solid-state imaging element, wherein... Solid-state imaging elements include: Multiple pixels arranged in a two-dimensional direction; Multiple on-chip lenses, each on-chip lens configured for a corresponding pixel group within a pixel group, the pixel group comprising two or more pixels; and The phase difference and sensitivity difference characteristic removal unit removes high-frequency components that are generated due to the phase difference and sensitivity difference between two or more pixel signals of the same color output for each pixel group.

[0013] The phase difference sensitivity difference removal unit can remove artifacts caused by setting on-chip lenses for each pixel group.

[0014] Light passing through one of the on-chip lenses and incident on a group of pixels can include two or more pixels of the same color; and

[0015] The phase difference and sensitivity difference removal unit can remove artifacts caused by phase difference and sensitivity difference between two or more pixel signals, which are output from two or more pixels of the same color in a pixel group that are incident on by light passing through the on-chip lens.

[0016] Solid-state imaging elements may further include: A first substrate, the first substrate including a plurality of pixels and a plurality of on-chip lenses, each on-chip lens being configured for a corresponding pixel group in a pixel group; and The second substrate is stacked on the first substrate and includes a phase difference sensitivity difference characteristic removal section.

[0017] A pixel group may include two or more pixels of the same color arranged in a predetermined pixel array; and

[0018] The second substrate can output pixel signals of a predetermined pixel array after removing high-frequency components caused by phase difference and sensitivity difference, the phase difference and sensitivity difference of the pixel signals of the predetermined pixel array being the same as the phase difference and sensitivity difference when an on-chip lens is provided for each pixel.

[0019] The second substrate may include: A first processing unit generates an intermediate signal based on two or more pixel signals of each pixel group output from a first substrate; and The second processing unit generates a pixel signal for a predetermined pixel array based on an intermediate signal, after removing high-frequency components caused by phase difference and sensitivity difference.

[0020] The first processing unit may include at least one of the following: A color balance calculation unit calculates a gain for adjusting the color balance of each pixel included in a local region, based on two or more pixel signals for each pixel group. A direction-determining unit that determines the gradient of the pixel value at the location of the target pixel, and A flatness determination unit determines whether the pixel values ​​of a target pixel are flat based on the gradient of the pixel values ​​determined by the direction determination unit; and The first processing unit can generate an intermediate signal, which includes the output signal of at least one of the color balance calculation unit, the direction determination unit, and the flatness determination unit.

[0021] The second processing unit may include: A phase difference and sensitivity difference component extraction unit extracts the phase difference and sensitivity difference components of two or more pixel signals in each pixel group based on an intermediate signal generated by a first processing unit. A low-frequency component extraction unit extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on an intermediate signal generated by a first processing unit; and The phase difference and sensitivity difference characteristic removal unit generates a pixel signal of a predetermined pixel array based on the phase difference component, sensitivity difference, and low-frequency component, thereby removing the high-frequency component caused by the phase difference and sensitivity difference.

[0022] The first processing unit or the second processing unit may include a defect detection unit for detecting defective pixels; and

[0023] The phase difference and sensitivity difference feature removal unit can generate a pixel signal of a predetermined pixel array that has removed the high-frequency components caused by the phase difference and sensitivity difference based on the phase difference and sensitivity difference components, the low-frequency components, and the defective pixels detected by the defect detection unit.

[0024] The signal processing device can output pixel signals of a predefined reference array after performing heavy mosaic processing.

[0025] The reference array can be a Bayer array; and

[0026] The array of pixel groups can be any array other than the Bayer array.

[0027] The signal processing device can be built into a third substrate that is arranged separately from the stacked first and second substrates.

[0028] The signal processing device can perform re-mosaic processing with an on-chip lens set for each pixel.

[0029] According to this disclosure, a solid-state imaging element is provided, the solid-state imaging element comprising: A first substrate includes a plurality of pixels and a plurality of on-chip lenses, the plurality of pixels being arranged in a two-dimensional direction, each on-chip lens being configured for a corresponding pixel group, the pixel group comprising two or more pixels; and A second substrate is stacked on the first substrate and outputs a signal without performing a re-mosaic process, removing high-frequency components from the multiple pixel signals output from the first substrate, which are generated due to phase and sensitivity differences.

[0030] Light passing through one of the on-chip lenses and incident on a group of pixels can include two or more pixels of the same color; and

[0031] The second substrate can remove artifacts caused by phase and sensitivity differences between two or more pixel signals, which are output from two or more pixels of the same color in a pixel group incident on light passing through an on-chip lens.

[0032] A pixel group may include two or more pixels of the same color arranged in a predetermined pixel array; and

[0033] The second substrate can output pixel signals of a predetermined pixel array after removing high-frequency components caused by phase difference and sensitivity difference, the phase difference and sensitivity difference of the pixel signals of the predetermined pixel array being the same as the phase difference and sensitivity difference when an on-chip lens is provided for each pixel.

[0034] The second substrate may include: A first processing unit generates an intermediate signal based on two or more pixel signals of each pixel group output from a first substrate; and The second processing unit generates a pixel signal for a predetermined pixel array based on an intermediate signal, after removing high-frequency components caused by phase difference and sensitivity difference.

[0035] The first processing unit may include at least one of the following: A color balance calculation unit calculates a gain for adjusting the color balance of each pixel included in a local region, based on two or more pixel signals for each pixel group. A direction-determining unit that determines the gradient of the pixel value at the location of the target pixel, and A flatness determination unit determines whether the pixel values ​​of a target pixel are flat based on the gradient of the pixel values ​​determined by the direction determination unit. The first processing unit can generate an intermediate signal, which includes an output signal of at least one of a color balance calculation unit, a direction determination unit, and a flatness determination unit; and The second processing unit may include: The phase difference and sensitivity difference component extraction unit extracts the phase difference and sensitivity difference components of two or more pixel signals in each pixel group based on the intermediate signal generated by the first processing unit. A low-frequency component extraction unit extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on an intermediate signal generated by a first processing unit. The phase difference and sensitivity difference characteristic removal unit generates a pixel signal of a predetermined pixel array based on the phase difference and sensitivity difference components and low-frequency components, thereby removing the high-frequency components caused by the phase difference and sensitivity difference.

[0036] According to this disclosure, a signal processing apparatus is provided, the signal processing apparatus comprising: The receiving unit receives pixel signals output from a solid-state imaging element and removes artifacts caused by setting on-chip lenses for each pixel group, which includes two or more pixels; A heavy mosaic processing unit performs heavy mosaic processing based on signals received by a receiving unit; and The output section outputs the signal that has undergone heavy mosaic processing.

[0037] The receiving unit can receive pixel signals from a predetermined pixel array, the phase difference and sensitivity difference of which are the same as those in the case where an on-chip lens is provided for each pixel; and

[0038] The heavy mosaic processing unit can perform heavy mosaic processing when an on-chip lens is set for each pixel. Attached Figure Description

[0039] Figure 1 This is a block diagram illustrating a schematic configuration of a solid-state imaging element according to the present disclosure.

[0040] Figure 2 It is a circuit diagram showing the circuit configuration of the pixels.

[0041] Figure 3A This is a schematic three-dimensional diagram of a solid-state imaging element with a two-layer structure.

[0042] Figure 3B This is a schematic three-dimensional diagram of a solid-state imaging element with a three-layer structure.

[0043] Figure 4 This is a cross-sectional view showing an example of a cross-sectional structure of a solid-state imaging element according to the present disclosure.

[0044] Figure 5 This is a more detailed cross-sectional view of the pixel sensor substrate.

[0045] Figure 6 This is a diagram illustrating an example of the pixel layout and on-chip lens arrangement according to the present disclosure.

[0046] Figure 7A It is a diagram showing the pixel layout and on-chip lens arrangement according to the first modified example.

[0047] Figure 7B This is a diagram showing the pixel layout and on-chip lens arrangement according to the second variation.

[0048] Figure 7C This is a diagram showing the pixel layout and on-chip lens arrangement according to the third variant example.

[0049] Figure 7D This is a diagram showing the pixel layout and on-chip lens arrangement according to the fourth variation.

[0050] Figure 8A This is a schematic diagram illustrating the state of pixel value changes of multiple pixels 32 of the same color that receive light incident through the same on-sheet lens.

[0051] Figure 8B This is a diagram illustrating an example of an image captured by a solid-state imaging element with a multi-OCL configuration.

[0052] Figure 9 This is a block diagram illustrating a schematic configuration of an electronic device including a solid-state imaging element according to an embodiment.

[0053] Figure 10 This is a more detailed block diagram showing the internal configuration of the first processing unit, the second processing unit, and the signal processing device.

[0054] Figure 11 This is a block diagram illustrating a schematic configuration of an electronic device including a solid-state imaging element according to a comparative example.

[0055] Figure 12 This is a block diagram illustrating an example of a schematic configuration of a vehicle control system.

[0056] Figure 13 This is an illustrative diagram showing an example of the installation location of the vehicle's external information detection unit and imaging unit. Detailed Implementation

[0057] In the following description, embodiments of the electronic device, solid-state imaging element, and signal processing apparatus will be described with reference to the accompanying drawings. Although the main components of the electronic device, solid-state imaging element, and signal processing apparatus will be described primarily below, the electronic device, solid-state imaging element, and signal processing apparatus may have components and functions not shown or described. The following description is not intended to exclude components and functions not shown or described.

[0058] Figure 1 This is a block diagram illustrating a schematic configuration of the solid-state imaging element 11 according to the present disclosure. Figure 1 As shown, the solid-state imaging element 11 according to this disclosure includes a pixel array section 1, a vertical driving circuit 2, a column signal processing circuit 3, a horizontal driving circuit 4, an output circuit 5, and a control circuit 6.

[0059] The pixel array unit 1 includes a plurality of pixels 32 arranged in a first direction X and a second direction Y. The detailed configuration of the pixels 32 will be described later. In this specification, the first direction X may be referred to as the row direction or the horizontal direction, and the second direction Y may be referred to as the column direction or the vertical direction. The specific directions of the first direction X and the second direction Y are not limited, as long as they intersect each other.

[0060] The pixel array unit 1 includes multiple row selection lines 9 and multiple vertical signal lines VSL. The multiple row selection lines are arranged for each pixel group (pixel row) comprising two or more pixels 32 arranged in the first direction X, and the multiple vertical signal lines are arranged for each pixel group (pixel column) comprising two or more pixels 32 arranged in the second direction Y. The multiple row selection lines 9 extend in the first direction X and are arranged in the second direction Y. The multiple vertical signal lines VSL extend in the second direction Y and are arranged in the first direction X. Each of the multiple pixels 32 outputs a pixel signal corresponding to the charge generated by photoelectric conversion to its corresponding vertical signal line VSL.

[0061] Multiple row selection lines 9 are connected to the vertical drive circuit 2. The vertical drive circuit 2 uses, for example, a shift register to sequentially drive the multiple row selection lines 9. Therefore, for each pixel row of the pixel array section 1, the pixel signals of multiple pixels 32 are output in parallel to multiple vertical signal lines VSL.

[0062] Multiple vertical signal lines VSL are connected to column signal processing circuit 3. Column signal processing circuit 3 includes multiple analog-to-digital (AD) converters 10 that perform analog-to-digital conversion (hereinafter, AD conversion) of multiple pixel signals transmitted through the multiple vertical signal lines VSL. Furthermore, column signal processing circuit 3 can perform correlated double sampling (CDS) processing to detect the difference between the signal level of the reset level of each pixel 32 and the pixel signal level based on the charge generated by photoelectric conversion.

[0063] The horizontal drive circuit 4 sequentially selects multiple AD converters 10 in the column signal processing circuit 3 by sequentially outputting horizontal scanning pulses with different phases. As a result, the digital pixel signals converted by the multiple AD converters 10 are sequentially output and input to the output circuit 5.

[0064] Output circuit 5 performs various types of digital signal processing on the digital pixel signal and outputs the digital pixel signal. The specific content of the digital signal processing performed by output circuit 5 is arbitrary, and can include, for example, black level adjustment, column change correction, etc., and can also perform the aforementioned CDS processing.

[0065] Control circuit 6 controls the timing of the operation of vertical drive circuit 2, column signal processing circuit 3, and horizontal drive circuit 4. Furthermore, control circuit 6 provides a reference signal for AD conversion to column signal processing circuit 3. Note that the circuit generating the reference signal can be separate from control circuit 6.

[0066] Figure 2 This is a circuit diagram showing the circuit configuration of pixel 32 (hereinafter referred to as pixel circuit 31).

[0067] like Figure 2 As shown, the pixel circuit 31 according to this disclosure includes a photoelectric conversion element 33, a transfer transistor 34, an amplification transistor 35, a selection transistor 36, a reset transistor 37, and a discharge transistor 38.

[0068] The photoelectric conversion element 33 is, for example, a photodiode 51, and accumulates a charge corresponding to the amount of incident light. A transfer transistor 34 switches whether to transfer the charge accumulated in the photodiode 51 to the floating diffusion region (floating diffusion, hereinafter referred to as FD) based on a transfer signal TRG input to its gate. An amplifying transistor 35 and a selecting transistor 36 constitute a source follower circuit. The gate of the amplifying transistor 35 is connected to FD, the drain of the amplifying transistor 35 is connected to the power supply voltage node VDD, and the source of the amplifying transistor 35 is connected to the drain of the selecting transistor 36. The selecting transistor 36 switches whether to output a pixel signal corresponding to the voltage level of FD to the vertical signal line VSL based on a select signal SEL input to its gate. A reset transistor 37 switches whether to initialize the voltage level of FD to a reset level based on a reset signal RST input to its gate. A discharge transistor 38 switches whether to discharge the charge (e.g., electrons) accumulated in the photodiode 51 to the power supply voltage node VDD based on an OFG signal input to its gate.

[0069] Figure 2This is an example of pixel circuit 31, and it can be modified in various ways. For example, by adding another transfer transistor 34 and placing the memory between the two transfer transistors 34, a pixel circuit 31 compatible with the global shutter method can be configured.

[0070] The solid-state imaging element 11 according to this disclosure can be formed on a substrate, or it can be formed by dividing a plurality of stacked substrates. Figure 3A This is a schematic three-dimensional view of a solid-state imaging element 11 with a two-layer structure, and Figure 3B This is a schematic three-dimensional view of a solid-state imaging element 11 with a three-layer structure.

[0071] Figure 3A The solid-state imaging element 11 includes a stacked first substrate 21 and a second substrate 22. A pixel array 1 is disposed on the first substrate 21. The pixel array 1 includes a plurality of pixels 32 arranged in a first direction X and a second direction Y, and each pixel 32 includes a pixel circuit 31.

[0072] Logic circuit 24 is disposed on the second substrate 22. Logic circuit 24 includes... Figure 1 The circuit includes a vertical drive circuit 2, a column signal processing circuit 3, a horizontal drive circuit 4, an output circuit 5, and a control circuit 6. For example, multiple first substrates 21 and multiple second substrates 22 are formed on a semiconductor wafer, then monolithically diced and stacked. Therefore, the first substrate 21 can be referred to as the first chip, and the second substrate 22 can be referred to as the second chip.

[0073] Figure 3B The solid-state imaging element 11 includes a stacked first substrate 21, a second substrate 22, and a third substrate 25. Figure 3B In the solid-state imaging element 11, the pixel array section 1 is divided and arranged into a first substrate 21 and a second substrate 22. Specifically, a first portion 23a of a pixel circuit 31, including a photodiode in each pixel 32, is arranged on the first substrate 21. A second portion 23b of a plurality of pixel circuits 31 is arranged on the second substrate 22. A logic circuit 24 is arranged on a third substrate 25. The circuits, etc., arranged on each of the stacked substrates are arbitrary and various modifications are conceivable. For example, a portion of the logic circuit 24 may be arranged on... Figure 3A On the first substrate 21 and Figure 3B On the second substrate 22.

[0074] Figure 3A The first substrate 21 and the second substrate 22 are joined by, for example, copper-copper interconnects (CCC), vias, bumps, etc., to perform signal transmission. Similarly, in Figure 3BIn this configuration, the first substrate 21 and the second substrate 22, as well as the second substrate 22 and the third substrate 25, are joined by means of, for example, copper-copper interconnects (CCC), vias, bumps, etc., to perform signal transmission.

[0075] (Cross-sectional structure of the solid-state imaging device 11 with a two-layer structure)

[0076] Figure 4 This is a cross-sectional view showing an example of the cross-sectional structure of the solid-state imaging element 11 according to the present disclosure. Figure 4 The solid-state imaging element 11 according to this disclosure is shown to have, as Figure 3A An example of a two-layer structure is shown. In the following text, in the stacked first substrate 21 and second substrate 22, the first substrate 21 disposed on the light incident surface side can be referred to as pixel sensor substrate 21, and the second substrate 22 disposed on the opposite side of the light incident surface can be referred to as logic substrate 22.

[0077] The logic substrate 22 includes a multilayer wiring layer 82 disposed on the upper side (pixel sensor substrate 21 side) of a semiconductor substrate 81 (hereinafter referred to as silicon substrate 81) including, for example, silicon (Si). Figure 3A and 3B The logic circuit 24 shown is formed in a multilayer wiring layer 82.

[0078] The multilayer wiring layer 82 includes multiple wiring layers 83 and an interlayer insulating film 84 formed between the various wiring layers 83. The multiple wiring layers include the uppermost wiring layer 83a closest to the pixel sensor substrate 21, the middle wiring layer 83b, the lowermost wiring layer 83c closest to the silicon substrate 81, etc.

[0079] The multiple wiring layers 83 include, for example, copper (Cu), aluminum (Al), tungsten (W), etc. The interlayer insulating film 84 includes, for example, a silicon oxide film or a silicon nitride film. For each of the multiple wiring layers 83 and the interlayer insulating film 84, all layers may include the same material, or two or more materials may be selectively used depending on the layers.

[0080] A through-silicon cavity 85 is formed in the silicon substrate 81, penetrating the silicon substrate 81. A connecting conductor 87 is embedded in the inner wall of the through-silicon cavity 85 through an insulating film 86 to form a through-silicon via (TSV) 88. The insulating film 86 may include, for example, a SiO2 film, a SiN film, etc.

[0081] exist Figure 4In the illustrated through-silicon via 88, an insulating film 86 and a connecting conductor 87 are formed along the inner wall surface, and the interior of the through-silicon cavity 85 is hollow. However, depending on the inner diameter, the connecting conductor 87 can also be integrally embedded inside the through-silicon cavity 85. In other words, a conductor can be embedded inside the cavity, or a portion of the cavity can be hollow. This is similarly applied to through-chip vias (TCVs) such as 105, which will be described later.

[0082] The connecting conductor 87 of the through-silicon via 88 is connected to a redistribution line 90 formed on the lower surface side of the silicon substrate 81. The redistribution line 90 is connected to solder balls 11e. The connecting conductor 87 and the redistribution line 90 may include, for example, copper (Cu), tungsten (W), polysilicon, etc.

[0083] A solder resist layer (solder resist) 91 is formed on the lower surface side of the silicon substrate 81 to cover the redistribution 90 and insulating film 86 outside the area where solder balls 11e are formed.

[0084] On the other hand, the pixel sensor substrate 21 includes a multilayer wiring layer 102 disposed on the underside (logic substrate 22 side) of a semiconductor substrate 101 (hereinafter referred to as silicon substrate 101), such as silicon (Si). In the multilayer wiring layer 102, a multilayer wiring layer is formed... Figure 3A and Figure 3B The pixel circuit 31 of the first substrate 21 shown.

[0085] The multilayer wiring layer 102 includes multiple wiring layers 103 and interlayer insulating films 104 formed between each wiring layer 103. The multiple wiring layers include the uppermost wiring layer 103a closest to the silicon substrate 101, the middle wiring layer 103b, the lowermost wiring layer 103c closest to the logic substrate 22, etc.

[0086] The materials used for the multiple wiring layers 103 and the interlayer insulating film 104 can be the same type of materials as those used for the wiring layers 83 and the interlayer insulating film 84 described above. Alternatively, the multiple wiring layers 103 and the interlayer insulating film 104 can be formed by selectively using one or more materials, similar to the wiring layers 83 and the interlayer insulating film 84 described above.

[0087] Note that in Figure 4 In the example, the multilayer wiring layer 102 of the pixel sensor substrate 21 includes three wiring layers 103, and the multilayer wiring layer 82 of the logic substrate 22 includes four wiring layers 83. However, the total number of wiring layers is not limited to these, and any number of wiring layers can be used to form a multilayer wiring layer.

[0088] In the silicon substrate 101, a photodiode 51 formed by a PN junction is formed for each pixel 32. In addition, although not shown, transfer transistors and the like are also formed on the multilayer wiring layer 102 and the silicon substrate 101.

[0089] At predetermined positions on the silicon substrate 101 where the color filter 11c and on-chip lens 11d are not formed, a silicon via 109 connected to the wiring layer 103a of the pixel sensor substrate 21 and a chip via 105 connected to the wiring layer 83a of the logic substrate 22 are formed.

[0090] Chip vias 105 and silicon vias 109 are connected by interconnect wiring 106 formed on the upper surface of silicon substrate 101. Additionally, an insulating film 107 is formed between each of the silicon vias 109 and chip vias 105 and the silicon substrate 101. Furthermore, a color filter 11c and an on-chip lens 11d are formed on the upper surface of silicon substrate 101, separated by a planarization film (insulating film) 108.

[0091] As mentioned above, Figure 2 The solid-state imaging element 11 shown has a stacked structure in which the multilayer wiring layer 102 side of the logic substrate 22 and the multilayer wiring layer 82 side of the pixel sensor substrate 21 are bonded together. Figure 4 In the diagram, the bonding surface between the multilayer wiring layer 82 side of the logic substrate 22 and the multilayer wiring layer 102 side of the pixel sensor substrate 21 is indicated by a dashed line.

[0092] Furthermore, in the solid-state imaging element 11, the wiring layer 103 of the pixel sensor substrate 21 and the wiring layer 83 of the logic substrate 22 are connected through two vias: a through-silicon via 109 and a through-chip via 105. The wiring layer 83 of the logic substrate 22 and the solder balls (back electrode) 11e are connected through a through-silicon via 88 and a rewiring 90. This allows the size of the solid-state imaging element 11 to be minimized to its limit. Additionally, by forming the space between the solid-state imaging element 11 and the glass substrate 12 into a cavity-free structure and bonding the solid-state imaging element 11 and the glass substrate 12 with adhesive 13, the height can also be reduced.

[0093] Therefore, according to Figure 1 The solid-state imaging element 11 shown can realize a semiconductor device (semiconductor package) with further reduced size.

[0094] (Detailed cross-sectional structure of the pixel sensor substrate)

[0095] Figure 5 This is a more detailed cross-sectional view of the pixel sensor substrate. (Example) Figure 5As shown, the pixel sensor substrate 21 of this embodiment further includes: a silicon substrate 101 including a plurality of pixels 32; a multilayer wiring layer 102 disposed on the front surface side of the silicon substrate 101; and an insulating film (hereinafter referred to as a fixed charge film) 220 including a fixed charge; the insulating film 221, the light-shielding film 225, the planarization film 108, the color filter 11c and the on-chip lens 11d are arranged sequentially on the rear surface side of the silicon substrate 101.

[0096] The silicon substrate 101 has a thickness of, for example, 1 μm or more and 6 μm or less. In the pixel region of the silicon substrate 101, a plurality of pixels 32, each including a photodiode 51 and a plurality of pixel transistors, are formed into a two-dimensional matrix. Furthermore, adjacent photodiodes 51 are electrically isolated from each other by inter-pixel isolation portions 219.

[0097] The photodiode 51 includes p-type regions 223 and 224 formed on the front and rear surfaces of the silicon substrate 101, respectively, and an n-type region 222 formed between the p-type regions 223 and 224. A pn ​​junction is formed between the p-type regions 223 and 224 and the n-type region 222. In the photodiode 51, a signal charge corresponding to the amount of incident light is generated and accumulated in the n-type region 222. In addition, electrons that cause dark current to be generated at the interface of the silicon substrate 101 are absorbed by holes formed on the front and rear surfaces of the silicon substrate 101, which are the main charge carriers of the p-type regions 223 and 224, thereby suppressing the dark current. Furthermore, each photodiode 51 is electrically isolated by a p-type region 218 and an inter-pixel isolation portion 219 formed in the p-type region 218.

[0098] like Figure 5 As shown, FD 55 includes an (n+) type region formed by implanting a high concentration of n-type impurity ions into a p-well layer 229 formed on the front surface side of the silicon substrate 101. Furthermore, a transfer gate electrode 216, which serves as the gate electrode of the transfer transistor 34, is formed on the front surface side of the silicon substrate 101 between the photodiode 51 and FD 55, with a gate insulating film 217 between them.

[0099] The inter-pixel isolation portion 219 has a trench isolation structure. For example, the inter-pixel isolation portion 219 includes: a trench 239 formed in the depth direction from the rear surface side of the silicon substrate 101, a fixed charge film 220 formed to cover the inner surface of the trench 239, and an insulating film 221 embedded in the trench 239 through the fixed charge film 220. The inter-pixel isolation portion 219 is formed by drilling into the p-type region 218 formed in the silicon substrate 101. For example, the inter-pixel isolation portion 219 is formed in a lattice shape to surround the pixel 32. Furthermore, when the pixel transistor is formed between two adjacent photodiodes 51, the inter-pixel isolation portion 219 is arranged to overlap with the source / drain regions of the FD 55 and the pixel transistor in a planar view.

[0100] Furthermore, the inter-pixel isolation portion 219 is formed at a depth reaching the p-well layer 29 where the pixel transistor is formed, and at a depth not reaching the FD 55 and the source / drain region. That is, the inter-pixel isolation portion 219 extends from the rear surface of the silicon substrate 101 ( Figure 5 The upper surface is formed at the middle position in the thickness direction of the silicon substrate 101.

[0101] The trench 239 constituting the inter-pixel isolation portion 219 opens toward the rear surface of the silicon substrate 101 and has a bottom surface in the silicon substrate 101. The trench 239 is formed at a depth that does not reach the front surface of the silicon substrate 101. For example, when the silicon substrate 101 has a thickness of 1 μm or more and 6 μm or less, the trench 239 is formed at a depth of 0.25 μm or more and 5.0 μm or less from the rear surface of the silicon substrate 101.

[0102] Note that, although in Figure 5 The diagram shows the inter-pixel isolation portion 219 being formed with a depth reaching the p-well layer 229; however, the inter-pixel isolation portion does not necessarily need to reach the p-well layer 229. For example, the inter-pixel isolation portion 219 may not reach the p-well layer 229 and may be formed to remain within the p-type region 218. Even in the configuration where the inter-pixel isolation portion 219 does not reach the p-well layer 229, the effect of insulation isolation can still be obtained.

[0103] Furthermore, a fixed charge film 220 formed in the trench 239 is formed on the inner peripheral surface and bottom surface of the trench 239, and on the entire rear surface of the silicon substrate 101. Note that in the following description, the inner peripheral surface and bottom surface of the trench 239 are collectively referred to as the "inner wall surface". As the fixed charge film 220, a material that can enhance pinning by depositing material on a substrate including silicon to generate a fixed charge is preferably used, and a high refractive index material film or a high dielectric film with a negative charge can be used.

[0104] As a specific material for the fixed-charge film 220, an oxide or nitride containing at least one element selected from hafnium (Hf), aluminum (Al), zirconium (Zr), tantalum (Ta), or titanium (Ti) can be used. Examples of methods for forming the fixed-charge film 220 include chemical vapor deposition (CVD), sputtering, atomic layer deposition (ALD), etc. By using the ALD method, a SiO2 film with reduced interfacial states during film formation can be formed simultaneously to a thickness of approximately 1 nm. Furthermore, examples of materials other than those described above for the fixed charge film 220 include oxides, nitrides, etc., containing at least one element selected from lanthanum (La), praseodymium (Pr), cerium (Ce), neodymium (Nd), praseodymium (Pm), samarium (Sm), europium (Eu), gadolinium (Gd), terbium (Tb), dysprosium (Dy), holmium (Ho), thulium (Tm), ytterbium (Yb), lutetium (Lu), and yttrium (Y). Additionally, the fixed charge film can be formed using a hafnium oxynitride film or an aluminum oxynitride film.

[0105] Silicon (Si) or nitrogen (N) can be added to the material of the fixed charge film 220 described above, within a range that does not impair the insulating properties. Its concentration is appropriately determined within a range that does not impair the insulating properties of the film. As described above, the addition of silicon (Si) or nitrogen (N) enables the film to improve its heat resistance and its ability to block ion implantation during the process.

[0106] In this embodiment, since a fixed charge film 220 with a negative charge is formed on the inner wall surface of the trench 239 and the rear surface of the silicon substrate 101, an inversion layer is formed on the surface in contact with the fixed charge film 220. As a result, the generation of dark current is suppressed because the silicon interface is pinned by the inversion layer. Furthermore, when the trench 239 is formed in the silicon substrate 101, physical damage may occur on the sidewalls and bottom surface of the trench 239, and pinning failure may occur in the peripheral portion of the trench 239. Regarding this problem, in this embodiment, by forming a fixed charge film 220 with a large amount of fixed charge on the sidewalls and bottom surface of the trench 239, pinning failure can be prevented.

[0107] The insulating film 221 is embedded in the trench 239 where the fixed charge film 220 is formed, and is formed on the entire rear surface side of the silicon substrate 101. As the material for the insulating film 221, it is preferable to use a material with a refractive index different from that of the fixed charge film 220, and for example, silicon oxide, silicon nitride, silicon oxynitride, resin, etc., can be used. Furthermore, materials with the following characteristics can be used for the insulating film 221: no positive fixed charge or a small positive fixed charge.

[0108] Then, since the trench 239 is embedded in the insulating film 221, the photodiodes 51 constituting each pixel 32 are separated via the insulating film 221. As a result, signal charge is less likely to leak into adjacent pixels 32, and therefore, in the event of signal charge exceeding the saturation charge (Qs), leakage of signal charge that has overflowed into adjacent photodiodes 51 can be reduced. Thus, electronic color mixing can be suppressed.

[0109] Furthermore, the two-layer structure of the fixed charge film 220 and the insulating film 221 formed on the rear surface side of the silicon substrate 101, which serves as the incident surface side, acts as an anti-reflection film due to the difference in refractive index. As a result, reflection of light incident from the rear surface side of the silicon substrate 101 is prevented from occurring on the rear surface side of the silicon substrate 101.

[0110] A light-shielding film 225 is formed in a desired area on an insulating film 221 formed on the rear surface of the silicon substrate 101, and is formed in a grid shape to expose the photodiode 51 in the pixel area. That is, the light-shielding film 225 is formed at a position corresponding to the inter-pixel isolation portion 219. The light-shielding film 225 is formed at a position overlapping with the inter-pixel isolation portion 219 in a planar view. As the material constituting the light-shielding film 225, any material can be used as long as it blocks light, and for example, tungsten (W), aluminum (Al), or copper (Cu) can be used.

[0111] A planarization film 108 is formed on the entire surface of the insulating film 221, including the light-shielding film 225, thereby planarizing the surface of the back surface side of the silicon substrate 101. The planarization film 108 can be made of organic materials such as resins.

[0112] Color filters 11c are formed on the upper surface of the planarization film 108 and are formed for each pixel 32, for example, corresponding to red (R), green (G) and blue (B). In each color filter 11c, light with a desired wavelength is transmitted and the transmitted light is incident on a photodiode 51 in the silicon substrate 101.

[0113] An on-chip lens 11d is formed on the upper surface of the color filter 11c. The incident light is focused in the on-chip lens 11d, and the focused light is effectively incident on each photodiode 51 through the color filter 11c.

[0114] In the pixel sensor substrate 21 with the above configuration, from the rear surface of the silicon substrate 101 (on Figure 5Light is irradiated onto the front surface of the silicon substrate 101, and the light transmitted through the on-chip lens 11d and the color filter 11c is photoelectrically converted by the photodiode 51, thereby generating a signal charge. The signal charge generated by the photodiode 51 is then transmitted via a pixel transistor formed on the front surface of the silicon substrate 101, through vertical signal lines VSL 41 (see, for example, [reference needed]) partially configured in the multilayer wiring layer 102. Figure 4 ), which is output as a pixel signal.

[0115] (Pixel layout and OCL)

[0116] Figure 6 This is a diagram illustrating an example of the pixel layout and arrangement of the on-chip lens 11d according to this disclosure. For example... Figure 6 As shown, an on-chip lens 11d is arranged for each pixel group, each pixel group comprising 2x2 pixels 32 of the same color arranged adjacent to each other in the first direction X and the second direction Y. Adjacent pixel groups in the first direction X and the second direction Y have different colors, and one red pixel group and one blue pixel group are provided relative to the two green pixel groups. In this way, each pixel group has a Bayer array.

[0117] like Figure 5 As shown in detail, the inter-pixel isolation portion 219 is arranged in the boundary region of the pixel 32. By arranging the inter-pixel isolation portion 219 in the boundary regions of pixels 32 of the same color and pixels 32 of different colors, color mixing is suppressed. In addition, by providing an on-chip lens 11d for each pixel group including multiple pixels 32 of the same color, sensitivity can be improved in the dark by adding the pixel values ​​of multiple pixels 32 of the same color, and resolution can be improved in the bright by performing a re-mosaic process that changes the array of multiple pixels 32 of each color.

[0118] Figure 6 This is merely an example of the pixel layout and arrangement of the on-chip lens 11d, and various modifications are conceivable. Representative first to fourth variations will be described in sequence below.

[0119] Figure 7A This is a diagram showing the pixel layout and the arrangement of the on-chip lens 11d according to the first modified example. Figure 7A In this design, an on-chip lens 11d is arranged for each pixel group consisting of 2x1 pixels of the same color. Five green pixel groups are arranged adjacent to each other in the first direction X and the second direction Y, and each of the four red pixel groups and four blue pixel groups is arranged adjacent to each other in the first direction X and the second direction Y. The planar shape of the on-chip lens 11d is elliptical.

[0120] Figure 7BThis is a diagram showing the pixel layout and the arrangement of the on-chip lens 11d according to the second modified example. Figure 7B In this configuration, an on-chip lens 11d is arranged for each pixel group consisting of 3×3 pixels of the same color. Adjacent pixel groups in the first direction X and the second direction Y have different colors, and the pixel groups are arranged in such a ratio that there are two green pixel groups, and one red and one blue pixel group.

[0121] Figure 7C This is a diagram showing the pixel layout and the arrangement of the on-chip lens 11d according to the third variant example. Figure 7C In this design, an on-chip lens 11d is arranged for each pixel group consisting of 4×4 pixels of the same color. Adjacent pixel groups in the first direction X and the second direction Y have different colors, and the pixel groups are arranged in such a ratio that there are two green pixel groups, and one red and one blue pixel group.

[0122] Figure 7D This is a diagram showing the pixel layout and the arrangement of the on-chip lens 11d according to the fourth variant. Figure 7D In this embodiment, an on-chip lens 11d is arranged for each pixel group consisting of 2×2 pixels of the same color. Furthermore, in a fourth variation, two pixel groups of 2×2 pixels of the same color are arranged on each of the first direction X and the second direction Y. Figure 7D In this configuration, multiple pixel regions are arranged in units of 2×2 pixel groups of the same color along a first direction X and a second direction Y. Adjacent pixel regions along the first direction X and the second direction Y have different colors, and the pixel regions are arranged in such a ratio that there are two green pixel regions, and one red and one blue pixel region.

[0123] In a solid-state imaging element 11 with a multi-OCL configuration (where an on-chip lens 11d is arranged for each pixel group comprising a plurality of pixels 32 of the same color), the pixel values ​​of the plurality of pixels 32 that receive light incident through an on-chip lens 11d are not necessarily the same and may vary. Figure 8A This diagram schematically illustrates the state of pixel value changes for multiple pixels 32 of the same color that receive light incident through the same on-film lens 11d. For each pixel group, the pixel value of each pixel 32 changes, and the manner of change is not necessarily consistent.

[0124] Figure 8B This is a diagram illustrating an example of an image captured by a solid-state imaging element 11 with a multi-OCL configuration. For example... Figure 8B As shown, when multiple pixels 32 receive light incident through an on-chip lens 11d, the pixel values ​​are as follows: Figure 8AWhen the changes shown occur, stripes that were not present in the original image appear in the captured image, and the image quality deteriorates. This image quality degradation can be called artifacts.

[0125] The solid-state imaging element 11 according to this embodiment is characterized by taking measures to prevent the occurrence of events such as Figure 8B The image quality degradation characteristic of the multi-OCL configuration shown is, more specifically, characterized by the removal of artifacts caused by setting the on-chip lens 11d for each pixel group.

[0126] Figure 9 This is a block diagram illustrating a schematic configuration of an electronic device 41 including a solid-state imaging element 11 according to an embodiment. Figure 9 As shown, the electronic device 41 according to an embodiment includes a solid-state imaging element 11 and a signal processing device 42. The solid-state imaging element 11 has, for example, a two-layer structure and includes a stacked first substrate 21 and a second substrate 22. A pixel array portion 1 is disposed on the first substrate 21, and a logic circuit 24 is disposed on the second substrate 22. The logic circuit 24 includes a first processing unit 43 and a second processing unit 44. Note that at least a portion of the first processing unit 43 or the second processing unit 44 may also be disposed on the first substrate 21.

[0127] The first processing unit 43 generates an intermediate signal based on two or more pixel signals for each pixel group. The first processing unit 43 performs preprocessing for the second processing unit 44, and therefore can be called a preprocessing unit. The second processing unit 44 generates pixel signals for a predetermined pixel array based on the intermediate signals, wherein high-frequency components generated due to phase differences and sensitivity differences are removed from the pixel signals of the predetermined pixel array.

[0128] After removing high-frequency components caused by phase and sensitivity differences, the solid-state imaging element 11 outputs a pixel signal of a predetermined pixel array corresponding to the case where an on-chip lens 11d is provided for each pixel 32. When an on-chip lens 11d is provided for each pixel 32 and when an on-chip lens 11d is provided for each pixel group, the phase and sensitivity differences between adjacent pixels 32 of the same color are different, and "corresponds to the case where an on-chip lens 11d is provided for each pixel 32" means that the phase and sensitivity differences are the same as those in the case where an on-chip lens 11d is provided for each pixel 32. That is, the solid-state imaging element 11 according to this embodiment outputs a pixel signal of a predetermined pixel array having the same phase and sensitivity differences as those in the case where an on-chip lens 11d is provided for each pixel 32. The predetermined pixel array refers to the array of each pixel 32 constituting a pixel group of the solid-state imaging element 11. For example, when the pixel group of the solid-state imaging element 11 is as follows... Figure 6When the array is shown, the output consists of pixel signals in a pixel queue consisting of two green pixels 32 and one red and one blue pixel 32.

[0129] The signal processing device 42 includes a receiving unit 58, a heavy mosaic processing unit 59, and an output unit 60.

[0130] The receiving unit 58 receives pixel signals output from the solid-state imaging element 11, and these pixel signals are freed from artifacts caused by the on-chip lens 11d being provided for each pixel group comprising two or more pixels 32. More specifically, the receiving unit 58 receives pixel signals of a predetermined pixel array having the same phase difference and sensitivity difference as when the on-chip lens 11d is provided for each pixel 32.

[0131] The re-mosaic processing unit 59 performs re-mosaic processing on the pixel signals output from the solid-state imaging element 11. The re-mosaic processing performed by the signal processing device 42 is a general re-mosaic processing performed when an on-chip lens 11d is provided for each pixel 32. The signal processing device 42 is, for example, an application processor (hereinafter referred to as AP).

[0132] The output unit 60 outputs a signal that has undergone heavy mosaic processing.

[0133] As described above, since the heavy mosaic processing is not performed within the solid-state imaging element 11 according to the embodiment, the configuration of the solid-state imaging element 11 can be simplified. Furthermore, the solid-state imaging element 11 according to the embodiment performs a process to remove artifacts generated by setting on-chip lenses 11d for each pixel group, and outputs a pixel signal corresponding to the case where on-chip lenses 11d are set for each pixel 32. As a result, it is sufficient to perform general heavy mosaic processing in the signal processing apparatus 42, and the configuration of the signal processing apparatus 42 can be simplified. That is, even if the signal processing apparatus 42 performs general heavy mosaic processing, the electronic device 41 according to this embodiment can generate a high-quality captured image because the solid-state imaging element 11 performs artifact removal processing.

[0134] Figure 10 This is a more detailed block diagram showing the internal configuration of the first processing unit 43, the second processing unit 44, and the signal processing device 42. Note that in Figure 10 In this paper, the receiving unit 58 and the output unit 60 of the signal processing device 42 are omitted.

[0135] The first processing unit 43 includes at least one of a local color balance (LCB) calculation unit 50, a direction determination unit 52, and a flatness determination unit 53. Figure 9 and Figure 10An example is shown in which an LCB calculation unit 50, a direction determination unit 52, and a flatness determination unit 53 are provided in the first processing unit 43, but it is not always necessary to include all three.

[0136] The LCB calculation unit 50 calculates the color balance of each pixel 32 included in a local region based on two or more pixel signals of each pixel group, and calculates and outputs the LCB gain for adjusting the color balance. The local region is an area used to obtain the magnitude of the gradient of pixel values ​​(e.g., 5×5 pixels, 6×6 pixels, etc.). More specifically, the LCB calculation unit 50 performs color balance adjustment (e.g., white balance adjustment) based on the pixel values ​​of each color included in the local region, and calculates the red gain lcb_gain_r relative to green and the blue gain lcb_gain_b relative to green. The LCB calculation unit 50 provides two or more pixel signals of each pixel group, along with the gains lcb_gain_r and lcb_gain_b, to the direction determination unit 52. In this specification, the gains lcb_gain_r and lcb_gain_b may be referred to as LCB gains.

[0137] The direction determination unit 52 determines the gradient of the pixel value at the location of the target pixel. More specifically, the direction determination unit 52 detects the gradient of the pixel value at the location of the target pixel for each direction based on two or more pixel signals and LCB gain for each pixel group, and determines the direction with the smallest detected gradient. The direction determination unit 52 determines the gradient between pixels of the same color and the gradient between pixels of different colors.

[0138] The flatness determination unit 53 determines whether the pixel value of the target pixel is flat based on the gradient of the pixel value determined by the direction determination unit 52.

[0139] The LCB gain calculated by the LCB calculation unit 50, the signal indicating the determination result of the direction determination unit 52, and the signal indicating the determination result of the flatness determination unit 53 are collectively referred to as intermediate signals.

[0140] The second processing unit 44 includes a defect detection unit 54, a phase difference sensitivity difference component extraction unit 55, a DC component extraction unit 56, and a phase difference sensitivity difference characteristic removal unit 57.

[0141] The defect detection unit 54 detects defective pixels 32. The defect detection unit 54 may be provided in the first processing unit 43 instead of the second processing unit 44.

[0142] The phase difference and sensitivity difference component extraction unit 55 extracts the phase difference and sensitivity difference components of two or more pixel signals in each pixel group based on the intermediate signal output from the first processing unit 43. The phase difference and sensitivity difference components extracted by the phase difference and sensitivity difference component extraction unit 55 are high-frequency components.

[0143] The DC component extraction unit 56 extracts the DC component based on the intermediate signal output from the first processing unit 43. More precisely, the DC component is the low-frequency component that has passed through the low-pass filter. In this specification, for convenience, the low-frequency component is referred to as the DC component.

[0144] The phase difference and sensitivity difference characteristic removal unit 57 generates a pixel signal of a predetermined pixel array, which removes high-frequency components caused by phase difference and sensitivity difference, based on the signal indicating the detection result of the defective pixel 32, the phase difference and sensitivity difference components extracted by the phase difference and sensitivity difference component extraction unit 55, and the DC component extracted by the DC component extraction unit 56. The phase difference and sensitivity difference characteristic removal unit 57 removes high-frequency components caused by phase difference and sensitivity difference. More specifically, the phase difference and sensitivity difference characteristic removal unit 57 generates a pixel signal of a predetermined pixel array having the same phase difference and sensitivity difference as when an on-chip lens 11d is provided for each pixel 32.

[0145] As described above, the second processing unit 44 removes high-frequency components from the pixel signals of the predetermined pixel array, which are generated due to phase difference and sensitivity difference. The pixel signals of the predetermined pixel array are output from a solid-state imaging element 11 that includes an on-chip lens 11d for each pixel group. The second processing unit generates pixel signals of the predetermined pixel array that have the same phase difference and sensitivity difference as when an on-chip lens 11d is provided for each pixel 32.

[0146] The signal processing apparatus 42 includes a re-mosaic processing unit 59. The re-mosaic processing unit 59 performs general re-mosaic processing when an on-chip lens 11d is provided for each pixel 32. The re-mosaic processing unit 59 includes, for example, an orientation determination unit 61, a flatness determination unit 62, a high-frequency G-plane calculation unit 63, a DC component extraction unit 64, and a correlation interpolation processing unit 65.

[0147] The orientation determination unit 61 determines the gradient of the pixel value at the position of the target pixel.

[0148] The flatness determination unit 53 determines whether the pixel value of the target pixel is flat based on the gradient of the pixel value determined by the direction determination unit 61.

[0149] The high-frequency G-plane calculation unit 63 performs interpolation processing on the entire region of the image based on the signals indicating the determination results of the direction determination unit 61 and the flatness determination unit 62, and generates a full-image G-pixel signal. The full-image G-pixel signal is a signal of high-frequency components.

[0150] The DC component extraction unit 64 extracts the DC component based on the signal indicating the determination result of the direction determination unit 61 and the signal indicating the determination result of the flatness determination unit 62. More precisely, the DC component is the low-frequency component that has passed through the low-pass filter.

[0151] The correlation interpolation processing unit 65 generates the pixel signal of the reference array based on the full-image G-pixel signal calculated by the high-frequency G-plane calculation unit 63 and the DC component extracted by the DC component extraction unit 64. The reference array is, for example, a Bayer array.

[0152] Figure 11 This is a block diagram illustrating a schematic configuration of an electronic device 41a including a solid-state imaging element 11a according to a comparative example. Figure 9 Similarly, the electronic device 41a according to the comparative example includes a solid-state imaging element 11 and a signal processing device 42.

[0153] like Figure 10 As shown, the solid-state imaging element 11a according to the comparative example includes a first processing unit 43 and a second processing unit 44a. Figure 11 The first processing unit 43 executes and Figure 9 The first processing unit 43 in the middle performs a similar process, while Figure 10 The second processing unit 44a in the middle executes and Figure 9 The second processing unit 44a performs different processing operations.

[0154] Figure 11 The second processing unit 44a, in addition to the phase difference and sensitivity difference characteristic removal unit 57, also includes a re-mosaic processing unit 59a. Figure 11 The premise of the re-mosaic processing unit 59a provided in the second processing unit 44a is that an on-chip lens 11d is provided for each pixel group, and optimized re-mosaic processing is performed on the pixel array constituting the pixel group.

[0155] Since the solid-state imaging element 11a according to the comparative example performs optimized re-mosaic processing with on-chip lens 11d set for each pixel group, it is not necessary to perform re-mosaic processing in the subsequent signal processing device 42.

[0156] However, the internal configuration of the solid-state imaging element 11a according to the comparative example becomes more complex than the internal configuration of the solid-state imaging element 11 according to this embodiment, with increased chip size and power consumption.

[0157] As described above, the solid-state imaging element 11 according to this embodiment performs phase difference sensitivity difference characteristic removal processing to remove artifacts caused by the on-chip lens 11d provided for each pixel group, but does not perform re-mosaic processing. Furthermore, the solid-state imaging element 11 outputs pixel signals for a predetermined pixel array corresponding to the case where the on-chip lens 11d is provided for each pixel 32. Therefore, when the on-chip lens 11d is provided for each pixel 32, the signal processing device 42 located after the solid-state imaging element 11 can generate a high-quality captured image by performing general re-mosaic processing.

[0158] According to this embodiment, since the configuration of the solid-state imaging element 11 can be simplified, the solid-state imaging element 11 can be miniaturized, have low power consumption and low cost, and for example, a solid-state imaging element 11 suitable for mobile applications can be provided.

[0159] <Application Examples of Mobile Bodies> The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology disclosed herein can be implemented as a device to be installed on any kind of mobile body (such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, or robots).

[0160] Figure 12 This is a block diagram illustrating a schematic configuration example of a vehicle control system, which is an example of a mobile body control system to which the technology according to this disclosure can be applied.

[0161] The vehicle control system 12000 includes multiple electronic control units interconnected via a communication network 12001. Figure 12 In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. Furthermore, a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown as functional configurations of the integrated control unit 12050.

[0162] The drive system control unit 12010 controls the operation of devices related to the vehicle's drive system according to various programs. For example, the drive system control unit 12010 is used as a control device to control: drive force generating devices (such as internal combustion engines, drive motors, etc.) for generating the vehicle's driving force, drive force transmission mechanisms for transmitting the driving force to the wheels, steering mechanisms for adjusting the vehicle's steering angle, braking devices for generating the vehicle's braking force, etc.

[0163] The body system control unit 12020 controls the operation of various types of devices installed on the vehicle body according to various programs. For example, the body system control unit 12020 acts as a control device to control: keyless entry systems, smart key systems, power windows, or various lights such as headlights, reversing lights, brake lights, turn signals, fog lights, etc. In this case, radio waves transmitted from the moving device that replaces the key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals and controls the vehicle's door locking devices, power windows, lights, etc.

[0164] The exterior information detection unit 12030 detects information about the exterior of the vehicle, including that of the vehicle control system 12000. For example, the exterior information detection unit 12030 is connected to an imaging unit 12031. The exterior information detection unit 12030 causes the imaging unit 12031 to capture images of the exterior of the vehicle and receives the captured images. Based on the received images, the exterior information detection unit 12030 can perform processing for detecting objects (such as people, vehicles, obstacles, signs, characters on the road surface, etc.) or processing for determining the distance to the detected objects.

[0165] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output an electrical signal as an image or as information about the measured distance. Furthermore, the light received by the imaging unit 12031 can be visible light or invisible light such as infrared light.

[0166] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. The in-vehicle information detection unit 12040 is connected, for example, to a driver state detection unit 12041 that detects the driver's state. The driver state detection unit 12041 includes, for example, a camera that captures images of the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's level of fatigue or the driver's level of concentration, or it can determine whether the driver is dozing off.

[0167] The microcomputer 12051 can calculate control target values ​​for the drive force generation device, steering mechanism, or braking device based on information about the vehicle's interior or exterior obtained by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing functions of an advanced driver assistance system (ADAS), including collision avoidance or impact buffering for the vehicle, distance-based following, speed-maintaining driving, vehicle collision warning, lane departure warning, etc.

[0168] Furthermore, the microcomputer 12051 can control the drive force generation device, steering mechanism, braking device, etc., based on information about the exterior or interior of the vehicle obtained by the external information detection unit 12030 or the internal information detection unit 12040, thereby performing coordinated control intended for automatic driving and the like, which is independent of the driver's operation.

[0169] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12030 based on information about the vehicle's exterior obtained by the exterior information detection unit 12030. For example, the microcomputer 12051 can control the headlights to switch from high beam to low beam based on the position of the vehicle in front or oncoming vehicles detected by the exterior information detection unit 12030, thereby performing cooperative control aimed at preventing glare.

[0170] The sound / image output unit 12052 transmits at least one of sound and image output signals to an output device capable of visually or audibly notifying passengers of the vehicle or the outside of the vehicle. Figure 12 In this example, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are shown as output devices. The display unit 12062 may include, for example, at least one of a vehicle-mounted display and a head-up display.

[0171] Figure 13 This is a diagram showing an example of the mounting position of the imaging unit 12031.

[0172] exist Figure 13 In the imaging unit 12031, there are imaging units 12101, 12102, 12103, 12104 and 12105.

[0173] Imaging units 12101, 12102, 12103, 12104, and 12105 are, for example, arranged at positions such as the front nose, side mirrors, rear bumper, and rear door of vehicle 12100, as well as the upper part of the windshield inside the vehicle. Imaging unit 12101 at the front nose and imaging unit 12105 at the upper part of the windshield inside the vehicle primarily acquire images of the front of vehicle 12100. Imaging units 12102 and 12103 at the side mirrors primarily acquire images of the sides of vehicle 12100. Imaging unit 12104 at the rear bumper or rear door primarily acquires images of the rear of vehicle 12100. Imaging unit 12105 at the upper part of the windshield inside the vehicle is mainly used to detect vehicles, pedestrians, obstacles, signals, traffic signs, lanes, etc., ahead.

[0174] Notice, Figure 13 Examples of the imaging ranges of imaging units 12101 to 12104 are shown. Imaging range 12111 represents the imaging range of imaging unit 12101 installed at the front nose. Imaging ranges 12112 and 12113 represent the imaging ranges of imaging units 12102 and 12103 installed at the side mirrors, respectively. Imaging range 12114 represents the imaging range of imaging unit 12104 installed at the rear bumper or rear door. For example, a bird's-eye view of the vehicle 12100 viewed from above is obtained by overlaying image data captured by imaging units 12101 to 12104.

[0175] At least one of the imaging units 12101 to 12104 may have the function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0176] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging range 12111 to 12114 and the time change of that distance (relative speed to the vehicle 12100) based on distance information acquired from the imaging units 12101 to 12104, and thereby extract the nearest three-dimensional object as the vehicle ahead, which is specifically located on the driving path of the vehicle 12100 and is traveling in substantially the same direction as the vehicle 12100 at a predetermined speed (e.g., equal to or greater than 0 km / h). Furthermore, the microcomputer 12051 can preset the following distance to be maintained from the vehicle ahead and perform automatic braking control (including following stop control), automatic acceleration control (including following start control), etc. Therefore, it is possible to perform cooperative control intended for autonomous driving, etc., which allows the vehicle to drive automatically without relying on driver operation.

[0177] For example, the microcomputer 12051 can classify three-dimensional object data about three-dimensional objects into three-dimensional object data of two-wheeled vehicles, standard-sized vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information acquired from imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies whether obstacles around vehicle 12100 are obstacles that the driver of vehicle 12100 can visually recognize, or obstacles that are difficult for the driver of vehicle 12100 to visually recognize. Then, the microcomputer 12051 determines the collision risk, which indicates the risk of colliding with each obstacle. If the collision risk is equal to or higher than a set value and there is a possibility of collision, the microcomputer 12051 outputs an alarm to the driver via audio speaker 12061 or display unit 12062, and executes forced deceleration or evasive steering via drive system control unit 12010. Thus, the microcomputer 12051 can assist driving to avoid collisions.

[0178] At least one of the imaging units 12101 to 12104 can be an infrared camera that detects infrared light. For example, the microcomputer 12051 can identify a pedestrian by determining whether a pedestrian exists in the images captured by the imaging units 12101 to 12104. This pedestrian identification is performed, for example, by a procedure that extracts feature points from the images captured by the imaging units 12101 to 12104, which are infrared cameras, and a procedure that determines whether it is a pedestrian by performing pattern matching processing on a series of feature points representing the outline of an object. When the microcomputer 12051 determines that a pedestrian exists in the images captured by the imaging units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 so that a square outline for emphasis is displayed superimposed on the identified pedestrian. The sound / image output unit 12052 can also control the display unit 12062 so that an icon or the like representing a pedestrian is displayed at a desired location.

[0179] Examples of vehicle control systems to which the technology according to this disclosure can be applied have been described above. In the above configuration, the technology according to this disclosure can be applied, for example, to the imaging unit 12031. By applying the technology according to this disclosure to the imaging unit 12031, high-quality captured images can be acquired.

[0180] Note that this technology can have the following configurations.

[0181] (1) An electronic device, comprising: Solid-state imaging elements; and A signal processing device performs re-mosaic processing on the output signal of a solid-state imaging element, wherein... Solid-state imaging elements include: Multiple pixels arranged in a two-dimensional direction; Multiple on-chip lenses, each on-chip lens configured for a corresponding pixel group within a pixel group, the pixel group comprising two or more pixels; and The phase difference and sensitivity difference characteristic removal unit removes high-frequency components that are generated due to the phase difference and sensitivity difference between two or more pixel signals of the same color output for each pixel group.

[0182] (2) According to the electronic device in (1), wherein, The phase difference sensitivity difference feature removal unit removes artifacts caused by setting on-chip lenses for each pixel group.

[0183] (3) According to (2), the electronic device, Light passing through one of the on-chip lenses incident on a pixel group comprising two or more pixels of the same color; and The phase difference and sensitivity difference feature removal unit removes artifacts caused by phase difference and sensitivity difference between two or more pixel signals, which are output from two or more pixels of the same color in a pixel group that are incident on by light passing through the on-chip lens.

[0184] (4) An electronic device according to any one of (1) to (3), wherein, The solid-state imaging element further includes: A first substrate, the first substrate including a plurality of pixels and a plurality of on-chip lenses, each on-chip lens being configured for a corresponding pixel group in a pixel group; and The second substrate is stacked on the first substrate and includes a phase difference sensitivity difference characteristic removal section.

[0185] (5) According to (4) the electronic device, wherein, A pixel group comprises two or more pixels of the same color arranged in a predetermined pixel array; and The second substrate outputs a pixel signal of a predetermined pixel array after removing high-frequency components caused by phase difference and sensitivity difference. The phase difference and sensitivity difference of the pixel signal of the predetermined pixel array are the same as those in the case where an on-chip lens is provided for each pixel.

[0186] (6) The electronic device according to (5), wherein, The second substrate includes: A first processing unit generates an intermediate signal based on two or more pixel signals of each pixel group output from a first substrate; and The second processing unit generates a pixel signal for a predetermined pixel array based on an intermediate signal, after removing high-frequency components caused by phase difference and sensitivity difference.

[0187] (7) According to (6) the electronic device, wherein, The first processing unit includes at least one of the following: A color balance calculation unit calculates a gain for adjusting the color balance of each pixel included in a local region, based on two or more pixel signals for each pixel group. A direction-determining unit that determines the gradient of the pixel value at the location of the target pixel, and A flatness determination unit determines whether the pixel values ​​of a target pixel are flat based on the gradient of the pixel values ​​determined by the direction determination unit; and The first processing unit generates an intermediate signal, which includes the output signal of at least one of the color balance calculation unit, the direction determination unit, and the flatness determination unit.

[0188] (8) The electronic device according to (6) or (7), wherein, The second processing unit includes: A phase difference and sensitivity difference component extraction unit extracts the phase difference and sensitivity difference components of two or more pixel signals in each pixel group based on an intermediate signal generated by a first processing unit. A low-frequency component extraction unit extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on an intermediate signal generated by a first processing unit; and The phase difference and sensitivity difference characteristic removal unit generates a pixel signal of a predetermined pixel array based on the phase difference and sensitivity difference components and low-frequency components, thereby removing the high-frequency components caused by the phase difference and sensitivity difference.

[0189] (9) The electronic device according to (8), wherein, The first processing unit or the second processing unit includes a defect detection unit for detecting defective pixels; and The phase difference and sensitivity difference feature removal unit generates a pixel signal of a predetermined pixel array that removes the high-frequency components caused by the phase difference and sensitivity difference based on the phase difference and sensitivity difference components, the low-frequency components, and the defective pixels detected by the defect detection unit.

[0190] (10) An electronic device according to any one of (1) to (9), wherein, After performing heavy mosaic processing, the signal processing device outputs pixel signals of a predefined reference array.

[0191] (11) According to (10) the electronic device, wherein, The reference array is a Bayer array; and The pixel array is an array other than the Bayer array.

[0192] (12) An electronic device according to any one of (4) to (9), wherein, The signal processing device is built into a third substrate, which is arranged separately from the stacked first and second substrates.

[0193] (13) An electronic device according to any one of (1) to (12), wherein, The signal processing device performs re-mosaic processing with an on-chip lens set for each pixel.

[0194] (14) A solid-state imaging element, comprising: A first substrate includes a plurality of pixels and a plurality of on-chip lenses, the plurality of pixels being arranged in a two-dimensional direction, each on-chip lens being configured for a corresponding pixel group, the pixel group comprising two or more pixels; and A second substrate is stacked on the first substrate and outputs a signal without performing a re-mosaic process, removing high-frequency components from the multiple pixel signals output from the first substrate, which are generated due to phase and sensitivity differences.

[0195] (15) The solid-state imaging element according to (14), wherein, Light passing through one of the on-chip lenses incident on a pixel group comprising two or more pixels of the same color; and The second substrate removes artifacts caused by phase and sensitivity differences between two or more pixel signals, which are output from two or more pixels of the same color in a pixel group incident on light passing through an on-chip lens.

[0196] (16) A solid-state imaging element according to (14) or (15), wherein, A pixel group comprises two or more pixels of the same color arranged in a predetermined pixel array; and After removing high-frequency components caused by phase difference and sensitivity difference, the second substrate outputs pixel signals of a predetermined pixel array, the phase difference of which is the same as the phase difference and sensitivity difference when an on-chip lens is provided for each pixel.

[0197] (17) According to the solid-state imaging element of (16), wherein, The second substrate includes: A first processing unit generates an intermediate signal based on two or more pixel signals of each pixel group output from a first substrate; and The second processing unit generates a pixel signal for a predetermined pixel array based on an intermediate signal, after removing high-frequency components caused by phase difference and sensitivity difference.

[0198] (18) According to the solid-state imaging element of (17), wherein, The first processing unit includes at least one of the following: A color balance calculation unit calculates a gain for adjusting the color balance of each pixel included in a local region, based on two or more pixel signals from each pixel group. A direction-determining unit that determines the gradient of the pixel value at the location of the target pixel, and A flatness determination unit determines whether the pixel values ​​of a target pixel are flat based on the gradient of the pixel values ​​determined by the direction determination unit. The first processing unit generates an intermediate signal, which includes an output signal of at least one of a color balance calculation unit, a direction determination unit, and a flatness determination unit; and The second processing unit includes: The phase difference and sensitivity difference component extraction unit extracts the phase difference and sensitivity difference components of two or more pixel signals in each pixel group based on the intermediate signal generated by the first processing unit. A low-frequency component extraction unit extracts low-frequency components in a frequency band lower than the phase difference and sensitivity difference components based on an intermediate signal generated by a first processing unit. The phase difference and sensitivity difference characteristic removal unit generates a pixel signal of a predetermined pixel array based on the phase difference and sensitivity difference components and low-frequency components, thereby removing the high-frequency components caused by the phase difference and sensitivity difference.

[0199] (19) A signal processing apparatus, comprising: The receiving unit receives pixel signals output from a solid-state imaging element and removes artifacts caused by setting on-chip lenses for each pixel group, which includes two or more pixels; A heavy mosaic processing unit performs heavy mosaic processing based on signals received by a receiving unit; and The output section outputs the signal that has undergone heavy mosaic processing.

[0200] (20) The signal processing apparatus according to (19), wherein, The receiving unit receives pixel signals from a predetermined pixel array, the phase difference and sensitivity difference of which are the same as those in the case where an on-chip lens is provided for each pixel; and The heavy mosaic processing unit performs heavy mosaic processing when an on-chip lens is set for each pixel.

[0201] This disclosure is not limited to the various embodiments described above, but includes various modifications that can be conceived by those skilled in the art, and the effects of this disclosure are not limited to the above content. In other words, various additions, modifications, and partial deletions can be made without departing from the concept and spirit of this disclosure obtained from the content defined in the claims and their equivalents.

[0202] List of reference numerals

[0203] 1 pixel array

[0204] 2 Vertical drive circuit

[0205] 3-column signal processing circuit

[0206] 4. Horizontal drive circuit

[0207] 5 Output Circuit

[0208] 6. Control Circuit

[0209] 9-line selection line

[0210] 10 Converters

[0211] 11, 11a Solid-state imaging element

[0212] 11c color filter

[0213] 11D on-chip lens

[0214] 11e solder ball

[0215] 12 Glass substrate

[0216] 13 Adhesives

[0217] 21 First substrate (pixel sensor substrate)

[0218] 22 Second substrate (logic substrate)

[0219] 23a Part 1

[0220] 23b Part Two

[0221] 24 Logic Circuits

[0222] 25 Third substrate

[0223] 29 trap layers

[0224] 31-pixel circuit

[0225] 32 pixels

[0226] 33 Photoelectric conversion element

[0227] 34 Transfer transistors

[0228] 35 Amplifying Transistors

[0229] 36 Select Transistor

[0230] 37 Reset transistor

[0231] 38 Discharge transistors

[0232] 41 Electronic devices

[0233] 41a Electronic equipment

[0234] 42 Signal processing device

[0235] 43 First Processing Department

[0236] 44 Second Processing Department

[0237] 44a Second Processing Unit

[0238] 50 LCB Computing Unit

[0239] 51 Photodiode

[0240] 52 Direction Determination Department

[0241] 53 Flatness determination part

[0242] 54 Defect Detection Department

[0243] 55 Phase Difference Sensitivity Differential Component Extraction Unit

[0244] 56 DC component extraction unit

[0245] 57. Phase difference sensitivity difference characteristic removal unit

[0246] 58 Receiving Department

[0247] 59 Heavy Mosaic Processing Department

[0248] 59a Heavy Mosaic Processing Department

[0249] 60 Output Section

[0250] 61 Direction Determination Department

[0251] 62 Flatness determination part

[0252] 63 Planar Calculation Department

[0253] 64. Component Extraction Section

[0254] 65. Correlation Interpolation Processing Department

[0255] 81 Silicon substrate

[0256] 82 Multilayer Wiring

[0257] 83 Wiring Layer

[0258] 83a Wiring Layer

[0259] 83b Intermediate Wiring Layer

[0260] 83c bottom wiring layer

[0261] 84 interlayer insulating film

[0262] 85 through-silicon cavity

[0263] 86 Insulating film

[0264] 87 Connecting conductors

[0265] 88 Through-Silicon Vias

[0266] 90 Rewiring

[0267] 91 Solder resist layer (solder resist flux)

[0268] 101 Silicon substrate

[0269] 102 Multilayer Wiring

[0270] 103 Wiring Layer

[0271] 103a Wiring Layer

[0272] 103a Top wiring layer

[0273] 103b Intermediate Wiring Layer

[0274] 103c bottom wiring layer

[0275] 104 interlayer insulating film

[0276] 105 chip via

[0277] 106 Connection wiring

[0278] 107 Insulating Film

[0279] 108 Planarization film

[0280] 109 Through-Silicon Vias

[0281] 216 Transfer gate electrode

[0282] 217 Gate insulating film

[0283] 219 pixel isolation section

[0284] 220 Fixed charge film

[0285] 221 Insulating film

[0286] 225 shading film

[0287] 229 well layers

[0288] 239. Groove.

Claims

1. An electronic apparatus comprising: a solid-state imaging element; and a signal processing device that performs re-mosaicking processing on an output signal of the solid-state imaging element, wherein the solid-state imaging element includes: a plurality of pixels arranged in a two-dimensional direction; a plurality of on-chip lenses each provided for a corresponding one of pixel groups including two or more of the pixels; and a phase difference and sensitivity difference characteristic removal section that removes a high-frequency component due to a phase difference and a sensitivity difference between two or more pixel signals of the same color output for each pixel group. 2.The electronic apparatus according to claim 1, wherein the phase difference and sensitivity difference characteristic removal section removes an artifact due to the on-chip lens provided for each of the pixel groups. 3.The electronic apparatus according to claim 2, wherein the pixel group to which light that has passed through one of the on-chip lenses is incident includes the two or more pixels of the same color; and the phase difference and sensitivity difference characteristic removal section removes an artifact due to a phase difference and a sensitivity difference between the two or more pixel signals output from the two or more pixels of the same color included in the pixel group to which light that has passed through the on-chip lens is incident. 4.The electronic apparatus according to claim 1, wherein the solid-state imaging element further includes: a first substrate including the plurality of pixels and the plurality of on-chip lenses each provided for a corresponding one of the pixel groups; and a second substrate stacked on the first substrate and including the phase difference and sensitivity difference characteristic removal section. 5.The electronic apparatus according to claim 4, wherein the pixel group includes the two or more pixels of the same color arranged in a predetermined pixel array; and the second substrate outputs a pixel signal of the predetermined pixel array after removing a high-frequency component due to a phase difference and a sensitivity difference, the phase difference and the sensitivity difference of the pixel signal of the predetermined pixel array being the same as those in a case where an on-chip lens is provided for each pixel. 6.The electronic apparatus according to claim 5, wherein the second substrate includes: a first processing section that generates an intermediate signal based on the two or more pixel signals of each of the pixel groups output from the first substrate; and a second processing section that generates a pixel signal of the predetermined pixel array from which a high-frequency component due to a phase difference and a sensitivity difference is removed, based on the intermediate signal. 7.The electronic apparatus according to claim 6, wherein the first processing section includes at least one of: a color balance calculation section that calculates a gain for adjusting a color balance of each pixel included in a local area, based on the two or more pixel signals of each of the pixel groups, a phase difference and sensitivity difference calculation section that calculates a phase difference and a sensitivity difference between the two or more pixel signals of each of the pixel groups, and a gain calculation section that calculates a gain for adjusting a phase difference and a sensitivity difference between the two or more pixel signals of each of the pixel groups, based on the two or more pixel signals of each of the pixel groups. a direction determination section that determines a gradient of a pixel value at a position of a target pixel, and a flatness determination section that determines whether or not the pixel value of the target pixel is flat based on the gradient of the pixel value determined by the direction determination section; and the first processing section generates the intermediate signal including an output signal of at least one of the color balance calculation section, the direction determination section, and the flatness determination section.

8. The electronic device according to claim 6, wherein the second processing section includes: a phase difference and sensitivity difference component extraction section that extracts a phase difference and a sensitivity difference component of the two or more pixel signals of each of the pixel groups based on the intermediate signal generated by the first processing section; a low frequency component extraction section that extracts a low frequency component in a frequency band lower than the phase difference and sensitivity difference component based on the intermediate signal generated by the first processing section; and a phase difference and sensitivity difference characteristic removal section that generates a pixel signal of the predetermined pixel array from which an influence of a phase difference and a sensitivity difference is removed based on the phase difference and sensitivity difference component and the low frequency component.

9. The electronic device according to claim 8, wherein the first processing section or the second processing section includes a defect detection section that detects a defective pixel; and the phase difference and sensitivity difference characteristic removal section generates a pixel signal of the predetermined pixel array from which a high frequency component due to a phase difference and a sensitivity difference is removed based on the phase difference and sensitivity difference component, the low frequency component, and the defective pixel detected by the defect detection section.

10. The electronic device according to claim 1, wherein the signal processing device outputs a pixel signal of a reference array defined in advance after performing the remosaicking processing.

11. The electronic device according to claim 10, wherein the reference array is a Bayer array; and the array of the pixel groups is an array other than a Bayer array.

12. The electronic device according to claim 4, wherein the signal processing device is built in a third substrate arranged separately from the first substrate and the second substrate that are stacked.

13. The electronic device according to claim 1, wherein the signal processing device performs the remosaicking processing performed in a case where an on-chip lens is provided for each pixel.

14. A solid-state imaging element, comprising: a first substrate including a plurality of pixels arranged in a two-dimensional direction and a plurality of on-chip lenses provided for each of a plurality of pixel groups including two or more of the pixels; and a second substrate stacked on the first substrate and outputting a signal from which a high frequency component included in a plurality of pixel signals output from the first substrate is removed without performing remosaicking processing, the high frequency component being generated due to a phase difference and a sensitivity difference.

15. The solid-state imaging device according to claim 14, wherein the pixel group to which light incident through one of the on-chip lenses includes the two or more pixels of the same color; and the second substrate removes artifacts due to a phase difference and a sensitivity difference between two or more pixel signals output from the two or more pixels of the same color included in the pixel group to which light incident through the on-chip lens.

16. The solid-state imaging device according to claim 14, wherein the pixel group includes the two or more pixels of the same color arranged in a predetermined pixel array; and the second substrate outputs a pixel signal of the predetermined pixel array after removing a high-frequency component due to a phase difference and a sensitivity difference, the phase difference and the sensitivity difference of the pixel signal of the predetermined pixel array being the same as those in a case where an on-chip lens is provided for each pixel.

17. The solid-state imaging device according to claim 16, wherein the second substrate includes: a first processing section that generates an intermediate signal based on the two or more pixel signals of each of the pixel groups output from the first substrate; and a second processing section that generates a pixel signal of the predetermined pixel array in which a high-frequency component due to a phase difference and a sensitivity difference is removed, based on the intermediate signal.

18. The solid-state imaging device according to claim 17, wherein the first processing section includes at least one of: a color balance calculation section that calculates a gain for adjusting a color balance of each pixel included in a local area, based on the two or more pixel signals of each of the pixel groups, a direction determination section that determines a gradient of a pixel value at a position of a target pixel, and a flatness determination section that determines whether or not the pixel value of the target pixel is flat, based on the gradient of the pixel value determined by the direction determination section, the first processing section generates the intermediate signal including an output signal of at least one of the color balance calculation section, the direction determination section, and the flatness determination section; and the second processing section includes: a phase difference and sensitivity difference component extraction section that extracts a phase difference and a sensitivity difference component of the two or more pixel signals of each of the pixel groups, based on the intermediate signal generated by the first processing section, a low-frequency component extraction section that extracts a low-frequency component in a frequency band lower than the phase difference and the sensitivity difference component, based on the intermediate signal generated by the first processing section, and a phase difference and sensitivity difference characteristic removal section that generates a pixel signal of the predetermined pixel array in which a high-frequency component due to a phase difference and a sensitivity difference is removed, based on the phase difference and the sensitivity difference component and the low-frequency component.

19. A signal processing device comprising: a reception section that receives a pixel signal that is output from a solid-state imaging device and from which an artifact due to provision of an on-chip lens for each pixel group including two or more pixels is removed; a remosaic processing section that performs remosaic processing based on a signal received by the reception section; and an output section that outputs a signal on which the remosaic processing is performed.

20. The signal processing apparatus according to claim 19, wherein the reception section receives the pixel signal of a predetermined pixel array whose phase difference and sensitivity difference are the same as those in a case where an on-chip lens is provided for each pixel; and the remosaic processing section performs the remosaic processing performed in the case where the on-chip lens is provided for each pixel.

Images