Image sensor, electronic device including the same, and method of generating image

Abstract

An image sensor, an electronic device including the same, and a method of generating an image. There is provided an image sensor including a sensor substrate including a plurality of unit photosensitive cells, each of the plurality of unit photosensitive cells including: a central photosensitive cell and a plurality of peripheral photosensitive cells surrounding the central photosensitive cell; and a nano optical lens array including a plurality of unit areas respectively corresponding to the plurality of unit photosensitive units, and at least one nano structure configured to converge incident light onto the plurality of unit photosensitive units and form a phase distribution, incident light transmitted through the nano-optical lens array in a phase distribution converges onto a central photosensitive unit and a plurality of peripheral photosensitive units, where the central photosensitive unit is configured to output a first image, and the plurality of peripheral photosensitive units are configured to output a second image.

Description

[0001] Cross-reference of related applications

[0002] This application is based on and claims priority to Korean Patent Application No. 10-2024-0167753 filed with the Korean Intellectual Property Office on November 21, 2024, and Korean Patent Application No. 10-2025-0018087 filed with the Korean Intellectual Property Office on February 12, 2025, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to image sensors, electronic devices including image sensors, and methods for generating high dynamic range (HDR) images. Background Technology

[0004] Image sensors typically sense the color of incident light using color filters. However, because color filters absorb all colors of light except the specific color they filter, their light utilization efficiency can be low. For example, when using an RGB color filter, only one-third of the incident light is transmitted, and the other two-thirds are absorbed; therefore, the light utilization efficiency of this filter can be only about 33%. Consequently, most optical losses occur at the color filters in color display devices or color image sensors. Summary of the Invention

[0005] An image sensor with improved light efficiency by including a nano-optical lens array is provided, an electronic device including the image sensor, and a method for generating high dynamic range (HDR) images by using the electronic device.

[0006] Other aspects will be set forth in part in the description which follows, and will be apparent in part from the description, or may be learned by practicing the embodiments presented in this disclosure.

[0007] According to an aspect of the present invention, an image sensor is provided, comprising: a sensor substrate including a plurality of unit light-sensing cells, each of the plurality of unit light-sensing cells including: a central light-sensing cell and a plurality of peripheral light-sensing cells surrounding the central light-sensing cell; and a nano-optical lens array including: a plurality of unit regions respectively corresponding to the plurality of unit light-sensing cells, and at least one nanostructure configured to: converge incident light onto the plurality of unit light-sensing cells and form a phase profile, wherein incident light transmitted through the nano-optical lens array in the phase profile is converged onto the central light-sensing cell and the plurality of peripheral light-sensing cells, wherein the central light-sensing cell is configured to output a first image, and the plurality of peripheral light-sensing cells are configured to output a second image.

[0008] Each of the plurality of unit photosensitive units may include a plurality of photosensitive units arranged in a 3x3 array, wherein a central photosensitive unit may be disposed in the central portion of the 3x3 array, and wherein a plurality of peripheral photosensitive units may be disposed in the peripheral portion of the 3x3 array.

[0009] The phase distribution of the incident light transmitted through the nano-optical lens array has a peak at the center of the corresponding region of the central photosensitive unit and at the center of the corresponding region of each of the multiple peripheral photosensitive units.

[0010] At least one nanostructure can also be configured to separate a first light of a first wavelength band, a second light of a second wavelength band different from the first wavelength band, and a third light of a third wavelength band different from the first wavelength band and the second wavelength band, and to converge the first light, the second light and the third light onto a plurality of unit photosensitive units respectively.

[0011] The plurality of unit photosensitive units may include a first unit photosensitive unit, a second unit photosensitive unit, a third unit photosensitive unit, and a fourth unit photosensitive unit. The nano-optical lens array may include a first unit region corresponding to the first unit photosensitive unit, a second unit region corresponding to the second unit photosensitive unit, a third unit region corresponding to the third unit photosensitive unit, and a fourth unit region corresponding to the fourth unit photosensitive unit. At least one nanostructure may include a first nanostructure, a second nanostructure, a third nanostructure, and a fourth nanostructure. The first nanostructure disposed in the first unit region is arranged to separate and converge light from a first wavelength band of incident light onto the first unit photosensitive unit. The second nanostructure disposed in the second unit region is arranged to separate and converge light from a second wavelength band of incident light onto the second unit photosensitive unit. The third nanostructure disposed in the third unit region is arranged to separate and converge light from a third wavelength band of incident light onto the third unit photosensitive unit. The fourth nanostructure disposed in the fourth unit region is arranged to separate and converge light from a fourth wavelength band of incident light onto the fourth unit photosensitive unit.

[0012] The first nanostructure disposed in the first unit region can be arranged symmetrically with respect to the first direction, the second nanostructure disposed in the second unit region can be arranged symmetrically with respect to the first direction and the second direction perpendicular to the first direction, and the fourth nanostructure disposed in the fourth unit region can be arranged symmetrically with respect to the second direction.

[0013] The phase distribution of incident light transmitted through a nano-optical lens array can take the form of a Bessel function.

[0014] Each of the multiple photosensitive units may include multiple photosensitive units, and the size of the unit region is larger than the size of the photosensitive units.

[0015] Each of the multiple photosensitive units may include multiple photosensitive units, and the size of the unit region is the same as the size of the photosensitive units.

[0016] According to an aspect of this disclosure, an electronic device is provided, comprising: a lens assembly that forms an optical image of an object; an image sensor configured to convert the optical image formed by the lens assembly into an electrical signal; and a processor configured to process the signal generated from the image sensor, wherein the image sensor may include: a sensor substrate including a plurality of unit photosensitive units, each of the plurality of unit photosensitive units including: a central photosensitive unit and a plurality of peripheral photosensitive units surrounding the central photosensitive unit; and a nano-optical lens array including: a plurality of unit regions respectively corresponding to the plurality of unit photosensitive units, and at least one nanostructure configured to: converge incident light onto the plurality of unit photosensitive units and form a phase distribution in which incident light transmitted through the nano-optical lens array is converged onto the central photosensitive unit and the plurality of peripheral photosensitive units, wherein the central photosensitive unit is configured to output a first image, and the plurality of peripheral photosensitive units are configured to output a second image, and wherein the processor is configured to obtain a high dynamic range (HDR) image based on the first image and the second image.

[0017] According to an aspect of this disclosure, a method for generating a high dynamic range (HDR) image is provided, the method comprising: obtaining a single-shot image from a central photosensitive unit among a plurality of unit photosensitive units and a plurality of peripheral photosensitive units surrounding the central photosensitive unit; binning the images obtained from the plurality of peripheral photosensitive units in the single-shot image; performing HDR-merging by using the images obtained by binning and the images obtained from the central photosensitive unit in the single-shot image; performing signal processing on the image obtained by HDR-merging; and outputting the HDR image. Attached Figure Description

[0018] The above and other aspects, features and advantages of certain embodiments of this disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, wherein:

[0019] Figure 1 This is a schematic block diagram of an image sensor according to an embodiment;

[0020] Figure 2 This is a plan view illustrating an example of the pixel arrangement of a pixel array of an image sensor according to an embodiment;

[0021] Figure 3AAccording to the embodiments along Figure 2 A cross-sectional view of the pixel array of the image sensor taken by line A-A';

[0022] Figure 3B According to the embodiments along Figure 2 A cross-sectional view of the pixel array of the image sensor taken by line B-B';

[0023] Figure 4 This is a plan view showing the arrangement of a plurality of photosensitive units of a sensor substrate disposed in a pixel array of an image sensor according to an embodiment;

[0024] Figure 5 This is a plan view illustrating an example arrangement of multiple nanostructures of a nano-optical lens array according to an embodiment;

[0025] Figure 6 This is a plan view illustrating an example of a green light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment;

[0026] Figure 7 This is a plan view illustrating an example of a red light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment;

[0027] Figure 8 This is a plan view illustrating an example of a blue light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment;

[0028] Figure 9A , Figure 9B and Figure 9C It is shown Figure 5 A diagram of the target phase distribution of the nano-optical lens array;

[0029] Figure 10 This is a plan view of the color filter layer of the pixel array of the image sensor according to an embodiment;

[0030] Figure 11A According to another embodiment, along Figure 2 The image is a cross-sectional view of the pixel array of the image sensor taken by line A-A', and Figure 11B According to another embodiment, along Figure 2 A cross-sectional view of the pixel array of the image sensor taken by line B-B';

[0031] Figure 12A , Figure 12B , Figure 12C , Figure 12D and Figure 12E It is shown Figure 11A and Figure 11BA graph showing the signal intensity and target phase distribution at a unit photosensitive unit of a pixel array nano-optical lens array;

[0032] Figure 13 and Figure 14 Each is a plan view illustrating an example of the pixel arrangement of a pixel array of an image sensor according to another embodiment;

[0033] Figure 15 It is a graph showing a comparison of color separation performance between an image sensor according to some embodiments and an image sensor according to a comparative example;

[0034] Figure 16 This is a block diagram illustrating the high dynamic range (HDR) driving process of an image sensor according to an embodiment;

[0035] Figure 17 This is a schematic block diagram of an electronic device including an image sensor according to an embodiment;

[0036] Figure 18 It is set in Figure 17 A schematic block diagram of a camera module in an electronic device;

[0037] Figure 19 It is a block diagram of an electronic device including a multi-camera module; and

[0038] Figure 20 It is set in Figure 19 A detailed block diagram of a single camera module in an electronic device. Detailed Implementation

[0039] Reference will now be made in detail to embodiments, examples of which are shown in the accompanying drawings, wherein the same reference numerals always denote the same elements. In this respect, the embodiments may take different forms and should not be construed as limited to the description set forth herein. Therefore, the embodiments are described below only by reference to the accompanying drawings to explain various aspects. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of…” modify the entire list of elements when preceding it and do not modify individual elements in the list.

[0040] Embodiments will now be described more fully with reference to the accompanying drawings. The embodiments described below are provided by way of example only, and various modifications can be made from the embodiments. In the drawings, the same reference numerals denote the same elements, and the dimensions of components in the drawings may be exaggerated for clarity and ease of explanation.

[0041] It should be understood that when a component is referred to as being "on" or "above" another component, the component may be directly on or above the other component in a non-contact manner.

[0042] While terms such as "first" and "second" can be used to describe various components, these terms are only used to distinguish one component from another. These terms are not intended to specify that the components are made of different materials or have different structures.

[0043] A singular expression may encompass a plural expression unless it has a distinctly different meaning in the context. When a part "includes" a component, it may further include another component, rather than excluding the other component, unless otherwise stated.

[0044] Furthermore, the terms "part," "module," etc., refer to a unit that processes at least one function or operation, and can be implemented by hardware, software, or a combination thereof.

[0045] In the context of describing the embodiments (especially in the context of the appended claims), the terms “a”, “an”, and “the”, as well as similar indicators, should be interpreted to cover both singular and plural forms.

[0046] Furthermore, unless otherwise stated herein or clearly contradicted by the context, the steps of all methods described herein may be performed in any suitable order. Unless otherwise stated, the use of any and all exemplary language (e.g., “such as”) provided herein is intended only to better illustrate the technical concepts and does not constitute a limitation on the scope of the embodiments.

[0047] Figure 1 This is a schematic block diagram of an image sensor according to an embodiment. (Reference) Figure 1 The image sensor 1000 may include a pixel array 1100, a timing controller (T / C) 1010, a line decoder 1020, and an output circuit 1030. The image sensor may be a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0048] Pixel array 1100 may include pixels arranged in multiple rows and columns in a two-dimensional manner. Row decoder 1020 may select a row from the rows of pixel array 1100 in response to a row address signal output from timing controller 1010. Output circuitry 1030 may output photosensing signals column by column from multiple pixels arranged along the selected row. For this purpose, output circuitry 1030 may include column decoders and analog-to-digital converters (ADCs). For example, output circuitry 1030 may include multiple ADCs arranged between the column decoder and pixel array 1100 for each column decoder, or ADCs arranged at the output terminals of the column decoders. Timing controller 1010, row decoder 1020, and output circuitry 1030 may be implemented in a single chip or on separate chips. A processor for processing the image signals output through output circuitry 1030 may be implemented on a single chip along with timing controller 1010, row decoder 1020, and output circuitry 1030.

[0049] The pixel array 1100 may include multiple pixels for sensing light of different wavelengths from each other. The arrangement of the pixels can be achieved in various ways.

[0050] The pixel array 1100 may include multiple pixels for sensing light of different wavelengths. The arrangement of the pixels can be implemented in various ways. For example, Figure 2 An example of the pixel arrangement of the pixel array 1100 of the image sensor 1000 according to an embodiment is shown. (Reference) Figure 2 The pixel array 1100 may include multiple unit pixel structures arranged in a two-dimensional (2D) manner. For example, each of the multiple unit pixel structures may have a structure in which a first green pixel G1, a red pixel R, a blue pixel B, and a second green pixel G2 are arranged in a 2x2 array.

[0051] The first green pixel G1, red pixel R, blue pixel B, and second green pixel G2 can each include multiple pixels arranged in a 3x3 array. The first green pixel G1, red pixel R, blue pixel B, and second green pixel G2 can each include a central pixel located at the center and multiple peripheral pixels surrounding the central pixel. For example, as... Figure 2As shown, a first green center pixel G1a can be set at the center of the first green pixel G1, and a plurality of first green peripheral pixels G1b can surround the first green center pixel G1a. For example, the first green center pixel G1a can be set at the second row and second column of the first green pixel G1, and the plurality of first green peripheral pixels G1b can be set at the first row and first column, the first row and second column, the first row and third column, the second row and first column, the second row and third column, the third row and first column, the third row and second column, and the third row and third column. For example, the first green pixel G1 may include a first green center pixel G1a set at the center and eight first green peripheral pixels G1b surrounding the first green center pixel G1a.

[0052] According to an embodiment, similarly, a red center pixel Ra can be located at the center of a red pixel R, and a plurality of red peripheral pixels Rb can surround the red center pixel Ra. A blue center pixel Ba can be located at the center of a blue pixel B, and a plurality of blue peripheral pixels Bb can surround the blue center pixel Ba. A second green center pixel G2a can be located at the center of a second green pixel G2, and a plurality of second green peripheral pixels G2b can surround the second green center pixel G2a.

[0053] The image sensor 1000, including the pixel array 1100 with this pixel arrangement, can be, for example, a high dynamic range (HDR) image sensor. In this case, in low-light environments, an image can be generated using the signal output from the center pixel, and in high-light environments, an image can be generated using the signals output from both the center pixel and the peripheral pixels. Therefore, by using the center pixel with relatively high sensitivity and the peripheral pixels with relatively low sensitivity, the contrast range of the image can be significantly improved.

[0054] Figure 3A According to the embodiments along Figure 2 The image is a cross-sectional view of the pixel array of the image sensor taken by line A-A', and Figure 3B According to the embodiments along Figure 2 The image is a cross-sectional view of the pixel array of the image sensor taken by line B-B'.

[0055] refer to Figure 3A and Figure 3BThe pixel array 1100 may include a sensor substrate 110 and a nano-optical lens array 130 disposed on the sensor substrate 110. According to an embodiment, the pixel array 1100 may include a spacer layer 120 disposed between the sensor substrate 110 and the nano-optical lens array 130, and a color filter layer 140 disposed between the sensor substrate 110 and the spacer layer 120. However, this disclosure is not limited thereto, and therefore, according to another embodiment, the color filter layer 140 may be omitted.

[0056] The sensor substrate 110 may include a plurality of photosensitive units configured to generate an image signal by converting incident light into an electrical signal. For example, the sensor substrate 110 may include a first photosensitive unit 111 and a fourth photosensitive unit 114 configured to sense light in a first wavelength band and convert it into an electrical signal, a second photosensitive unit 112 configured to sense light in a second wavelength band and convert it into an electrical signal, and a third photosensitive unit 113 configured to sense light in a third wavelength band and convert it into an electrical signal.

[0057] Spacer layer 120 may be disposed between sensor substrate 110 and nano-optical lens array 130 to maintain a constant distance between the sensor substrate 110 and nano-optical lens array 130. Spacer layer 120 may comprise a material that is transparent to visible light, such as polymethylmethacrylate (PMMA), siloxane-based spin-on glass (SOG), SiO2, Si3N4, Al2O3, etc., which are dielectric materials with lower refractive indices and lower absorption ranges in the visible light band than nanostructures NP.

[0058] The color filter layer 140 may include a plurality of color filters that transmit light in a specific wavelength band and absorb light in other wavelength bands. For example, the color filter layer 140 may include a green color filter GF configured to transmit light in a first wavelength band and absorb light in other wavelength bands, a red color filter RF configured to transmit light in a second wavelength band and absorb light in other wavelength bands, and a blue color filter BF configured to transmit light in a third wavelength band and absorb light in other wavelength bands.

[0059] A green color filter GF can be arranged on the first unit photosensitive unit 111 and the fourth unit photosensitive unit 114, a red color filter RF can be arranged on the second unit photosensitive unit 112, and a blue color filter BF can be arranged on the third unit photosensitive unit 113. According to the embodiment, since the incident light can be significantly color-separated by the nano-optical lens array 130, even in the example case where the color filter layer 140 is used, the absorption loss caused by the color filter layer 140 can be relatively small. Furthermore, since the nano-optical lens array 130 and the color filter layer 140 are used together, color purity can be improved.

[0060] The nano-optical lens array 130 may include multiple nanostructures NP and may also include a dielectric layer DL filling the spaces between the multiple nanostructures NP. The nano-optical lens array 130 may include multiple nanostructures NP in various ways, enabling it to perform the aforementioned color separation and light convergence functions. For example, the multiple nanostructures NP can be arranged to change the phase of light transmitted through the nano-optical lens array 130 according to their position. The phase distribution of the transmitted light achieved by the nano-optical lens array 130 can be determined based on the size (e.g., width or diameter), shape, cross-sectional height of each nanostructure NP, the arrangement period (or pitch) of the multiple nanostructures NP, and the arrangement pattern of the multiple nanostructures NP. Furthermore, the movement of light transmitted through the nano-optical lens array 130 can be determined based on the phase distribution of the transmitted light.

[0061] Nanostructured NPs can have dimensions smaller than the wavelength of visible light. For example, nanostructured NPs can have dimensions smaller than the blue wavelength. For instance, the cross-sectional width (or diameter) of a nanostructured NP can be 400 nm, 300 nm, or less than 200 nm and greater than 80 nm. The height of the nanostructured NP can be from about 500 nm to about 1,500 nm, and can be greater than the width of its cross-section.

[0062] Nanostructured NPs can include materials with a relatively high refractive index and relatively low absorptivity in the visible light band compared to their surrounding materials. For example, nanostructured NPs can include, but are not limited to, c-Si, p-Si, a-Si, III-V compound semiconductors (GaP, GaN, GaAs, etc.), SiC, TiO2, SiN3, ZnS, ZnSe, Si3N4, and / or combinations thereof. The periphery of the nanostructured NP can be filled with a dielectric layer DL, which has a relatively low refractive index and relatively low absorptivity in the visible light band compared to the nanostructured NP. For example, the dielectric layer DL can be filled with PMMA, SOG, SiO2, Si3N4, Al2O3, air, etc.

[0063] The nanostructure NP can have a refractive index of 2.0 or greater relative to light with a wavelength of approximately 630 nm, and the dielectric layer DL can have a refractive index of 1.0 or greater but less than 2.0 relative to light with a wavelength of approximately 630 nm. Furthermore, the difference between the refractive index of the nanostructure NP and the refractive index of the dielectric layer DL can be 0.5 or greater. The nanostructure NP, having a refractive index different from that of the surrounding material, can alter the phase of light passing through it. This is due to the phase delay caused by the subwavelength shape dimension of the nanostructure NP, and the degree of phase delay can be determined by the detailed shape dimension, arrangement, etc., of the nanostructure NP.

[0064] The nanostructure NP may include nanostructures (NP1 and NP2) having a multilayer structure. The nanostructure NP may include at least one of a first nanostructure NP1 and a second nanostructure NP2. The first nanostructure NP1 may be disposed on the spacer layer 120, and the second nanostructure NP2 may be disposed on the first nanostructure NP1. The arrangement of the first nanostructure NP1 and the arrangement of the second nanostructure NP2 may be identical to each other. Alternatively, the arrangement of the first nanostructure NP1 and the arrangement of the second nanostructure NP2 may be different from each other.

[0065] According to an embodiment, depending on the arrangement of the nanostructures NP, the form of color separation and light convergence performed by the nano-optical lens array 130 can vary according to the color of the photosensitive unit of the sensor substrate 110 facing the nano-optical lens array 130.

[0066] According to an embodiment, an etch stop layer may be disposed between the spacer layer 120 and the nano-optical lens array 130. The etch stop layer may be configured to protect the spacer layer 120, which is the lower structure of the nano-optical lens array 130, during the fabrication of the nano-optical lens array 130. In an example where the nano-optical lens array 130 is fabricated on the spacer layer 120, the dielectric layer DL may be entirely formed on the spacer layer 120, and an etching process may be performed thereon to a certain depth. In this process, etching may be performed beyond the desired depth, thereby damaging the spacer layer 120, and color separation performance may degrade when the thickness of the spacer layer 120 does not meet the distance requirement between the nano-optical lens array 130 and the sensor substrate 110. However, in an example where the etch stop layer comprises a material having a lower etch selectivity than the material being etched, the etch stop layer may not be easily removed and may remain, preventing the spacer layer 120 from being damaged by the etching process. The etch stop layer may comprise HfO2. The thickness of the etch stop layer can be determined by taking into account the etch depth (i.e., the height of the nanostructure NP) and the etch dispersion in the process wafer. The thickness of the etch stop layer can be from about 3 nm to about 30 nm. Furthermore, in the example where the nanostructure NP has a multilayer structure, the etch stop layer can be arranged between the nanostructure layers.

[0067] According to an embodiment, a protective layer for protecting the nano-optical lens array 130 may be further disposed on the nano-optical lens array 130. For example, the protective layer may include a material used as an anti-reflection layer. For example, the anti-reflection layer may reduce the light reflected from the upper surface of the nano-optical lens array 130 in the incident light to improve the light utilization efficiency of the pixel array 1100. In other words, the anti-reflection layer may prevent incident light from the outside onto the pixel array 1100 from being reflected from the upper surface of the nano-optical lens array 130 and help the incident light pass through the nano-optical lens array 130 and be sensed at the sensor substrate 110. The anti-reflection layer may be a structure in which one or more layers are stacked, and may include, for example, a layer comprising a material different from the material included in the nano-optical lens array 130 or multiple layers having different refractive indices from each other.

[0068] Figure 4 This is a plan view illustrating the arrangement of a plurality of photosensitive units of a sensor substrate disposed in a pixel array of an image sensor according to an embodiment.

[0069] refer to Figure 4 The sensor substrate 110 may include a plurality of photosensitive units for sensing incident light.

[0070] The sensor substrate 110 may include multiple unit structures arranged in a 2D manner in a first direction (X direction) and a second direction (Y direction), and each unit structure may include a first unit photosensitive unit 111, a second unit photosensitive unit 112, a third unit photosensitive unit 113, and a fourth unit photosensitive unit 114 arranged in a 2x2 array. In addition, each unit photosensitive unit may include a central photosensitive unit (111a, 112a, 113a, and 114a) and multiple peripheral photosensitive units (111b, 112b, 113b, and 114b) arranged in a 3x3 array.

[0071] HDR images can be obtained by using images obtained from the central photosensitive unit (111a, 112a, 113a and 114a) and images obtained from multiple peripheral photosensitive units (111b, 112b, 113b and 114b).

[0072] refer to Figure 2 and Figure 4 The first photosensitive unit 111 can be configured to correspond to the first green pixel G1, the second photosensitive unit 112 can be configured to correspond to the red pixel R, the third photosensitive unit 113 can be configured to correspond to the blue pixel B, and the fourth photosensitive unit 114 can be configured to correspond to the second green pixel G2. The center photosensitive unit 111a of the first photosensitive unit 111 can be configured to correspond to the first green center pixel G1a, and the peripheral photosensitive unit 111b can be configured to correspond to the first green peripheral pixel G1b. The center photosensitive unit 112a of the second photosensitive unit 112 can be configured to correspond to the red center pixel Ra, and the peripheral photosensitive unit 112b can be configured to correspond to the red peripheral pixel Rb. The center photosensitive unit 113a of the third photosensitive unit 113 can be configured to correspond to the blue center pixel Ba, and the peripheral photosensitive unit 113b can be configured to correspond to the blue peripheral pixel Bb. The central photosensitive unit 114a of the fourth unit photosensitive unit 114 can be configured to correspond to the second green central pixel G2a, and the peripheral photosensitive unit 114b can be configured to correspond to the second green peripheral pixel G2b.

[0073] Figure 5 This is a plan view illustrating an example arrangement of multiple nanostructures of a nano-optical lens array according to an embodiment. Figure 5 This illustrates how, according to an embodiment, the area of ​​a nano-optical lens array disposed in the pixel array of an image sensor is divided.

[0074] refer to Figure 5The nano-optical lens array 130 can perform color separation and convergence of incident light. For example, the nano-optical lens array 130 can separate incident light into light of a first wavelength band (e.g., green light), light of a second wavelength band different from the first wavelength band (e.g., red light), and light of a third wavelength band different from the first and second wavelength bands (e.g., blue light), and cause the aforementioned light to travel along different paths. Furthermore, the nano-optical lens array 130 can also be used as a lens to converge the separated light of the first wavelength band, the second wavelength band, and the third wavelength band onto their respective photosensitive units.

[0075] refer to Figure 4 and Figure 5 The nano-optical lens array 130 may include a plurality of unit corresponding regions corresponding to each of the plurality of unit photosensitive units of the sensor substrate 110. According to embodiments, the plurality of unit corresponding regions may be referred to as "a plurality of unit regions," but this disclosure is not limited thereto. For example, the nano-optical lens array 130 may include a plurality of first unit corresponding regions 131 corresponding to a first unit photosensitive unit 111, a plurality of second unit corresponding regions 132 corresponding to a second unit photosensitive unit 112, a plurality of third unit corresponding regions 133 corresponding to a third unit photosensitive unit 113, and a plurality of fourth unit corresponding regions 134 corresponding to a fourth unit photosensitive unit 114. The size (or surface area) of each unit corresponding region (131, 132, 133, and 134) of the nano-optical lens array 130 may be larger than the size (or surface area) of each photosensitive unit.

[0076] According to an embodiment, a first unit corresponding region 131, a second unit corresponding region 132, a third unit corresponding region 133, and a fourth unit corresponding region 134 can be grouped to form a unit structure. The first unit corresponding region 131, the second unit corresponding region 132, the third unit corresponding region 133, and the fourth unit corresponding region 134 can be configured to face the corresponding first unit photosensitive unit 111, second unit photosensitive unit 112, third unit photosensitive unit 113, and fourth unit photosensitive unit 114 respectively in a third direction (Z direction) perpendicular to the first and second directions.

[0077] According to an embodiment, the first unit corresponding region 131, the second unit corresponding region 132, the third unit corresponding region 133, and the fourth unit corresponding region 134 forming the nano-optical lens array 130 can separate light of a first wavelength band from the incident light and focus the light of the first wavelength band onto the central photosensitive unit 111a and the peripheral photosensitive unit 111b of the first unit photosensitive unit 111 and the central photosensitive unit 114a and the peripheral photosensitive unit 114b of the fourth unit photosensitive unit 114; separate light of a second wavelength band from the incident light and focus the light of the second wavelength band onto the central photosensitive unit 112a and the peripheral photosensitive unit 112b of the second unit photosensitive unit 112; and separate light of a third wavelength band from the incident light and focus the light of the third wavelength band onto the central photosensitive unit 113a and the peripheral photosensitive unit 113b of the third unit photosensitive unit 113.

[0078] According to an embodiment, the nano-optical lens array 130 may include a plurality of nanostructures arranged according to a specific pattern. The plurality of nanostructures may be distributed in a first unit corresponding region 131, a second unit corresponding region 132, a third unit corresponding region 133, and a fourth unit corresponding region 134 forming the nano-optical lens array 130. Each of the first unit corresponding region 131, the second unit corresponding region 132, the third unit corresponding region 133, and the fourth unit corresponding region 134 may include at least one nanostructure NP.

[0079] The number of nanostructure NPs arranged in the first unit corresponding region 131 and the fourth unit corresponding region 134 can be greater than the number of nanostructure NPs arranged in the third unit corresponding region 133. The number of nanostructure NPs arranged in the second unit corresponding region 132 can be greater than the number of nanostructure NPs arranged in the first unit corresponding region 131 and the fourth unit corresponding region 134. However, this disclosure is not limited thereto.

[0080] The arrangement of at least one nanostructure NP in the first unit corresponding region 131 can have one-fold symmetry with respect to a first direction (X direction) as the axis of symmetry, and the arrangement of at least one nanostructure NP in the fourth unit corresponding region 134 can have one-fold symmetry with respect to a second direction (Y direction) as the axis of symmetry. Figure 5 As shown, the arrangement of at least one nanostructure NP in the first unit corresponding region 131 and the arrangement of at least one nanostructure NP in the fourth unit corresponding region 134 can be rotationally symmetrical about each other by 90 degrees. However, this disclosure is not limited thereto.

[0081] The arrangement of at least one nanostructure NP in the second unit corresponding region 132 can have double symmetry with respect to the first direction (X direction) and the second direction (Y direction) which are the axes of symmetry.

[0082] The nano-optical lens array 130 can form green light converging regions, red light converging regions, and blue light converging regions. Furthermore, incident light transmitted through the nano-optical lens array 130 can form a phase distribution that converges onto each of the peripheral photosensitive units, including the central photosensitive unit and the peripheral photosensitive units. Various examples of converging regions formed by the arrangement of the nanostructures NP of the nano-optical lens array 130 are described below.

[0083] Figure 6 This is a plan view showing the green light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment.

[0084] refer to Figure 4 and Figure 6 The nano-optical lens array 130 may include a first green light converging region GL1 and a second green light converging region GL2. The first green light converging region GL1 can converge light from a first wavelength band of incident light onto a first unit photosensitive unit 111 corresponding to the first unit corresponding region 131, and the second green light converging region GL2 can converge light from the first wavelength band of incident light onto a fourth unit photosensitive unit 114 corresponding to the fourth unit corresponding region 134. The size of the first green light converging region GL1 may be larger than the size of the first unit photosensitive unit 111, and the size of the second green light converging region GL2 may be larger than the size of the second unit photosensitive unit 114.

[0085] Figure 7 This is a plan view showing the red light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment.

[0086] refer to Figure 4 and Figure 7 The nano-optical lens array 130 may include a red light converging region RL. The red light converging region RL can converge the second wavelength band of the incident light onto the second unit photosensitive unit 112 corresponding to the second unit corresponding region 132. The size of the red light converging region RL may be larger than the size of the second unit photosensitive unit 112.

[0087] Figure 8 This is a plan view showing the blue light converging region formed by a nano-optical lens array of a pixel array of an image sensor according to an embodiment.

[0088] refer to Figure 4 and Figure 8The nano-optical lens array 130 may include a blue light converging region BL. The blue light converging region BL can converge the third wavelength band of the incident light onto the third unit photosensitive unit 113 corresponding to the third unit corresponding region 133. The size of the blue light converging region BL may be larger than the size of the third unit photosensitive unit 113.

[0089] Figure 9A , Figure 9B and Figure 9C It is shown Figure 5 A diagram showing the target phase distribution of the nano-optical lens array. The focal length was set to 4 μm, and the spacing between the photosensitive units was set to 0.64 μm. (Regarding...) Figure 5 The phase distribution of the nano-optical lens array is further described.

[0090] refer to Figure 5 and Figure 9A The nano-optical lens array 130 can form a phase distribution in which green light is color-separated and focused onto the central photosensitive unit 111a and peripheral photosensitive units 111b of the first unit photosensitive unit 111, and the central photosensitive unit 114a and peripheral photosensitive units 114b of the fourth unit photosensitive unit 114. In the first unit corresponding region 131, the green light transmitted through the nano-optical lens array 130 can have a phase delay peak (2π) at the center of the central corresponding region 131a corresponding to the central photosensitive unit 111a, and the phase delay value decreases away from the center. Similarly, it can have a phase delay peak (2π) at the center of each peripheral corresponding region 131b corresponding to the peripheral photosensitive unit 111b, and the phase delay value decreases away from the center.

[0091] Similarly, in the fourth unit corresponding region 134, the green light transmitted through the nano-optical lens array 130 can have a phase delay peak (2π) at the center of the central corresponding region 134a corresponding to the central photosensitive unit 114a and the phase delay value decreases away from the center, and can have a phase delay peak (2π) at the center of each peripheral corresponding region 134b corresponding to the peripheral photosensitive unit 114b and the phase delay value decreases away from the center.

[0092] refer to Figure 9BThe nano-optical lens array 130 can form a phase distribution in which red light is color-separated and focused onto the central photosensitive unit 112a and the peripheral photosensitive units 112b of the second unit photosensitive unit 112. In the corresponding region 132 of the second unit, the red light transmitted through the nano-optical lens array 130 can have a phase delay peak (2π) at the center of the corresponding region 132a corresponding to the central photosensitive unit 112a, and the phase delay value decreases away from the center. Similarly, it can have a phase delay peak (2π) at the center of each corresponding peripheral region 132b corresponding to the peripheral photosensitive unit 112b, and the phase delay value decreases away from the center.

[0093] refer to Figure 9C The nano-optical lens array 130 can form a phase distribution in which blue light is focused onto the central photosensitive unit 113a and the peripheral photosensitive units 113b of the third unit photosensitive unit 113. In the corresponding region 133 of the third unit, the blue light transmitted through the nano-optical lens array 130 can have a phase delay peak (2π) at the center of the corresponding region 133a of the central photosensitive unit 113a, and the phase delay value decreases away from the center. Similarly, it can have a phase delay peak (2π) at the center of each corresponding region 133b of the peripheral photosensitive unit 113b, and the phase delay value decreases away from the center.

[0094] Figure 10 This is a plan view of the color filter layer of the pixel array of the image sensor according to an embodiment.

[0095] refer to Figure 10 The color filter layer 140 may include a plurality of color filters that transmit light of a specific wavelength band and absorb light of other wavelength bands. For example, the color filter layer 140 may include a green color filter GF configured to transmit light of a first wavelength band and absorb light of other wavelength bands, a red color filter RF configured to transmit light of a second wavelength band and absorb light of other wavelength bands, and a blue color filter BF configured to transmit light of a third wavelength band and absorb light of other wavelength bands.

[0096] refer to Figure 4 A green color filter GF can be disposed on the first unit photosensitive unit 111 and the fourth unit photosensitive unit 114, a red color filter RF can be disposed on the second unit photosensitive unit 112, and a blue color filter BF can be disposed on the third unit photosensitive unit 113. Since the incident light is significantly color-separated by the nano-optical lens array 130, even in the example case where a color filter layer 140 is used, the absorption loss caused by the color filter layer 140 is likely to be small. Furthermore, since the nano-optical lens array 130 and the color filter layer 140 are used together, color purity can be improved. The color filter layer 140 can be omitted.

[0097] Figure 11A According to another embodiment, along Figure 2 The image is a cross-sectional view of the pixel array of the image sensor taken by line A-A', and Figure 11B According to another embodiment, along Figure 2 A cross-sectional view of the pixel array of the image sensor, taken by line B-B'. This will be viewed through a reference... Figure 3A and Figure 3B The implementation examples are described by focusing on the differences.

[0098] refer to Figure 11A and Figure 11B ,and Figure 3A and Figure 3B Compared to the previous embodiment, the nano-optical lens array 130 can be configured to perform only light converging without color separation. For this purpose, multiple nanostructures NP can be included in the nano-optical lens array 130 in various ways. For example, the multiple nanostructures NP can be arranged to change the phase of light transmitted through the nano-optical lens array 130 according to their position relative to the phase of the nano-optical lens array 130. The phase distribution of the transmitted light achieved by the nano-optical lens array 130 can be determined based on the size (e.g., width or diameter), shape, cross-sectional height of each of the nanostructures NP, the arrangement period (or spacing) of the multiple nanostructures NP, and the arrangement pattern of the multiple nanostructures NP. Furthermore, the movement of the light transmitted through the nano-optical lens array 130 can be determined based on the phase distribution of the transmitted light. The nano-optical lens array 130 can be configured such that incident light is converged onto each of the unit photosensitive units (111, 112, 113, and 114). In this case, the size (or surface area) of a unit corresponding region of the nano-optical lens array 130 can be the same as the size of each photosensitive unit. Incident light transmitted through the nano-optical lens array 130 can be focused onto the central photosensitive unit of each unit photosensitive unit (111, 112, 113 and 114) and multiple peripheral photosensitive units surrounding the central photosensitive unit.

[0099] Figure 12A , Figure 12B , Figure 12C and Figure 12D It is shown Figure 11A and Figure 11B The graph shows the signal intensity and target phase distribution at a unit photosensitive unit of the pixel array nano-optical lens array.

[0100] refer to Figure 12A , Figure 12B , Figure 12C and Figure 12D A nano-optical lens array 130 that performs light converging function can be configured to form a phase distribution in the form of a Bessel function.

[0101] Figure 12A This is a view showing the phase distribution in the form of a zero-order Bessel function formed by the nano-optical lens array 130. Figure 12B It shows the basis Figure 12A A view of the phase distribution of the signal intensity at a unit photosensitive cell, and Figure 12C This is shown from one direction (e.g., the first direction (X direction)) of the nano-optical lens array 130. Figure 12A A cross-sectional view of the phase distribution.

[0102] refer to Figure 12A , Figure 12B and Figure 12C In an example where the nano-optical lens array 130 forms a phase distribution in the form of a zero-order Bessel function, the incident light is focused not only onto the central photosensitive unit but also onto the peripheral photosensitive units in a unit photosensitive unit, and the signal intensity at the unit photosensitive unit can be in the form of a Bessel function.

[0103] Figure 12D This is a view showing the phase distribution in the form of a first-order Bessel function formed by the nano-optical lens array 130. Figure 12E It shows the basis Figure 12D A view showing the phase distribution of signal intensity per unit photosensitive element. (Reference) Figure 12D and Figure 12E In an example where the nano-optical lens array 130 forms a phase distribution in the form of a first-order Bessel function, the incident light converges to the central photosensitive unit in the unit photosensitive unit, and also converges uniformly to the peripheral photosensitive units in a form having a larger radius than the convergence form on the central photosensitive unit.

[0104] Figure 13 and Figure 14 Each is a plan view illustrating an example pixel arrangement of a pixel array of an image sensor according to another embodiment. In the description, for the sake of brevity, only those shown may be highlighted. Figure 2 The differences.

[0105] refer to Figure 13 The pixel array 1100a may include multiple unit pixel structures arranged in a 2D manner, and the multiple unit pixels may have a structure in which green pixel G, red pixel R, blue pixel B and infrared pixel IR are arranged in a 2x2 array.

[0106] The green pixel G, red pixel R, blue pixel B, and infrared pixel IR can each include multiple pixels arranged in a 3x3 array. Each of the green pixel G, red pixel R, blue pixel B, and infrared pixel IR can also include a central pixel located at the center and multiple peripheral pixels surrounding the central pixel.

[0107] For example, such as Figure 13 As shown, the green center pixel Ga can be located at the center of the green pixel G, and multiple green peripheral pixels Gb can surround the green center pixel Ga. Similarly, the red center pixel Ra can be located at the center of the red pixel R, and multiple red peripheral pixels Rb can surround the red center pixel Ra. The blue center pixel Ba can be located at the center of the blue pixel B, and multiple blue peripheral pixels Bb can surround the blue center pixel Ba. The infrared center pixel IRa can be located at the center of the infrared pixel IR, and multiple infrared peripheral pixels IRb can surround the infrared center pixel IRa.

[0108] refer to Figure 14 The pixel array 1100b may include multiple unit pixel structures arranged in a 2D manner, and the multiple unit pixels may have a structure in which the first yellow pixel Y1, the red pixel R, the blue pixel B, and the second yellow pixel Y2 are arranged in a 2x2 array.

[0109] The first yellow pixel Y1, the red pixel R, the blue pixel B, and the second yellow pixel Y2 can each include multiple pixels arranged in a 3x3 array. The first yellow pixel Y1, the red pixel R, the blue pixel B, and the second yellow pixel Y2 can each include a central pixel located at the center and multiple peripheral pixels surrounding the central pixel.

[0110] For example, such as Figure 14 As shown, a first yellow center pixel Y1a can be located at the center of the first yellow pixel Y1, and multiple first yellow peripheral pixels Y1b can surround the first yellow center pixel Y1a. Similarly, a red center pixel Ra can be located at the center of the red pixel R, and multiple red peripheral pixels Rb can surround the red center pixel Ra. A blue center pixel Ba can be located at the center of the blue pixel B, and multiple blue peripheral pixels Bb can surround the blue center pixel Ba. A second yellow center pixel Y2a can be located at the center of the second yellow pixel Y2, and multiple second yellow peripheral pixels Y2b can surround the second yellow center pixel Y2a.

[0111] about Figures 2 to 12E The provided description can also be applied to Figure 13 pixel array 1100a and Figure 14 The pixel array is 1100b.

[0112] Figure 15 This is a graph showing a comparison of color separation performance between an image sensor according to some embodiments and an image sensor according to a comparative example. Figure 15In the diagram, the solid line represents the quantum efficiency (QE) at the central photosensitive unit of the image sensor according to the embodiment (Example 1), the dotted-dash line represents the QE at the peripheral photosensitive unit of the image sensor according to the embodiment (Example 2), and the dashed line represents the QE at the photosensitive unit of the image sensor according to the comparative example (Comparative Example).

[0113] refer to Figure 15 When the image sensor 1000 according to the example, which includes a nano-optical lens array 130, is compared with the image sensor according to the comparative example, the QE increases by approximately 48% at the central photosensitive unit and by approximately 5% at the peripheral photosensitive units. According to the example, by applying the nano-optical lens array 130, which forms the above-described phase distribution, to the image sensor 1000, light can be focused onto the peripheral photosensitive units, while increasing the amount of light focused onto the central photosensitive unit.

[0114] In this way, by designing the phase distribution using the nano-optical lens array 130, the amount of light converged onto the central photosensitive unit and the peripheral photosensitive unit can be controlled, and the tuning freedom in the dynamic range of the central photosensitive unit and the peripheral photosensitive unit can be increased.

[0115] Figure 16 This is a block diagram illustrating the HDR driving process of an image sensor according to an embodiment.

[0116] refer to Figure 16 In operation S110, the HDR driving process may include acquiring a single-shot image. For example, the single-shot image may be generated based on signals from each of the peripheral photosensitive units surrounding the central photosensitive unit of the image sensor. For example, the single-shot image may be generated by acquiring signals from each of the peripheral photosensitive units surrounding the central photosensitive unit of the image sensor. In operation S120, the HDR driving process may include binning the images acquired from the peripheral photosensitive units in the single-shot image. In operation S130, after binning the images acquired from the peripheral photosensitive units, the HDR driving process may include performing HDR merging using the images acquired through binning and the images acquired from the central photosensitive unit in the single-shot image. For example, HDR merging weights may be applied to each image. In operation S140, the HDR driving process may include image signal processing on the HDR-merged image. For example, image signal processing may include, but is not limited to, performing demosaic processing, automatic white balance (AWB) correction, color correction matrix (CCM), or gamma correction. In operation S150, the HDR driving process may include outputting an HDR image after image signal processing is complete. The signal-processed image can be output as an HDR image.

[0117] The image sensor 1000 according to the embodiment can be used together with a modular lens having various functions to form a camera module and can be used in various electronic devices.

[0118] Figure 17 This is a block diagram illustrating an example of an electronic device ED01 including an image sensor 1000.

[0119] refer to Figure 17 In the network environment ED00, electronic device ED01 can communicate with another electronic device ED02 via a first network ED98 (such as a short-range wireless communication network), or with another electronic device ED04 and / or server ED08 via a second network ED99 (such as a long-range wireless communication network). Electronic device ED01 can communicate with electronic device ED04 via server ED08. Electronic device ED01 may include a processor ED20, a memory ED30, an input device ED50, an audio output device ED55, a display device ED60, an audio module ED70, a sensor module ED76, an interface ED77, a haptic module ED79, a camera module ED80, a power management module ED88, a battery ED89, a communication module ED90, a subscriber identification module ED96, and / or an antenna module ED97. In electronic device ED01, some of the constituent components (such as the display device ED60) may be omitted, or other constituent components may be added. Some of the aforementioned components may be implemented as a single integrated circuit. For example, the sensor module ED76 (fingerprint sensor, iris sensor, illuminance sensor, etc.) can be implemented by embedding it in the display device ED60 (monitor, etc.).

[0120] Processor ED20 can control one or more other components (hardware and software components, etc.) of electronic device ED01 connected to processor ED20 by executing software (program ED40, etc.) and perform various data processing or calculations. As part of the data processing or calculation, processor ED20 can load commands and / or data received from other components (sensor module ED76, communication module ED90, etc.) into volatile memory ED32, process the commands and / or data stored in volatile memory ED32, and store the result data in non-volatile memory ED34. Processor ED20 may include a main processor ED21 (central processing unit, application processor, etc.) and an auxiliary processor ED23 (graphics processing unit, image signal processor, sensor hub processor, communication processor, etc.) that can operate independently of or in conjunction with the main processor ED21. Auxiliary processor ED23 can use less power than the main processor ED21 and can perform dedicated functions.

[0121] In the example where the main processor ED21 is in an inactive state (sleep state), or in the example where the main processor ED21 is in an active state (application execution state), the auxiliary processor ED23 can control the functions and / or states related to some components of the electronic device ED01 (display device ED60, sensor module ED76, communication module ED90, etc.). The auxiliary processor ED23 (image signal processor, communication processor, etc.) can be implemented as part of other functionally related components (camera module ED80, communication module ED90, etc.).

[0122] The memory ED30 can store various data required by the constituent elements of the electronic device ED01 (processor ED20, sensor module ED76, etc.). The data may include, for example, software (program ED40, etc.) and input and / or output data regarding related commands. The memory ED30 may include volatile memory ED32 and / or non-volatile memory ED34.

[0123] The program ED40 can be stored as software in the memory ED30 and may include the operating system ED42, middleware ED44 and / or application ED46.

[0124] Input device ED50 can receive commands and / or data from outside the electronic device ED01 (such as from a user) for use by the constituent elements (such as processor ED20) of the electronic device ED01. Input device ED50 may include a microphone, mouse, keyboard, and / or digital pen (such as a stylus pen).

[0125] Audio output device ED55 can output audio signals to the external device ED01. Audio output device ED55 may include a speaker and / or a receiver. The speaker can be used for general purposes, such as multimedia playback, and the receiver can be used to receive incoming calls. The receiver can be integrated into the speaker or implemented as a separate device.

[0126] Display device ED60 can visually provide information to the outside of electronic device ED01. Display device ED60 may include a display, hologram device or projector, and control circuitry for controlling the corresponding device. Display device ED60 may include touch circuitry configured to detect touch and / or sensor circuitry configured to measure the intensity of the force generated by the touch (pressure sensor, etc.).

[0127] The audio module ED70 can convert sound into electrical signals or vice versa. The audio module ED70 can obtain sound through the input device ED50, or output sound through the speakers and / or headphones of another electronic device (such as electronic device ED02) connected to the audio output device ED55 and / or electronic device ED01 via wired or wireless means.

[0128] The sensor module ED76 can detect the operating status (power, temperature, etc.) or external environmental status (user status, etc.) of the electronic device ED01, and generate electrical signals and / or data values ​​corresponding to the detected status. The sensor module ED76 may include gesture sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, grip sensors, proximity sensors, color sensors, IR sensors, biometric sensors, temperature sensors, humidity sensors, and / or illuminance sensors.

[0129] Interface ED77 can support one or more specified protocols for electronic device ED01 to connect to another electronic device (electronic device ED02, etc.) via wired or wireless means. Interface ED77 may include a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, an SD card interface, and / or an audio interface.

[0130] The connection terminal ED78 may include a connector for physically connecting electronic device ED01 to another electronic device (electronic device ED02, etc.). The connection terminal ED78 may include an HDMI connector, a USB connector, an SD card connector, and / or an audio connector (headphone connector, etc.).

[0131] The ED79 haptic module can convert electrical signals into mechanical stimuli (vibration, movement, etc.) or electrical stimuli that can be perceived by the user through touch or motion. The ED79 haptic module may include a motor, piezoelectric devices, and / or electrical stimulation devices.

[0132] The ED80 camera module can capture still images and video. The ED80 camera module may include a lens assembly, which includes one or more lenses. Figure 1 The image sensor 1000, image signal processor, and / or flash are included. The lens assembly included in the camera module ED80 can collect light emitted from the subject for image capture.

[0133] The power management module ED88 manages the power supplied to the electronic device ED01. The power management module ED88 can be implemented as part of a power management integrated circuit (PMIC).

[0134] Battery ED89 can supply power to the constituent elements of electronic device ED01. Battery ED89 may include a non-rechargeable primary battery, a rechargeable secondary battery, and / or a fuel cell.

[0135] Communication module ED90 can establish wired and / or wireless communication channels between electronic device ED01 and another electronic device (electronic device ED02, electronic device ED04, server ED08, etc.), and support communication through the established communication channels. Communication module ED90 can operate independently of processor ED20 (application processor, etc.) and can include one or more communication processors supporting wired and / or wireless communication. Communication module ED90 may include wireless communication module ED92 (cellular communication module, short-range wireless communication module, Global Navigation Satellite System (GNSS) communication module, etc.) and / or wired communication module ED94 (local area network (LAN) communication module, power line communication module, etc.). In the above communication modules, the corresponding communication module can communicate with another electronic device through a first network ED98 (short-range communication network, such as Bluetooth, WiFi Direct, or Infrared Data Association (IrDA)) or a second network ED99 (long-range communication network, such as cellular network, Internet, or computer network (LAN, WAN, etc.)). Such various types of communication modules can be integrated into a single component (e.g., a single chip, etc.) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module ED92 can verify and authenticate electronic devices ED01 in communication networks such as the first network ED98 and / or the second network ED99 by using subscriber information (International Mobile Subscriber Identifier (IMSI), etc.) stored in the subscriber identification module ED96.

[0136] Antenna module ED97 can transmit signals and / or power to or from external sources (other electronic devices, etc.). The antenna may include a radiator composed of conductive patterns formed on a substrate (e.g., a printed circuit board (PCB), etc.). Antenna module ED97 may include one or more antennas. In an example where antenna module ED97 includes multiple antennas, communication module ED90 can select from the antennas an appropriate antenna for a communication method used in a communication network such as a first network ED98 and / or a second network ED99. Signals and / or power can be transmitted or received between communication module ED90 and another electronic device via the selected antenna. Other components besides antennas (such as radio frequency integrated circuits (RFICs)) may be included as part of antenna module ED97.

[0137] Some components can connect to each other and exchange signals (e.g., commands, data, etc.) through communication methods between peripheral devices (e.g., buses, general purpose input and output (GPIO), serial peripheral interfaces (SPI), mobile industrial processor interfaces (MIPI), etc.).

[0138] Commands or data can be sent or received between electronic device ED01 and external electronic device ED04 via server ED08 connected to the second network ED99. Electronic devices ED02 and ED04 can be of the same or different type as electronic device ED01. All or part of the operations performed in electronic device ED01 can be performed in one or more of the electronic devices (ED02, ED04, and ED08). In an example where electronic device ED01 needs to perform a function or service, instead of performing that function or service, electronic device ED01 can request one or more electronic devices to perform all or part of the function or service. The one or more electronic devices receiving the request can perform additional functions or services related to the request and send the results of the execution back to electronic device ED01. For this purpose, cloud computing, distributed computing, and / or client-server computing technologies can be used.

[0139] Figure 18 It is shown that it includes Figure 17 A block diagram of an example camera module ED80 in the electronic device ED01.

[0140] refer to Figure 18 The camera module ED80 may include a lens assembly 1110, a flash 1120, an image sensor 1000, an image stabilizer 1140, a memory 1150 (e.g., buffer memory, etc.), and / or an image signal processor 1160. The lens assembly 1110 can collect light emitted from the subject for image capture. The camera module ED80 may include multiple lens assemblies 1110, and in this case, the camera module ED80 may include a dual-camera setup, a 360-degree camera, or a spherical camera. Some of the lens assemblies 1110 may have the same lens properties (angle of view, focal length, autofocus, F-number, optical zoom, etc.) or different lens properties. The lens assembly 1110 may include a wide-angle lens or a telescope lens.

[0141] The flash unit 1120 can emit light to enhance light emitted or reflected from the subject. The flash unit 1120 can emit visible light or infrared light. The flash unit 1120 may include one or more light-emitting diodes (RGB LEDs, white LEDs, infrared LEDs, ultraviolet LEDs, etc.) and / or a xenon lamp. The image sensor 1000 may include... Figure 1The image sensor converts light emitted or reflected from the subject and transmitted through the lens assembly 1110 into electrical signals, thereby obtaining an image corresponding to the subject.

[0142] Image stabilizer 1140 can move one or more lenses, including in lens assembly 1110 or image sensor 1000, in a specific direction in response to movement of camera module ED80 or electronic device ED01 including camera module ED80, or can compensate for negative effects caused by movement by controlling (adjusting readout timing, etc.) the movement characteristics of image sensor 1000. Image stabilizer 1140 can detect movement of camera module ED80 or electronic device ED01 using a gyroscope sensor or accelerometer arranged inside or outside camera module ED80. Image stabilizer 1140 can be implemented in an optical form.

[0143] The memory 1150 can store some or all of the image data acquired by the image sensor 1000 for subsequent image processing operations. In an example where multiple images are acquired at high speed, only the low-resolution image is displayed, while the acquired raw data (Bayer patterning data, high-resolution data, etc.) is stored in the memory 1150. The memory 1150 can then be used to send the raw data of the selected (user-selected, etc.) image to the image signal processor 1160. The memory 1150 can be incorporated into the memory ED30 of the electronic device ED01, or configured as a separate memory for independent operation.

[0144] Image signal processor 1160 can acquire an image using electrical signals output from image sensor 1000. For example, image signal processor 1160 can directly perform a portion of image processing according to embodiments of the present disclosure in association with image sensor 1000. Additionally, image signal processor 1160 can request image data of a specific format from image sensor 1000 according to a desired image data format.

[0145] Furthermore, the image signal processor 1160 can perform additional image processing on images acquired by the image sensor 1000 or image data stored in the memory 1150. Image processing may include depth map generation, 3D modeling, panorama generation, feature point extraction, image compositing, and / or image compensation (noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, softening, etc.). The image signal processor 1160 can also perform control (exposure time control or readout timing control, etc.) on the constituent elements (image sensor 1000, etc.) included in the camera module ED80.

[0146] Images processed by image signal processor 1160 can be stored again in memory 1150 for further processing, or provided to external components of camera module ED80 (memory ED30, display device ED60, electronic device ED02, electronic device ED04, server ED08, etc.). Image signal processor 1160 can be incorporated into processor ED20, or configured as a separate processor operating independently of processor ED20. In an example where image signal processor 1160 is configured as a separate processor from processor ED20, images processed by image signal processor 1160 can undergo additional image processing by processor ED20 and then be displayed by display device ED60.

[0147] The image signal processor 1160 can independently receive two output signals from adjacent photosensitive units in each pixel or subpixel of the image sensor 1000, and generate an autofocus signal based on the difference between the two output signals. The image signal processor 1160 can control the lens assembly 1110 to precisely apply the focus of the lens assembly 1110 onto the surface of the image sensor 1000 based on the autofocus signal.

[0148] The electronic device ED01 may also include one or more camera modules having different characteristics or functions from each other. The camera modules may include similar... Figure 18 The camera module ED80 is a component therein, and the image sensor disposed therein can be implemented as a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor, either of which can include one or more image sensors selected from image sensors with different properties, such as RGB sensors, black-and-white (BW) sensors, infrared (IR) sensors, and ultraviolet (UV) sensors. In this case, one of the camera modules ED80 can be a wide-angle camera, and the other can be a telescope camera. Similarly, one of the camera modules ED80 can be a front-side camera, and the other can be a rear-side camera.

[0149] Figure 19 It is a block diagram of an electronic device that includes a multi-camera module, and Figure 20 It is set in Figure 19 A detailed block diagram of a camera module in an electronic device is shown.

[0150] refer to Figure 19 The electronic device 1200 may include a camera module group 1300, an application processor 1400, a power management integrated circuit (PMIC) 1500, an external memory 1600, and an image generator 1700.

[0151] Camera module group 1300 may include multiple camera modules 1300a, 1300b, and 1300c. Although the accompanying drawings depict an embodiment with three camera modules 1300a, 1300b, and 1300c arranged, the embodiments are not limited thereto. In some embodiments, camera module group 1300 may be modified to include only two camera modules. Furthermore, in some embodiments, camera module group 1300 may be modified to include n camera modules (n is a natural number of 4 or greater).

[0152] In the following text, see references Figure 19 The detailed configuration of camera module 1300b is further described, and according to embodiments, this description can also be applied to other camera modules 1300a and 1300c.

[0153] refer to Figure 19 The camera module 1300b may include a prism 1305, an optical path folding element (OPFE) 1310, an actuator 1330, an image sensing device 1340, and a storage device 1350.

[0154] The prism 1305 may include a reflector 1307 made of a light-reflecting material and change the direction of light L incident from the outside.

[0155] In some embodiments, the prism 1305 can change the direction of light ray L incident along the first direction (X direction) to a second direction (Y direction) perpendicular to the first direction (X direction). Further, the prism 1305 can rotate the reflector 1307, made of light-reflecting material, about the central axis 1306 in direction A or rotate the central axis 1306 in direction B to change the direction of light ray L incident along the first direction (X direction) to a second direction (Y direction) perpendicular to the first direction. At this time, the OPFE 1310 can also move in a third direction (Z direction) perpendicular to the first direction (X direction) and the second direction (Y direction).

[0156] In some embodiments, as shown in the accompanying drawings, the maximum rotation angle of the prism 1305 in the A direction may be less than or equal to 15 degrees in the +A direction and greater than 15 degrees in the -A direction; however, the embodiments are not limited thereto.

[0157] In some embodiments, the prism 1305 can move within 20 degrees in the +B or -B direction, or within 10 to 20 degrees, or within 15 to 20 degrees, and can move at the same or similar angles in the +B and -B directions, i.e., within a 1-degree angle difference.

[0158] In some embodiments, the prism 1305 can move the reflector 1307, which includes light-reflecting material, in a third direction (e.g., the Z direction) parallel to the extension direction of the central axis 1306.

[0159] OPFE 1310 may include an optical lens consisting of, for example, m lenses (m being a natural number). The optical zoom ratio of the camera module 1300b can be changed by moving the m lenses in a second direction (Y direction). In an example case where the initial optical zoom ratio of the camera module 1300b is Z and the m optical lenses included in OPFE 1310 are moved, the optical zoom ratio of the camera module 1300b can be changed to 3Z, 5Z, or greater than 10Z.

[0160] Actuator 1330 can move OPFE 1310 or optical lens to a specific position. For example, actuator 1330 can adjust the position of optical lens so that image sensor 1342 can be positioned at the focal length of optical lens for accurate sensing.

[0161] Image sensing device 1340 may include image sensor 1342, control logic 1344, and memory 1346. Image sensor 1342 can sense an image of an object using light L provided through an optical lens. Control logic 1344 can control the overall operation of camera module 1300b. For example, control logic 1344 can control the operation of camera module 1300b according to control signals provided through control signal line CSLb.

[0162] The memory 1346 may store data required for the operation of the camera module 1300b, such as calibration data 1347. Calibration data 1347 may include information required by the camera module 1300b to generate image data using light L provided externally. Calibration data 1347 may include, for example, information regarding the degree of rotation, information regarding the focal length, information regarding the optical axis, etc., as described above. In an example where the camera module 1300b is implemented as a multi-state camera in which the focal length varies depending on the position of the optical lens, calibration data 1347 may include information regarding the focal length depending on the position (or state) of the optical lens and information regarding autofocus.

[0163] Storage device 1350 can store image data sensed by image sensor 1342. Storage device 1350 can be disposed outside image sensing device 1340 and can be implemented in a structure in which storage device 1350 and sensor chip constituting image sensing device 1340 are stacked. In some embodiments, storage device 1350 can be implemented as electrically erasable programmable read-only memory (EEPROM), but embodiments are not limited thereto.

[0164] refer to Figure 19 and Figure 20 In some embodiments, each of the plurality of camera modules 1300a, 1300b, and 1300c may include an actuator 1330. Therefore, depending on the operation of the actuator 1330 included therein, each of the plurality of camera modules 1300a, 1300b, and 1300c may include the same or similar calibration data 1347.

[0165] In some embodiments, one of the plurality of camera modules 1300a, 1300b, and 1300c (e.g., 1300b) may be a folding lens camera module including the aforementioned prism 1305 and OPFE 1310, and the remaining camera modules (e.g., 1300a and 1300c) may be vertical camera modules excluding the prism 1305 and OPFE 1310. However, the embodiments are not limited thereto.

[0166] In some embodiments, one of the plurality of camera modules 1300a, 1300b and 1300c (e.g. 1300c) may be a vertical depth camera, for example, that extracts depth information by using infrared (IR).

[0167] In some embodiments, at least two camera modules (e.g., 1300a and 1300b) of the plurality of camera modules 1300a, 1300b and 1300c may have different fields of view. In this case, for example, at least two camera modules (e.g., 1300a and 1300b) of the plurality of camera modules 1300a, 1300b and 1300c may have different optical lenses, but this disclosure is not limited thereto.

[0168] Furthermore, in some embodiments, the fields of view of the multiple camera modules 1300a, 1300b, and 1300c may be different from each other. In this case, the optical lenses included in the multiple camera modules 1300a, 1300b, and 1300c may also be different from each other, but this disclosure is not limited thereto.

[0169] In some embodiments, the plurality of camera modules 1300a, 1300b, and 1300c may be physically separated from each other. That is, the sensing area of ​​an image sensor 1342 is not divided for use by the plurality of camera modules 1300a, 1300b, and 1300c, but an independent image sensor 1342 may be arranged in each of the plurality of camera modules 1300a, 1300b, and 1300c.

[0170] Return to reference Figure 19The application processor 1400 may include an image processing device 1410, a memory controller 1420, and internal memory 1430. The application processor 1400 may be implemented separately from multiple camera modules 1300a, 1300b, and 1300c. For example, the application processor 1400 and the multiple camera modules 1300a, 1300b, and 1300c may be implemented in separate semiconductor chips.

[0171] Image processing device 1410 may include multiple image signal processors (ISPs), namely ISP 1 1411, ISP 2 1412 and ISP 3 1413, and camera module controller 1414.

[0172] Image data generated from each of the camera modules 1300a, 1300b, and 1300c can be provided to the image processing device 1410 via separate image signal lines ISLa, ISLb, and ISLc. This image data transmission can be performed, for example, by a camera serial interface (CSI) based on the Mobile Industry Processor Interface (MPI), but the embodiments are not limited thereto.

[0173] Image data sent to image processing device 1410 may be stored in external memory 1600 before being sent to ISP 1 1411 and ISP 2 1412. The image data stored in external memory 1600 may be provided to ISP 1 1411 and / or ISP 2 1412. ISP 1 1411 may correct the received image data to generate video. ISP 2 1412 may correct the received image data to generate still images. For example, ISP 1 1411 and ISP 2 1412 may perform preprocessing operations on the image data, such as color calibration, gamma correction, etc.

[0174] ISP 1 1411 may include subprocessors. In an example where the number of subprocessors is the same as the number of camera modules 1300a, 1300b, and 1300c, each subprocessor can process image data provided from one camera module. In an example where the number of subprocessors is less than the number of camera modules 1300a, 1300b, and 1300c, at least one of the subprocessors can process image data provided from multiple camera modules using a timing-sharing process. Image data processed by ISP 1 1411 and / or ISP 2 1412 may be stored in external memory 1600 before being sent to ISP 3 1413. Image data stored in external memory 1600 may be sent to ISP 3 1413. ISP 3 1413 can perform post-processing operations on the image data, such as noise correction, sharpening correction, etc.

[0175] Image data processed by ISP 3 1413 can be provided to image generator 1700. Image generator 1700 can generate a final image based on image generation information or pattern signals using the image data provided from ISP 3 1413.

[0176] Specifically, the image generator 1700 can generate an output image by combining at least some of the image data generated from camera modules 1300a, 1300b, and 1300c, which have different fields of view, according to image generation information or a mode signal. Furthermore, the image generator 1700 can generate an output image by selecting any one of the image data generated from camera modules 1300a, 1300b, and 1300c, which have different fields of view from each other, according to image generation information or a mode signal.

[0177] In some embodiments, image generation information may include a zoom signal or zoom factor. Furthermore, in some embodiments, the mode signal may be, for example, a signal based on a user-selected mode.

[0178] In an example where the image generation information is a zoom signal (or zoom factor) and each of the camera modules 1300a, 1300b, and 1300c has a different field of view than the others, the image generator 1700 can perform different operations depending on the type of zoom signal. In an example where the zoom signal is a first signal, the image generator 1700 can generate an output image by merging image data output from camera module 1300a and image data output from camera module 1300c, and then using the merged image signal along with image data output from camera module 1300b that has not yet been used for data merging. In an example where the zoom signal is a second signal different from the first signal, the image generator 1700 may not perform this image data merging and can generate an output image by selecting any one piece of image data output from each of the camera modules 1300a, 1300b, and 1300c. However, the embodiments are not limited to this, and the method of processing image data can be modified in various ways if desired.

[0179] The camera module controller 1414 can provide control signals to each of the camera modules 1300a, 1300b, and 1300c. The control signals generated from the camera module controller 1414 can be provided to the corresponding camera modules 1300a, 1300b, and 1300c via separate control signal lines CSLa, CSLb, and CSLc.

[0180] In some embodiments, control signals provided from camera module controller 1414 to a plurality of camera modules 1300a, 1300b, and 1300c may include mode information based on mode signals. Based on the mode information, the plurality of camera modules 1300a, 1300b, and 1300c may operate in a first operating mode and a second operating mode relative to the sensing speed.

[0181] Multiple camera modules 1300a, 1300b, and 1300c can generate image signals at a first speed (e.g., generate image signals at a first frame rate) in a first operating mode, encode the image signals at a second speed higher than the first speed (e.g., encode them into image signals at a second frame rate higher than the first frame rate), and send the encoded image signals to the application processor 1400. In this case, the second speed can be equal to or less than 30 times the first speed.

[0182] Application processor 1400 can store the received image signal (i.e., the encoded image signal) in internal memory 1430 or external memory 1600 outside of application processor 1400, read the encoded image signal from internal memory 1430 or external memory 1600 for decoding, and display image data generated based on the decoded image signal. For example, ISP 1 1411 and ISP 2 1412 of image processing device 1410 can perform decoding and process the image according to the decoded image signal.

[0183] Multiple camera modules 1300a, 1300b, and 1300c can generate image signals at a third speed lower than the first speed (e.g., generate image signals at a third frame rate lower than the first frame rate) in a second operating mode and send the image signals to the application processor 1400. The image signals provided to the application processor 1400 may be signals that have not yet been encoded. The application processor 1400 can perform image processing on the received image signals or store the image signals in internal memory 1430 or external memory 1600.

[0184] The PMIC 1500 can supply power, such as a power supply voltage, to each of the multiple camera modules 1300a, 1300b, and 1300c. For example, under the control of the application processor 1400, the PMIC 1500 can supply a first power to camera module 1300a via power signal line PSLa, a second power to camera module 1300b via power signal line PSLb, and a third power to camera module 1300c via power signal line PSLc.

[0185] The PMIC 1500 can generate and adjust the power level corresponding to each of the plurality of camera modules 1300a, 1300b, and 1300c in response to a power control signal PCON from the application processor 1400. The power control signal PCON can include a power adjustment signal for each operating mode of the plurality of camera modules 1300a, 1300b, and 1300c. For example, the operating mode can include a low-power mode, and in this case, the power control signal PCON can include information about the camera module operating in the low-power mode and the set power level. The power levels provided to the plurality of camera modules 1300a, 1300b, and 1300c can be the same or different from each other. Furthermore, the power levels can be changed dynamically.

[0186] According to an embodiment of the present invention, an image sensor is provided, comprising: a sensor substrate including a plurality of unit photosensitive units, each of the plurality of unit photosensitive units including: a central photosensitive unit and a plurality of peripheral photosensitive units surrounding the central photosensitive unit; and a nano-optical lens array including: a plurality of unit regions respectively corresponding to the plurality of unit photosensitive units, and at least one nanostructure, the at least one nanostructure being configured to: converge incident light onto the plurality of unit photosensitive units and form a phase distribution, wherein incident light transmitted through the nano-optical lens array in the phase distribution is converged onto the central photosensitive unit and the plurality of peripheral photosensitive units, wherein the central photosensitive unit is configured to output a first image, and the plurality of peripheral photosensitive units are configured to output a second image.

[0187] Each of the plurality of unit photosensitive units may include a plurality of photosensitive units arranged in a 3x3 array, wherein a central photosensitive unit may be disposed in the central portion of the 3x3 array, and wherein a plurality of peripheral photosensitive units may be disposed in the peripheral portion of the 3x3 array.

[0188] The phase distribution of the incident light transmitted through the nano-optical lens array has a peak at the center of the corresponding region of the central photosensitive unit and at the center of the corresponding region of each of the multiple peripheral photosensitive units.

[0189] At least one nanostructure can also be configured to: separate a first light of a first wavelength band, a second light of a second wavelength band different from the first wavelength band, and a third light of a third wavelength band different from the first wavelength band and the second wavelength band from the incident light, and to converge the first light, the second light and the third light onto a plurality of unit photosensitive units respectively.

[0190] The plurality of unit photosensitive units may include a first unit photosensitive unit, a second unit photosensitive unit, a third unit photosensitive unit, and a fourth unit photosensitive unit. The nano-optical lens array may include a first unit region corresponding to the first unit photosensitive unit, a second unit region corresponding to the second unit photosensitive unit, a third unit region corresponding to the third unit photosensitive unit, and a fourth unit region corresponding to the fourth unit photosensitive unit. At least one nanostructure may include a first nanostructure, a second nanostructure, a third nanostructure, and a fourth nanostructure. The first nanostructure disposed in the first unit region is arranged to separate and focus light from a first wavelength band of incident light onto the first unit photosensitive unit. The second nanostructure disposed in the second unit region is arranged to separate and focus light from a second wavelength band of incident light onto the second unit photosensitive unit. The third nanostructure disposed in the third unit region is arranged to separate and focus light from a third wavelength band of incident light onto the third unit photosensitive unit. The fourth nanostructure disposed in the fourth unit region is arranged to separate and focus light from a fourth wavelength band of incident light onto the fourth unit photosensitive unit.

[0191] The first nanostructure disposed in the first unit region can be arranged symmetrically with respect to the first direction, the second nanostructure disposed in the second unit region can be arranged symmetrically with respect to the first direction and the second direction perpendicular to the first direction, and the fourth nanostructure disposed in the fourth unit region can be arranged symmetrically with respect to the second direction.

[0192] The phase distribution of incident light transmitted through a nano-optical lens array can take the form of a Bessel function.

[0193] Each of the multiple photosensitive units may include multiple photosensitive units, and the size of the unit region is larger than the size of the photosensitive units.

[0194] Each of the multiple photosensitive units may include multiple photosensitive units, and the size of the unit region is the same as the size of the photosensitive units.

[0195] According to embodiments of this disclosure, an electronic device is provided, comprising: a lens assembly that forms an optical image of an object; an image sensor configured to convert the optical image formed by the lens assembly into an electrical signal; and a processor configured to process the signal generated from the image sensor, wherein the image sensor may include: a sensor substrate including a plurality of unit photosensitive units, each of the plurality of unit photosensitive units including: a central photosensitive unit and a plurality of peripheral photosensitive units surrounding the central photosensitive unit; and a nano-optical lens array including: a plurality of unit regions respectively corresponding to the plurality of unit photosensitive units, and at least one nanostructure, the at least one nanostructure being configured to: converge incident light onto the plurality of unit photosensitive units and form a phase distribution in which incident light transmitted through the nano-optical lens array is converged onto the central photosensitive unit and the plurality of peripheral photosensitive units, wherein the central photosensitive unit is configured to output a first image, and the plurality of peripheral photosensitive units are configured to output a second image, and wherein the processor is configured to obtain a high dynamic range (HDR) image based on the first image and the second image.

[0196] According to embodiments of this disclosure, a method for generating a high dynamic range (HDR) image is provided, the method comprising: obtaining a single-shot image from a central photosensitive unit among a plurality of unit photosensitive units and a plurality of peripheral photosensitive units surrounding the central photosensitive unit; binning the images obtained from the plurality of peripheral photosensitive units in the single-shot image; performing HDR merging by using the images obtained by binning and the images obtained from the central photosensitive unit in the single-shot image; performing signal processing on the image obtained by HDR merging; and outputting the HDR image.

[0197] Although an image sensor, an electronic device including an image sensor, and a method for generating HDR images have been described with reference to embodiments shown in the accompanying drawings, such embodiments are provided by way of example only, and it should be understood that various modifications and equivalents can be made from the embodiments by those skilled in the art. Therefore, the embodiments should be considered descriptive rather than limiting. The scope of the claims is not limited by the detailed description of the embodiments but by the appended claims, and all differences within the scope will be construed as included within the scope of the claims.

[0198] According to an embodiment, an image sensor and an electronic device including the image sensor can improve the amount of light converged onto the central photosensitive unit and the peripheral photosensitive unit based on the phase distribution formed by the nano-optical lens array.

[0199] According to the embodiments, even in low-light or high-light environments, the image sensor can dynamically utilize pixels and can be applied to, for example, HDR sensors.

[0200] It should be understood that the embodiments described herein should be considered in a descriptive sense only and not for limiting purposes. The description of features or aspects within each embodiment should generally be considered applicable to other similar features or aspects in other embodiments. Although one or more embodiments have been described with reference to the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope defined by the appended claims.

Claims

1. An image sensor, comprising: A sensor substrate comprising a plurality of unit photosensitive units, each of the plurality of unit photosensitive units comprising: Central photosensitive unit, and Multiple peripheral photosensitive units surrounding the central photosensitive unit; and Nano-optical lens array, comprising: Multiple unit regions, respectively corresponding to the multiple unit photosensitive units, and At least one nanostructure is configured as follows: The incident light is focused onto the plurality of photosensitive units, and A phase distribution is formed in which the incident light transmitted through the nano-optical lens array is focused onto the central photosensitive unit and the plurality of peripheral photosensitive units. The central photosensitive unit is configured to output a first image and the plurality of peripheral photosensitive units are configured to output a second image.

2. The image sensor according to claim 1, in, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units arranged in a 3x3 array. The central photosensitive unit is located in the central portion of the 3x3 array, and The plurality of peripheral photosensitive units are disposed in the peripheral portion of the 3x3 array.

3. The image sensor according to claim 1, wherein, The phase distribution of the incident light transmitted through the nano-optical lens array has a peak at the center of the central corresponding region corresponding to the central photosensitive unit and at the center of each peripheral corresponding region corresponding to each of the plurality of peripheral photosensitive units.

4. The image sensor according to claim 1, wherein, The at least one nanostructure is further configured to: Separating from the incident light a first light of a first wavelength band, a second light of a second wavelength band different from the first wavelength band, and a third light of a third wavelength band different from both the first and second wavelength bands, and The first light, the second light, and the third light are respectively focused onto the plurality of unit photosensitive units.

5. The image sensor according to claim 1, in, The plurality of photosensitive units include a first photosensitive unit, a second photosensitive unit, a third photosensitive unit, and a fourth photosensitive unit. The nano-optical lens array includes a first unit region corresponding to the first unit photosensitive unit, a second unit region corresponding to the second unit photosensitive unit, a third unit region corresponding to the third unit photosensitive unit, and a fourth unit region corresponding to the fourth unit photosensitive unit. The at least one nanostructure includes a first nanostructure, a second nanostructure, a third nanostructure, and a fourth nanostructure. The first nanostructure disposed in the first unit region is arranged to separate and converge light from the first wavelength band of the incident light onto the first unit photosensitive unit. The second nanostructure disposed in the second unit region is arranged to separate and converge light from the second wavelength band of the incident light onto the second unit photosensitive unit. The third nanostructure disposed in the third unit region is arranged to separate and converge light from the third wavelength band of the incident light onto the third unit photosensitive unit, and The fourth nanostructure disposed in the fourth unit region is arranged to separate and converge light from the fourth wavelength band of the incident light onto the fourth unit photosensitive unit.

6. The image sensor according to claim 5, wherein, The first nanostructure disposed in the first unit region is arranged symmetrically with respect to the first direction. The second nanostructure disposed in the second unit region is arranged symmetrically with respect to the first direction and a second direction perpendicular to the first direction, and The fourth nanostructure disposed in the fourth unit region is arranged symmetrically with respect to the second direction.

7. The image sensor according to claim 1, wherein, The phase distribution of the incident light transmitted through the nano-optical lens array has the form of a Bessel function.

8. The image sensor according to claim 1, wherein, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units, and The size of the unit area is larger than the size of the photosensitive unit.

9. The image sensor according to claim 1, wherein, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units, and The size of the unit area is the same as the size of the photosensitive unit.

10. An electronic device, comprising: Lens assembly that forms an optical image of an object; An image sensor configured to convert the optical image formed by the lens assembly into an electrical signal; and A processor, configured to process signals generated from the image sensor, The image sensor includes: A sensor substrate comprising a plurality of unit photosensitive units, each of the plurality of unit photosensitive units comprising: Central photosensitive unit, and Multiple peripheral photosensitive units surrounding the central photosensitive unit; and Nano-optical lens array, comprising: Multiple unit regions, respectively corresponding to the multiple unit photosensitive units, and At least one nanostructure is configured as follows: The incident light is focused onto the plurality of photosensitive units, and A phase distribution is formed in which the incident light transmitted through the nano-optical lens array is focused onto the central photosensitive unit and the plurality of peripheral photosensitive units. Wherein, the central photosensitive unit is configured to output a first image and the plurality of peripheral photosensitive units are configured to output a second image, and The processor is configured to obtain a high dynamic range (HDR) image based on the first image and the second image.

11. The electronic device according to claim 10, in, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units arranged in a 3x3 array. The central photosensitive unit is located in the central portion of the 3x3 array, and The plurality of peripheral photosensitive units are disposed in the peripheral portion of the 3x3 array.

12. The electronic device according to claim 10, wherein, The phase distribution of the incident light transmitted through the nano-optical lens array has a peak at the center of the central corresponding region corresponding to the central photosensitive unit and at the center of each peripheral corresponding region corresponding to each of the plurality of peripheral photosensitive units.

13. The electronic device according to claim 10, wherein, The at least one nanostructure is further configured to: Separating from the incident light a first light of a first wavelength band, a second light of a second wavelength band different from the first wavelength band, and a third light of a third wavelength band different from both the first and second wavelength bands, and The first light, the second light, and the third light are respectively focused onto the plurality of unit photosensitive units.

14. The electronic device according to claim 10, wherein, in, The plurality of photosensitive units include a first photosensitive unit, a second photosensitive unit, a third photosensitive unit, and a fourth photosensitive unit. The nano-optical lens array includes a first unit region corresponding to the first unit photosensitive unit, a second unit region corresponding to the second unit photosensitive unit, a third unit region corresponding to the third unit photosensitive unit, and a fourth unit region corresponding to the fourth unit photosensitive unit. The at least one nanostructure includes a first nanostructure, a second nanostructure, a third nanostructure, and a fourth nanostructure. The first nanostructure disposed in the first unit region is arranged to separate and converge light from the first wavelength band of the incident light onto the first unit photosensitive unit. The second nanostructure disposed in the second unit region is arranged to separate and converge light from the second wavelength band of the incident light onto the second unit photosensitive unit. The third nanostructure disposed in the third unit region is arranged to separate and converge light from the third wavelength band of the incident light onto the third unit photosensitive unit, and The fourth nanostructure disposed in the fourth unit region is arranged to separate and converge light from the fourth wavelength band of the incident light onto the fourth unit photosensitive unit.

15. The electronic device according to claim 14, wherein, The first nanostructure disposed in the first unit region is arranged symmetrically with respect to the first direction. The second nanostructure disposed in the second unit region is arranged symmetrically with respect to the first direction and a second direction perpendicular to the first direction, and The fourth nanostructure disposed in the fourth unit region is arranged symmetrically with respect to the second direction.

16. The electronic device according to claim 10, wherein, The phase distribution of the incident light transmitted through the nano-optical lens array has the form of a Bessel function.

17. The electronic device according to claim 10, wherein, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units, and The size of the unit area is larger than the size of the photosensitive unit.

18. The electronic device according to claim 10, wherein, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units, and The size of the unit area is the same as the size of the photosensitive unit.

19. A method for generating a high dynamic range (HDR) image, the method comprising: A single-shot image is obtained from a central photosensitive unit among multiple photosensitive units and multiple peripheral photosensitive units surrounding the central photosensitive unit; The images obtained from the plurality of peripheral photosensitive units in the single-shot image are binned; HDR merging is performed by using the image obtained through the binning and the image obtained from the central photosensitive unit in the single-shot image; The image obtained through the HDR merging is subjected to signal processing; and Output HDR images.

20. The method according to claim 19, wherein, Each of the plurality of unit photosensitive units includes a plurality of photosensitive units arranged in a 3x3 array, the central photosensitive unit being disposed in the central portion of the 3x3 array, and the plurality of peripheral photosensitive units being disposed in the peripheral portion of the 3x3 array.

Images