Image sensor, optical apparatus, and method for designing image sensor

Abstract

The invention provides an image sensor with high energy utilization rate, optical equipment and a design method of the image sensor, the image sensor comprises a photoelectric sensing device and a metasurface, the photoelectric sensing device comprises at least three pixel units, each pixel unit comprises a plurality of pixels for sensing light of different wave bands, and the metasurface is arranged on the surface of the photoelectric sensing device. The metasurface comprises metasurface units in one-to-one correspondence with the pixel units, and each metasurface unit is configured to perform phase modulation on incident light, so that light of different wavebands is respectively routed to pixels matched with the wavebands in the pixel units; the metasurface is adopted to route the light of different wavebands to the pixels sensing the light of different wavebands, the light energy received by each pixel is improved, light energy loss caused by an optical filter is avoided, and the energy utilization rate is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of optoelectronic sensors, and in particular, to an image sensor, an optical device, and a design method of an image sensor. Background Art

[0002] An image sensor is a device that converts optical signals into electrical signals, and it includes pixels arranged in an array. Existing image sensors mainly converge incident light through a microlens array respectively and then distinguish the intensities of different color components (different wavelengths) of RGB through a Bayer filter. The color of the incident light can be reconstructed according to the received intensities of pixels corresponding to different colors. Taking the case where the light intensity contents of red, green, and blue are evenly distributed as an example, the maximum incident light energy received by each pixel is only 1 / 3. Coupled with the fact that the Bayer color filter itself cannot completely transmit monochromatic light and causes certain losses, the energy utilization rate of the image sensor is relatively low. Summary of the Invention

[0003] The purpose of the present invention is to provide an image sensor, an optical device, and a design method of an image sensor with high energy utilization rate.

[0004] To achieve one of the above-mentioned invention purposes, an embodiment of the present invention provides an image sensor, including:

[0005] An optoelectronic sensing device, the optoelectronic sensing device includes at least three pixel units, and each pixel unit includes a plurality of pixels that sense light in different wavelength bands;

[0006] A metasurface, the metasurface includes metasurface units corresponding to the pixel units one by one;

[0007] Wherein, each metasurface unit is configured to perform phase modulation on incident light so that light in different wavelength bands is respectively routed to the pixels in the pixel unit that match the wavelength band.

[0008] As a further improvement of an embodiment of the present invention, each pixel unit includes target pixels that sense the target wavelength band and reference pixels that sense the reference wavelength band. The target wavelength bands sensed by different pixel units are different, and the reference wavelength bands sensed by different pixel units are different or at least partially the same.

[0009] As a further improvement of an embodiment of the present invention, each metasurface unit routes the light in the target wavelength band and at least part of the light in the reference wavelength band to the pixels in the corresponding pixel unit that match the wavelength band.

[0010] As a further improvement of an embodiment of the present invention, the reference wavelength bands sensed by different pixel units are the same and are all different from the target wavelength band of each pixel unit.

[0011] As a further improvement of an embodiment of the present invention, all metasurface units are configured as metalenses or gratings;

[0012] Alternatively, some of the metasurface units are configured as metalenses and some of the metasurface units are configured as gratings.

[0013] As a further improvement of an embodiment of the present invention, the distance between the metasurface and the optoelectronic sensing device along the optical axis is not greater than the modulation focal length of each metasurface unit for the light of the target band of the corresponding pixel unit.

[0014] As a further improvement of an embodiment of the present invention, each pixel unit includes at least one target pixel and a plurality of reference pixels. All the target pixels together form a target area, and all the reference pixels together form a reference area. The reference area surrounds at least one side of the target area.

[0015] As a further improvement of an embodiment of the present invention, each metasurface unit includes a target phase set for the light of the target band of the corresponding pixel unit and a reference phase set for the light of the reference band of the corresponding pixel unit. One of the target phase and the reference phase is selected as the focusing phase, and the other of the target phase and the reference phase is selected as the vortex phase.

[0016] As a further improvement of an embodiment of the present invention, each pixel unit includes a target area and a reference area. Each metasurface unit includes a target part corresponding to the target area and a reference part corresponding to the reference area. The reference phase is arranged in the target part to route the light of the reference band to the reference area, and the target phase is arranged in the reference part to route the light of the target band to the target area.

[0017] As a further improvement of an embodiment of the present invention, the optoelectronic sensing device includes a plurality of color pixel groups arranged in an array. Each color pixel group includes at least four pixel units, and the light bands sensed by two of the at least four pixel units are the same.

[0018] As a further improvement of an embodiment of the present invention, a filter film is provided on the light incident side of each target pixel, and the filter film is configured to transmit light corresponding to the target band.

[0019] To achieve one of the above-mentioned objects of the present invention, the present invention further provides an optical device, which includes an optical system and the image sensor as described above, and the image sensor is located on the image plane of the optical system.

[0020] To achieve one of the above-mentioned objects of the present invention, the present invention further provides a design method for an image sensor. The design method for the image sensor is applicable to the image sensor as described above, and the design method includes:

[0021] Determine at least three pixel units;

[0022] Each pixel unit is provided with a plurality of pixels for sensing light of different bands;

[0023] Design a metasurface unit such that incident light of different bands is respectively routed to the pixels in the pixel unit that match the band.

[0024] As a further improvement of an embodiment of the present invention, each pixel unit is provided with target pixels for sensing a target band and reference pixels for sensing a reference band;

[0025] Design a metasurface unit such that light of the target band and at least part of the light of the reference band converge or deflect to the pixels in the corresponding pixel unit that match the band.

[0026] As a further improvement of an embodiment of the present invention, the metasurface unit is designed as a metalens and / or a grating.

[0027] As a further improvement of an embodiment of the present invention, select the focusing phase as the modulation phase for the light of the target band, and design the modulation focal length of the metasurface unit for the corresponding light of the target band to be not less than the distance between the metasurface and the optoelectronic sensing device along the optical axis.

[0028] As a further improvement of an embodiment of the present invention, select the vortex phase as the modulation phase for the light of the reference band, and set the topological charge number according to the corresponding reference pixels.

[0029] As a further improvement of an embodiment of the present invention, each metasurface unit includes a target part and a reference part. In the target part, reference phases for modulating the light of the corresponding reference band are arranged, and in the reference part, target phases for modulating the light of the corresponding target band are arranged.

[0030] As a further improvement of an embodiment of the present invention, perform Bloch boundary scanning on the superstructure unit to obtain the transmittance curves and phase curves of nanostructures with different characteristic sizes under light of different target bands, and construct a characteristic size-phase library with high transmittance corresponding to each target band.

[0031] As a further improvement of an embodiment of the present invention, determine at least four pixel units that form a color pixel group, and the bands of light sensed by two of the at least four pixel units are the same.

[0032] Compared with the prior art, in the embodiments of the present invention, a metasurface is used to respectively route light of different bands to pixels that sense light of different bands, which increases the light energy received by each pixel, also avoids the light energy loss caused by the filter, and improves the energy utilization rate. Description of the Drawings

[0033] Figure 1 It is a schematic exploded view of an image sensor in a preferred embodiment of the present invention;

[0034] Figure 2 is Figure 1 a top view of the optoelectronic sensing device in

[0035] Figure 3 It is a schematic diagram of a partial optical path of an image sensor in a preferred embodiment of the present invention;

[0036] Figure 4 It is a schematic diagram of a partial optical path of an image sensor in another preferred embodiment of the present invention;

[0037] Figure 5 It is a schematic diagram of the optical path at the first metasurface unit of an image sensor in yet another preferred embodiment of the present invention;

[0038] Figure 6 It is a schematic diagram of the optical path at the second metasurface unit of an image sensor in yet another preferred embodiment of the present invention;

[0039] Figure 7 It is a schematic diagram of the optical path at the third metasurface unit of an image sensor in yet another preferred embodiment of the present invention;

[0040] Figure 8 is Figure 2 a top view of the first pixel unit in

[0041] Figure 9 It is a top view of various embodiments of a color pixel group;

[0042] Figure 10 It is a top view of an embodiment of a color pixel group;

[0043] Figure 11 It is a top view of various embodiments of a color pixel group;

[0044] Figure 12 is Figure 2 a schematic diagram of a light spot at the reference pixel of the first pixel unit in

[0045] Figure 13 is Figure 2 a top view of various embodiments of the first metasurface unit corresponding to the first pixel unit in

[0046] Figure 14 It is a characteristic size-phase library with high transmittance corresponding to the first target band;

[0047] Figure 15 It is a characteristic size-phase library with high transmittance corresponding to the second target band;

[0048] Figure 16 is the characteristic size-phase library with high transmittance corresponding to the third target band;

[0049] Figure 17 is Figure 2 the top view of the first metasurface unit in and the local phase diagram;

[0050] Figure 18 is for Figure 2 the wavelength-transmittance relationship diagram obtained after simulating the first metasurface unit in ;

[0051] Figure 19 is for Figure 2 the wavelength-transmittance relationship diagram obtained after simulating the color supercell in . Specific Embodiments

[0052] The present invention will be described in detail below in conjunction with the specific embodiments shown in the drawings. However, these embodiments do not limit the present invention, and any structural, methodical, or functional transformations made by those of ordinary skill in the art based on these embodiments are included within the protection scope of the present invention.

[0053] It should be understood that the terms such as "having", "including", and "comprising" used herein do not exclude the presence or addition of one or more other elements or their combinations.

[0054] Furthermore, it should be understood that although the terms first, second, etc. may be used herein to describe various elements or structures, the objects described should not be limited by these terms. These terms are only used to distinguish these described objects from each other. For example, the first pixel unit may be referred to as the second pixel unit, and similarly, the second pixel unit may also be referred to as the first pixel unit, which does not deviate from the protection scope of this application.

[0055] In the various diagrams of the present invention, for the convenience of illustration, the sizes of some structures or parts are exaggerated relative to other structures or parts. Therefore, it is only used to illustrate the basic structure of the subject matter of the present invention.

[0056] Specifically, referring to Figure 1 shown, an image sensor includes a photoelectric sensing device 10 and a metasurface 20. In this embodiment, the metasurface 20 is located on the light incident side of the photoelectric sensing device 10, and the metasurface 20 is used to modulate the incident light so that the light is routed or transmitted to the photoelectric sensing device 10. The photoelectric sensing device 10 can convert the received optical signal into an electrical signal.

[0057] Among them, the metasurface 20 refers to an artificial layered material with a size smaller than or approximately equal to the wavelength, which can be regarded as the two-dimensional counterpart of metamaterials. The metasurface 20 can realize the regulation of characteristics such as polarization, phase, amplitude, frequency, and propagation mode of electromagnetic waves through sub-wavelength superstructural units on the surface, and achieve characteristics such as beam shaping, beam deflection, superlens, superholography, optical rotation, and antireflection and antireflection enhancement.

[0058] Moreover, the metasurface 20 is a sub-wavelength-sized optical element, suitable for the current micron-scale sensor architecture. At the same time, its manufacturing process is compatible with mature semiconductor sensor technologies, and it has strong practicability and economy.

[0059] Specifically, with reference to Figure 2 As shown, the optoelectronic sensing device includes at least three pixel units (11, 12, 13), and each pixel unit includes a plurality of pixels (such as 111, 112) that sense light in different bands. In this embodiment, by dividing the optoelectronic sensing device 10, at least three pixel units are obtained, and the bands of light sensed by the three pixel units (11, 12, 13) are at least partially different. Each of the three pixel units (11, 12, 13) includes at least two pixels (such as 111, 112), so that each can sense at least two bands of light, that is, sense the light of a specific band through at least two pixels. Each pixel receives the light signal of a specific band, responds to the light signal of the specific band, and each pixel is configured to convert the light signal of the specific band into an electrical signal.

[0060] Furthermore, the metasurface 20 includes metasurface units (21, 22, 23) that correspond one-to-one with the pixel units (11, 12, 13). In this embodiment, after dividing the metasurface 20, metasurface units (21, 22, 23) with the same number as the pixel units (11, 12, 13) are obtained, and the structures between different metasurface units (21, 22, 23) are different. Each pixel unit (11, 12, 13) and each metasurface unit (21, 22, 23) correspond one-to-one along the optical axis direction ( Figure 1 the Z-axis in

[0061] Furthermore, each metasurface unit (21, 22, 23) is configured to perform phase modulation on the incident light, so that light of different bands is respectively routed to the pixels in the pixel unit that match the band. In this embodiment, after incident light of different bands is incident on the metasurface 20, each metasurface unit (21, 22, 23) can change the phase of the light of the corresponding band, so that the light of the corresponding band is routed or transmitted to the corresponding pixels in the pixel units (11, 12, 13), and thus is sensed by the corresponding pixels and converted into an electrical signal.

[0062] Exemplarily, such as Figure 1, the optoelectronic sensing device 10 includes a first pixel unit 11, a second pixel unit 12, and a third pixel unit 13, and the metasurface 20 includes corresponding first metasurface unit 21, second metasurface unit 22, and third metasurface unit 23.

[0063] Preferably, each metasurface unit (21, 22, 23) is configured to perform phase modulation on the incident light so that lights of different bands are respectively routed to the pixels in the corresponding pixel units (11, 12, 13) that match the bands. For example, the first metasurface unit 21 is configured to perform phase modulation on the incident light so that lights of different bands are respectively routed to the pixels (such as 111, 112) in the first pixel unit 11 that match the bands.

[0064] Of course, when performing phase modulation on the incident light, the metasurface unit is not limited to routing the light to the corresponding pixel unit, and can also route the light to adjacent or other pixel units. For example, the first metasurface unit 21 can route the incident light to the corresponding first pixel unit 11, and the first metasurface unit 21 can also route the incident light to the second pixel unit 12, the third pixel unit 13, or other pixel units.

[0065] Using the metasurface 20 to respectively route lights of different bands to the pixels that sense lights of different bands can improve the light energy received by each pixel, and also avoid the light energy loss caused by the filter, thereby improving the energy utilization rate.

[0066] Furthermore, each pixel unit (11, 12, 13) includes target pixels (111, 121, 131) that sense the target band and reference pixels (112, 122, 132) that sense the reference band. The target bands sensed by different pixel units are different, and the reference bands sensed by different pixel units are either different or at least partially the same. In this embodiment, the target band (labeled with R, G, B in the figure) is the specific band light required for imaging, such as the red (R), green (G), and blue (B) primary colors required for color imaging. The reference band (labeled with W in the figure) can be other bands of the incident light except the target band, for example, including near-infrared light to near-infrared cut-off, and there can be multiple reference bands, thus providing more selectivity for the reference band. For example, when the target band of the first pixel unit 11 is red (R) light, the reference band of the first pixel unit 11 at this time can be one or more of green (G) light, blue (B) light, and near-infrared light (IR), as long as it is different from the target band.

[0067] By setting reference pixels (112, 122, 132) within each pixel unit (11, 12, 13) to sense the light of the reference band, the light of the reference band can be used as reference light to provide more optical signals for color reconstruction. When performing color reconstruction, using the electrical signals generated by the reference pixels (112, 122, 132) as a benchmark can reduce the difficulty of color reconstruction. Moreover, sensing light other than the target band can also improve the energy utilization rate of the image sensor.

[0068] Exemplarily, Figure 2 , the first pixel unit 11 includes a first target pixel 111 for sensing the light of the first target band and a first reference pixel 112 for sensing the light of the first reference band, the second pixel unit 12 includes a second target pixel 121 for sensing the light of the second target band and a second reference pixel 122 for sensing the light of the second reference band, and the third pixel unit 13 includes a third target pixel 131 for sensing the light of the third target band and a third reference pixel 132 for sensing the light of the third reference band. Preferably, the first target band (or at least part of the first target band) is greater than the second target band, and the second target band (or at least part of the second target band) is greater than the third target band.

[0069] Further exemplarily, the first target band is between 585 nm and 650 nm, and can be 617 nm, i.e., red (R) light; the second target band is between 495 nm and 595 nm, and can be 545 nm, i.e., green (G) light; the third target band is between 400 nm and 505 nm, and can be 452 nm, i.e., blue (B) light.

[0070] Furthermore, each metasurface unit (21, 22, 23) routes the light of the target band and at least part of the light of the reference band to the pixels in the corresponding pixel unit (11, 12, 13) that match the band. In this embodiment, preferably, each metasurface unit (21, 22, 23) routes all the light of the target band and all the light of the reference band to the pixels in the corresponding pixel unit (11, 12, 13) that match the band, that is, the metasurface unit only needs to route the incident light to the corresponding pixel unit, thereby simplifying the design difficulty of the metasurface unit. For example, the first metasurface unit 21 routes all the light of the first target band to the first target pixel 111 in the first pixel unit 11 that matches the band, and routes all the light of the first reference band to the first reference pixel 112 in the first pixel unit 11 that matches the band.

[0071] Of course, it is also possible that each metasurface unit (21, 22, 23) routes all the target-band light and a part of the reference-band light to the pixels in the corresponding pixel units (11, 12, 13) that match the band, and routes another part of the reference-band light to the pixels in adjacent or other pixel units that match the band. That is, in addition to routing the incident light to the corresponding pixel units, the metasurface unit also routes the incident light to the pixels in adjacent or other pixel units. For example, the first metasurface unit 21 routes all the first target-band light to the first target pixel 111 in the first pixel unit 11 that matches the band, also routes a part of the first reference-band light to the first reference pixel 112 in the first pixel unit 11 that matches the band, and routes another part of the first reference-band light to the pixels in adjacent or other pixel units that match the band (this pixel includes but is not limited to the first reference pixel 112, such as the second target pixel 121, the third target pixel 131, etc. are all possible). Thus, the target-band light can be fully sensed, and the energy utilization rate can be improved.

[0072] Furthermore, the reference bands sensed by different pixel units (11, 12, 13) are the same and are all different from the target bands of each pixel unit (11, 12, 13). In this embodiment, each pixel unit (11, 12, 13) uses the same reference pixels (112, 122, 132), that is, the first reference pixel 112, the second reference pixel 122, and the third reference pixel 132 are all the same, which also means that the first reference band, the second reference band, and the third reference band are all the same. Thus, the optoelectronic sensing device 10 only needs to sense four bands of light (i.e., three target-band lights and one reference-band light), simplifying the optoelectronic sensing device 10. Moreover, each metasurface unit (21, 22, 23) only needs to perform phase modulation on two bands of light, simplifying the design difficulty of the metasurface unit.

[0073] Exemplarily, the first reference band, the second reference band, and the third reference band are all between 650 nm and 1100 nm, and can be 900 nm, that is, near-infrared (IR) light.

[0074] Specifically, the image sensor further includes a dielectric layer 30 disposed on the light-sensing side of the optoelectronic sensing device 10, and the metasurface 20 is formed on the light-incident side surface of the dielectric layer 30 through semiconductor manufacturing processes.

[0075] Furthermore, with reference to Figure 3 as shown, all metasurface units (21, 22, 23) are configured as metalenses. In this embodiment, the metalens can converge the incident light, such as focusing, deflected focusing, etc., which is achieved through phase selection.

[0076] Specifically, the metalens includes a plurality of superstructure units arranged in an array, and nanostructures 25 are provided at the centers and / or vertices of each superstructure unit. The superstructure units are obtained by dividing the metalens, and each nanostructure 25 is the center of a structural unit. The plurality of nanostructures 25 are arranged on the dielectric layer 30, and the nanostructures 25 in each period form a superstructure unit. The superstructure units are close-packed patterns, such as a regular quadrilateral, a regular hexagon, a sector, etc. Each period contains one nanostructure 25, and nanostructures 25 can be provided at the vertices and / or centers of the superstructure units. When the superstructure unit is a regular hexagon, at least one nanostructure 25 is provided at each vertex and the center of the regular hexagon. Similarly, the same is true for the cases of a sector and a square.

[0077] Specifically, the plurality of superstructure units of the metalens are arranged periodically or aperiodically. Periodic arrangement means that the period values of the superstructure units are equal. Aperiodic arrangement means that the period values of the superstructure units are at least partially different, where the different period values can be the differences in the radial period value and / or the circumferential period value, or the differences in the period value in the X-axis direction and / or the Y-axis direction.

[0078] Specifically, the nanostructures 25 are configured as polarization-related structures or polarization-unrelated structures. The nanostructures 25 can be selected as polarization-related structures or polarization-unrelated structures according to different usage scenarios. Polarization-unrelated structures, such as a cylindrical shape, a square column shape, a cross column shape, a circular hole square column shape, etc. Polarization-related structures, such as an elliptical cylinder shape, a rectangular column shape, a hexagonal column shape, etc. The nanostructures 25 can be positive structures or negative structures. For example, the shapes of the nanostructures 25 include a cylinder, a hollow cylinder, a square prism, a hollow square prism, etc.

[0079] Specifically, the refractive index of the dielectric layer 30 is lower than that of the nanostructure 25. The material of the dielectric layer 30 can be any material with a low refractive index and absorption coefficient in the visible or near-infrared band, such as: silicon dioxide (SiO2), spin-on glass (SOG), or polymers such as polymethyl methacrylate (PMMA), polydimethylsiloxane (PDMS), poly-4-methyl-1-pentene (PMP), etc., as well as combinations of the above materials. The nanostructure 25 can be selected from dielectric materials with a refractive index higher than that of the dielectric layer 30, such as: monocrystalline silicon (c-Si), polycrystalline silicon (p-Si), amorphous silicon (a-Si), compound semiconductors (such as GaN, GaP, GaAs, SiC, etc.), TiO2, Si3N4, AlSb, AlAs, AlGaAs, AlGaInP, BP, ZnGeP2 and other suitable materials, as well as combinations of the above materials. The metalens further includes a protective layer 26 covering the nanostructure 25. The material of the protective layer 26 is similar to that of the dielectric layer 30 and can be any material with a low refractive index and absorption coefficient in the visible or near-infrared band, such as: silicon dioxide (SiO2), spin-on glass (SOG), or polymers such as polymethyl methacrylate (PMMA), polydimethylsiloxane (PDMS), poly-4-methyl-1-pentene (PMP), as well as combinations of the above materials. It can also be air (i.e., without setting the protective layer 26).

[0080] Specifically, the metalens can modulate the amplitude, phase, and polarization of the incident light through the nanostructure 25 provided thereon. The phase distribution of the metalens at least satisfies:

[0081]

[0082]

[0083]

[0084] where λ is the designed wavelength in free space, f is the focal length propagating in the dielectric layer 30, and the dielectric here is SiO2, dx, dy are the deflection distances of the focus from the center point of the metasurface, θ is the deflection angle along the x direction, and the same applies if deflecting along the y direction. Moreover, Equation (1) is applicable to the scenario of focusing on the center, and Equations (2) and (3) are applicable to the scenario of deflected focusing.

[0085] Further, with reference to Figure 4 As shown, all metasurface units (21, 22, 23) are configured as gratings. In this embodiment, the grating (such as a metagrating) can deflect the incident light and improve the efficiency of large-angle routing. Since the metagrating has no phase gradient and is completely periodic, it can maintain a high efficiency in the case of large angles.

[0086] The deflection angle of the grating can be calculated by the grating equation, and the grating equation is as follows:

[0087]

[0088] where n0 is the refractive index of the medium in the incident direction, k0 is the incident wave vector 2π / λ0, θ0 is the incident angle in the x-axis direction, m is the diffraction order, which can be 0, ±1, ±2, etc., P0 is the period in the x-axis direction, n m is the refractive index of the medium in the exit direction, and θ m is the diffraction angle.

[0089] Therefore, the metasurface units (21, 22, 23) route lights of different bands respectively, or it can be said that the metasurface units (21, 22, 23) converge or deflect lights of different bands respectively.

[0090] In an embodiment not shown, some of the metasurface units (21, 22, 23) can be configured as a metalens, and the other part of the metasurface units can be configured as a grating, so as to meet different usage requirements.

[0091] Furthermore, with reference to Figure 5 , Figure 6 and Figure 7 shown, the distance between the metasurface 20 and the optoelectronic sensing device 10 along the optical axis is not greater than the modulation focal length of each metasurface unit (21, 22, 23) for the light of the target band of the corresponding pixel unit (11, 12, 13). In this embodiment, the modulation of each metasurface unit (21, 22, 23) for the light of the target band of the corresponding pixel unit (11, 12, 13) adopts a defocusing method, that is, the focus of the light of the target band is located on the side of the target pixel (111, 121, 131) away from the photosensitive surface, so that the spot area of the light of the target band on the target pixel is larger, and the energy collected by the target pixel is more, which is beneficial for the target pixel to sense the light of the target band.

[0092] Moreover, the distance between the metasurface 20 and the optoelectronic sensing device 10 along the optical axis is not equal to the modulation focal length of each metasurface unit (11, 12, 13) for the light of the target wavelength band of other pixel units. Since the three target wavelength bands are different, the modulation focal length of the same metasurface unit for the three target wavelength bands decreases as the wavelength increases. Therefore, by adjusting the distance between the metasurface 20 and the optoelectronic sensing device 10 (i.e., adjusting the thickness of the dielectric layer 30 along the optical axis direction), and / or adjusting the modulation focal length of each metasurface unit (11, 12, 13) for the light of the target wavelength band, the light that cannot be sensed by the target pixel (e.g., the first target pixel 111) (e.g., the second target wavelength band light and / or the third target wavelength band light) can be routed or diverged to the reference pixel (e.g., the first reference pixel 112) of the pixel unit or diverged to other pixel units (e.g., the second pixel unit 12 and / or the third pixel unit 13), reducing the influence of the light of other target wavelength bands on the target pixel.

[0093] Specifically, since the wavelengths of the first target wavelength band, the second target wavelength band, and the third target wavelength band decrease in sequence, the modulation focal lengths of the same metasurface unit (21, 22, 23) for the light of the first target wavelength band (R), the second target wavelength band (G), and the third target wavelength band (B) increase in sequence. As Figure 5 , the distance between the metasurface 20 and the optoelectronic sensing device 10 is set to be less than the modulation focal length of the first metasurface unit 21 for the light of the first target wavelength band (R), so that the first metasurface unit 21 can route the light of the second target wavelength band (G) and the third target wavelength band (B) to the first reference pixel 112 without focusing. As Figure 6 , the distance between the metasurface 20 and the optoelectronic sensing device 10 is greater than the modulation focal length of the second metasurface unit 22 for the light of the first target wavelength band (R) and less than the modulation focal length of the second metasurface unit 22 for the light of the second target wavelength band (G), so that the second metasurface unit 22 can route the light of the third target wavelength band (B) to the second reference pixel 122 without focusing, and the second metasurface unit 22 can focus and diverge the light of the first target wavelength band (R) to the second reference pixel 122. As Figure 7 , the distance between the metasurface 20 and the optoelectronic sensing device 10 is greater than the modulation focal length of the third metasurface unit 23 for the light of the second target wavelength band (G) and less than the modulation focal length of the third metasurface unit 23 for the light of the third target wavelength band (B), so that the third metasurface unit 23 can focus and diverge the light of the first target wavelength band (R) and the second target wavelength band (G) to the second reference pixel 122.

[0094] Specifically, in cooperation with reference to Figure 8As shown, each pixel unit includes at least one target pixel and multiple reference pixels. All the target pixels together form a target area 111s, and all the reference pixels together form a reference area 112s. In this embodiment, preferably, as Figure 8 The target area 111s and the reference area 112s represented by different shades in

[0095] Specifically, with reference to Figure 9 、 Figure 10 and Figure 11 shown, the reference area 112s surrounds at least one side of the target area 111s. In this embodiment, the number of pixels in the target area 111s is not greater than the number of pixels in the reference area 112s. As Figure 9 shown, the pixel unit adopts a 2×2 pixel array, and the reference area 112s surrounds two sides of the target area 111s. As Figure 10 shown, the pixel unit adopts a 3×3 pixel array, and the reference area 112s surrounds four sides of the target area 111s. As Figure 11 shown, the pixel unit adopts a 4×4 pixel array, Figure 11 In Figure 11 a, the reference area 112s surrounds one side of the target area 111s, Figure 11 In

[0096] b, the reference area 112s surrounds two sides of the target area 111s,

[0097] In Figure 2 、 Figure 10 c, the reference area 112s surrounds three sides of the target area 111s. Figure 9 、 Figure 11 Of course, in an embodiment not shown, the pixel unit can also adopt a pixel array with a larger number, as long as it is ensured that the reference area 112s surrounds at least one side of the target area 111s. Or, it can also be that the target area 111s surrounds at least one side of the reference area 112s.

[0098] With reference to Figure 12As shown, the present invention provides an implementation manner in which a metasurface is configured as a metalens. In this implementation manner, each metasurface unit can modulate light of two bands (i.e., target band light and reference band light) and route them to specific regions (such as target region 111s and reference region 112s), respectively.

[0099] Specifically, each metasurface unit includes a target phase set for the target band light of the corresponding pixel unit and a reference phase set for the reference band light of the corresponding pixel unit. One of the target phase and the reference phase selects a focusing phase, and the other of the target phase and the reference phase selects a vortex phase. In this embodiment, since it is preferably adopted that "the reference region 112s surrounds at least one side of the target region 111s" for each pixel unit, for this solution, it is preferably that the target phase selects the focusing phase and the reference phase selects the vortex phase, so as to route the light of the target band to the target region 111s and route the light of the reference band to the reference region 112s.

[0100] Of course, in an embodiment not shown, when each pixel unit adopts the technical solution that "the target region 111s surrounds at least one side of the reference region 112s", it may also be that the reference phase selects the focusing phase and the target phase selects the vortex phase.

[0101] Specifically, the focusing phase is the above formula (1), which focuses the light of the target band at the center position of the pixel unit (i.e., the target region 111s). The vortex phase can focus the light of the reference band to the edge position of the pixel unit (i.e., the reference region 112s), or in other words, the vortex phase can form a hollow light beam. Thus, the energy of the reference band light is increased, which is convenient for the reference pixel to sense, and the influence of the reference band light on the target band pixel is reduced.

[0102] Specifically, the phase formula of the vortex phase is:

[0103]

[0104] where m is the topological charge number.

[0105] Furthermore, in order to facilitate the reference pixel to better sense the reference band light modulated by the vortex phase, the vortex phase preferably selects a vortex focusing phase. Using the vortex focusing phase can adjust the incident light of the corresponding band (i.e., the reference band light) into a defocused vortex light, thereby forming a hollow circular light spot, similar to a "doughnut" shape. By adjusting the topological charge number, hollow light beams of different sizes can be realized.

[0106] Of course, in embodiments not shown, the vortex phase can also adopt a vortex divergence phase, or other vortex phases, as long as the light in the reference band can be routed to the reference area 112s corresponding to the reference pixel, so as to avoid the influence of the light in the reference band on the target pixel.

[0107] With reference to Figure 13 shown, the present invention provides another embodiment in which the metasurface is configured as a superlens. In this embodiment, each metasurface unit is divided into multiple parts, and only the phase of modulating light of one band is arranged in each part, which can simplify the design difficulty of the metasurface. In this embodiment, the same reference numerals represent the same elements with similar functions and will not be described in detail.

[0108] Specifically, each pixel unit includes a target area 111s and a reference area 112s, and each metasurface unit includes a target part 21a corresponding to the target area 111s and a reference part 21b corresponding to the reference area 112s. In this embodiment, the target part 21a of each metasurface unit is opposite to the target area 111s of the corresponding pixel unit along the optical axis direction, and the reference part 21b of each metasurface unit is opposite to the reference area 112s of the corresponding pixel unit along the optical axis direction. Taking the first metasurface unit 21 as an example, in combination with Figure 8 , the target part 21a is opposite to the target area 111s along the optical axis direction, and the reference part 21b is opposite to the reference area 112s along the optical axis direction. Among them, the area of the target part 21a can be exactly the same as that of the target area 111s (for example Figure 13 b), and the area of the target part 21a can also be partially the same as that of the target area 111s (for example Figure 13 a, the circle of the target part 21a is inscribed in the target area 111s), as long as it is ensured that the target part 21a is opposite to the target area 111s along the optical axis. Similarly, the reference part 21b and the reference area 112s are the same.

[0109] Further, reference phases are arranged within the target portion 21a to route light of a reference band to the reference region 112s, and target phases are arranged within the reference portion 21b to route light of a target band to the target region 111s. In this embodiment, the reference phase (such as a vortex phase) arranged within the target portion 21a modulates only light of one band (i.e., the reference band light), thereby routing the reference band light to the reference region 112s, and light of other bands (such as the target band light) can directly pass through the target portion 21a and be incident on the target region 111s (i.e., the target pixel). The target phase (such as a focusing phase) arranged within the reference portion 21b modulates only light of one band (i.e., the target band light), thereby routing the target band light to the target region 111s, and light of other bands (such as the reference band light) can directly pass through the reference portion 21b and be incident on the reference region 112s (i.e., the reference pixel).

[0110] Thus, this embodiment can also route light of two bands to specific regions respectively, and the phase arranged within each part of the metasurface unit only needs to consider one band, thereby simplifying the design difficulty of each part of the metasurface unit.

[0111] Further, the optoelectronic sensing device 10 includes a plurality of color pixel groups 100 arranged in an array, each color pixel group includes at least four pixel units, and two of the at least four pixel units sense light of the same band. In this embodiment, four pixel units together form a color pixel group 100. After a plurality of color pixel groups 100 are arranged in an array along the X-axis and / or Y-axis, they together form the optoelectronic sensing device 10, so as to meet the imaging requirements of a color image sensor. Three of the four pixel units sense different target bands, and the other pixel unit senses the same target band as one of the first three.

[0112] Specifically, the optoelectronic sensing device 10 further includes a fourth pixel unit 14. The fourth pixel unit 14 includes a fourth target pixel 141 that senses light of a fourth target band and a fourth reference pixel 142 that senses a fourth reference band. The fourth target pixel 141 senses the same band as the second target pixel 121, that is, the fourth target band is the same as the second target band, both are green (G) light. Moreover, the fourth reference band is also the same as the second reference band. The metasurface 20 further includes a fourth metasurface unit 24 corresponding to the fourth pixel unit 14. The structure and function of the fourth metasurface unit 24 are the same as those of the second metasurface unit 22. Thus, the first metasurface unit 21, the second metasurface unit 22, the third metasurface unit 23, and the fourth metasurface unit 24 together form a color super unit 200, and the color super unit 200 corresponds to the color pixel group 100 one by one.

[0113] Exemplarily, such asFigure 2 In any regular quadrilateral color pixel group 100, two green (G) pixels are arranged along one diagonal line of the regular quadrilateral, and red (R) pixels and blue (B) pixels are arranged along the other diagonal line, thereby forming a Bayer pattern arrangement.

[0114] In some embodiments not shown, the color pixel group 100 may also not be arranged in a Bayer pattern, as long as the first target band, the second target band, and the third target band use other bands, such as magenta, cyan, yellow, etc., or other bands of light outside the visible light band, such as any one or more combinations of the visible light band, near-infrared band, mid-infrared band, and far-infrared band.

[0115] Further, continue to cooperate with reference Figure 5 , Figure 6 and Figure 7 As shown, a filter film (113, 123, 133) is provided on the light incident side of each target pixel (111, 121, 131), and the filter film is configured to transmit light corresponding to the target wavelength band. In this embodiment, the filter film is configured as a bandpass filter, thereby filtering out light of other wavelengths and only transmitting light of the target wavelength band to the target pixel, thereby reducing the impact of other stray light on the sensing of the target pixel.

[0116] According to another aspect of the present invention, an optical device is further provided. The optical device comprises an optical system and an image sensor according to the present invention. The image sensor is located in an image plane of the optical system.

[0117] According to another aspect of the present invention, a method for designing an image sensor is further provided. The structure and function of the image sensor are as described above and will not be described in detail here.

[0118] Specifically, the image sensor design method includes the following steps:

[0119] Step S10, determining at least three pixel units (11, 12, 13). In this embodiment, according to the working band of the image sensor (for example, any one or more combinations of the visible light band, the near infrared band, the mid-infrared band and the far infrared band) or the use requirements (for example, color imaging), the corresponding pixel units are set, as long as the wavelengths of the light sensed by the three pixel units are different. For example, Figure 2 The Bayer pattern arrangement in the image processing unit is four pixel units, namely RGGB, which also includes three pixel units that sense light of different wavelength bands.

[0120] Step S20: Each pixel unit is provided with a plurality of pixels for sensing light of different wavelengths. In this embodiment, each pixel unit senses light of at least two different wavelengths, for example, visible light (eg, red, green, blue) and infrared light (eg, near infrared IR).

[0121] Step S30: Design the metasurface units (21, 22, 23) such that incident light of different bands is respectively routed to the pixels in the pixel units (11, 12, 13) that match the bands. In this embodiment, according to the different bands of the light sensed by the pixel units, the metasurface units are correspondingly designed to meet the routing requirements for incident light of different bands.

[0122] Furthermore, in step S20, each pixel unit (11, 12, 13) is provided with a target pixel (111, 121, 131) for sensing the target band and a reference pixel (112, 122, 132) for sensing the reference band. In this embodiment, by setting the reference pixels, the optoelectronic sensing device 10 can sense more optical signals, increasing the energy utilization rate, and can also be used as reference light for facilitating color reconstruction.

[0123] Furthermore, in step S30, design the metasurface units (21, 22, 23) such that the light of the target band and at least part of the light of the reference band converge or deflect to the pixels in the corresponding pixel units that match the bands. In this embodiment, for the target band and the reference band, the metasurface units are correspondingly designed to meet the routing requirements for incident light of different bands.

[0124] Furthermore, in step S10, determine at least four pixel units that form the color pixel group 100, and the bands of the light sensed by two of the at least four pixel units are the same. In this embodiment, four pixel units together form a color pixel group 100, and after multiple color pixel groups 100 are arranged in an array along the X-axis and / or Y-axis, the optoelectronic sensing device 10 is formed to meet the imaging requirements of a color image sensor.

[0125] Specifically, the first target band light sensed by the first pixel unit 11 is red (R) light, the target band lights sensed by the second pixel unit 12 and the fourth pixel unit 14 are green (G) light, and the third target band light sensed by the third pixel unit 13 is blue (B) light. In any regular quadrilateral color pixel group 100, two green (G) pixels are arranged along one diagonal of the regular quadrilateral, and a red (R) pixel and a blue (B) pixel are respectively arranged along the other diagonal, thus forming a Bayer pattern layout.

[0126] Furthermore, in step S30, design the metasurface unit as a metalens and / or a grating. In this embodiment, the metasurface unit realizes the routing requirements for incident light of different bands through the ways of convergence (i.e., metalens, such as focusing, deflecting and focusing) and / or deflection (i.e., grating) to meet more usage requirements.

[0127] For the solution where the metasurface is configured as a metalens, the present invention provides an implementation manner for designing the metasurface.

[0128] Specifically, in step S30, the focusing phase is selected as the modulation phase for the light of the target band, and the modulation focal length of the metasurface unit for the corresponding target band light is designed to be not less than the distance between the metasurface and the optoelectronic sensing device along the optical axis. In this embodiment, the metasurface unit selects a metalens, and for the phase modulation of the target band light, the above formulas (1), (2), and (3) can be used. By adjusting the distance between the metasurface 20 and the optoelectronic sensing device 10 (i.e., adjusting the thickness of the dielectric layer 30 along the optical axis direction), and / or adjusting the modulation focal length of each metasurface unit (11, 12, 13) for the target band light, the light that is not sensed by this target pixel can be diverged to the reference pixel of this pixel unit or diverged to other pixel units, reducing the influence of other target band light on this target pixel.

[0129] Furthermore, in step S30, the vortex phase is selected as the modulation phase for the reference band light, and the topological charge number is set according to the reference pixel. In this embodiment, the formula (5) is used to obtain the phase required to form a hollow beam. By adjusting the topological charge number, hollow beams of different sizes can be realized.

[0130] For the solution where the metasurface is configured as a metalens, the present invention provides another implementation manner for designing the metasurface.

[0131] Specifically, in step S30, each metasurface unit includes a target portion 21a and a reference portion 21b. The reference phase for modulating the corresponding reference band light is arranged in the target portion 21a, and the target phase for modulating the corresponding target band light is arranged in the reference portion 21b. In this embodiment, the phases are designed for the target portion and the reference portion respectively, and each portion only modulates the light of one band, and the light of other bands can directly pass through this portion, thereby simplifying the design difficulty of the metasurface and achieving the purpose of respectively routing different band lights to specific regions.

[0132] The design method of the image sensor further includes the following steps:

[0133] Step S40: Perform a Bloch boundary scan on the superstructure unit to obtain the transmittance curve and phase curve of nanostructures with different characteristic sizes under different target band lights, and construct a characteristic size-phase library with high transmittance corresponding to each target band.

[0134] In this embodiment, as Figure 14, the dashed box marks the characteristic dimension-phase library with high transmittance corresponding to the first target band, that is, when the characteristic dimension (CD, critical dimension) is selected between 100 nm and 220 nm, the first metasurface unit 21 can obtain a relatively high transmittance and meet the phase requirement of 0 - 2π. For example, Figure 15 , the dashed box marks the characteristic dimension-phase library with high transmittance corresponding to the second target band, that is, when the characteristic dimension is selected between 100 nm and 190 nm, the second metasurface unit 22 can obtain a relatively high transmittance and meet the phase requirement of 0 - 2π. For example, Figure 16 , the dashed box marks the characteristic dimension-phase library with high transmittance corresponding to the third target band, that is, when the characteristic dimension is selected between 100 nm and 160 nm, the third metasurface unit 23 can obtain a relatively high transmittance and meet the phase requirement of 0 - 2π.

[0135] Therefore, the range of the characteristic dimension decreases successively from the first target band, the second target band to the third target band. The target band with a larger characteristic dimension range is easier to control than the target band with a smaller characteristic dimension range, that is, the target band with a larger characteristic dimension range has more selectivity. Using this characteristic, it is possible to easily route the light of other target bands with relatively smaller characteristic dimension ranges to the reference pixels. For example, Figure 17 , after obtaining the phase diagram of the black part of the first pixel unit 21, it can be seen from the phase diagram that the light of the first target band (R) well meets the phase requirement of 0 - 2π and has a good focusing phase. Therefore, the light of the first target band is well focused on the first target pixel 111. On the contrary, the phases of the light of the second target band (G) and the third target band (B) are relatively chaotic and the focusing effect is poor. Therefore, it is difficult for the light of the second target band and the third target band to be focused on the first target pixel 111, but diverges to the first reference pixel 112 to avoid affecting the first target pixel 111.

[0136] Furthermore, in step S40, a suitable characteristic dimension of the nanostructure 25 is selected from the characteristic dimension-phase library with high transmittance corresponding to each target band to arrange the nanostructure 25 for the corresponding metasurface units (21, 22, 23), thereby improving the overall transmittance.

[0137] For example, Figure 18, after simulating the first pixel unit 21, it can be seen that the routing efficiency of the light in the first target band (R) (i.e., light with a wavelength of 630 nm) (R in the figure) reaches 75%, which is three times that of the prior art (the maximum incident light energy is only 1 / 3). Moreover, since the characteristic size-phase library with high transmittance corresponding to each target band is constructed, filtering out some nanostructures 25 with low transmittance, the routing efficiency of other wavelengths (such as the first reference band of 900 nm) (W in the figure) also reaches 75%. Furthermore, the transmittance of the entire first pixel unit 21 (ALL in the figure) is greater than 90%.

[0138] As Figure 19 , after simulating the color super unit 200, it can be seen that the routing efficiencies of the light in the first target band (R), the second target band (G), and the third target band (B) are all above 70%, and the mutual interference is within 25%.

[0139] It should be understood that although this specification is described according to the embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0140] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present invention, and they are not used to limit the protection scope of the present invention. Any equivalent embodiments or changes made without departing from the technical spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An image sensor, characterized in that, Comprising: An optoelectronic sensing device, the optoelectronic sensing device including at least three pixel units, each pixel unit including a plurality of pixels that sense light in different wavelength bands; A metasurface, the metasurface including metasurface units corresponding one-to-one with the pixel units; Wherein, each metasurface unit is configured to perform phase modulation on incident light such that light in different wavelength bands is respectively routed to the pixels in the pixel unit that match the wavelength band.

2. The image sensor according to claim 1, wherein, Each pixel unit includes target pixels that sense a target wavelength band and reference pixels that sense a reference wavelength band, the target wavelength bands sensed by different pixel units being different, and the reference wavelength bands sensed by different pixel units being different or at least partially the same.

3. The image sensor according to claim 2, wherein Each metasurface unit routes light in the target wavelength band and at least part of the light in the reference wavelength band to the pixels in the corresponding pixel unit that match the wavelength band.

4. The image sensor according to claim 2, wherein The reference wavelength bands sensed by different pixel units are the same and are all different from the target wavelength band of each pixel unit.

5. The image sensor according to claim 1, characterized in that, All metasurface units are configured as superlenses or gratings; Alternatively, a part of the metasurface units are configured as superlenses and another part of the metasurface units are configured as gratings.

6. The image sensor according to claim 2, wherein The distance between the metasurface and the optoelectronic sensing device along the optical axis is not greater than the modulation focal length of each metasurface unit for the light in the target wavelength band of the corresponding pixel unit.

7. The image sensor according to claim 2, wherein Each pixel unit includes at least one target pixel and a plurality of reference pixels, all the target pixels together form a target area, all the reference pixels together form a reference area, and the reference area surrounds at least one side of the target area.

8. The image sensor according to claim 2, wherein, Each metasurface unit includes a target phase set for the light in the target wavelength band of the corresponding pixel unit and a reference phase set for the light in the reference wavelength band of the corresponding pixel unit, one of the target phase and the reference phase being selected as the focusing phase and the other of the target phase and the reference phase being selected as the vortex phase.

9. The image sensor according to claim 2, characterized in that, Each pixel unit includes a target area and a reference area, each metasurface unit includes a target part corresponding to the target area and a reference part corresponding to the reference area, the reference phase is arranged in the target part to route the light in the reference wavelength band to the reference area, and the target phase is arranged in the reference part to route the light in the target wavelength band to the target area.

10. The image sensor according to claim 1, wherein, The optoelectronic sensing device includes a plurality of color pixel groups arranged in an array, each color pixel group including at least four pixel units, and the light wavelength bands sensed by two of the at least four pixel units are the same.

11. The image sensor according to claim 2, wherein, A filter film is provided on the light incident side of each target pixel, and the filter film is configured to transmit light corresponding to the target wavelength band.

12. An optical device, characterized in that, The optical device includes an optical system and an image sensor as described in any one of claims 1-11, and the image sensor is located at the image plane of the optical system.

13. A design method of an image sensor as described in any one of claims 1-11, characterized in that, Comprising: Determining at least three pixel units; Each pixel unit is provided with a plurality of pixels that sense light in different wavelength bands; Designing metasurface units such that incident light in different wavelength bands is respectively routed to the pixels in the pixel unit that match the wavelength band.

14. The design method of the image sensor according to claim 13, characterized in that, Each pixel unit is provided with target pixels that sense a target wavelength band and reference pixels that sense a reference wavelength band; Design the metasurface unit such that the light in the target band and at least part of the light in the reference band converge or deflect to the pixels matching the band in the corresponding pixel unit.

15. The design method of the image sensor according to claim 13, characterized in that, Design the metasurface unit as a metalens and / or grating.

16. The design method of the image sensor according to claim 14, characterized in that, Select the focusing phase as the modulation phase for the light in the target band, and design the modulation focal length of the metasurface unit for the corresponding light in the target band to be not less than the distance between the metasurface and the optoelectronic sensing device along the optical axis.

17. The design method of the image sensor according to claim 14, characterized in that, Select the vortex phase as the modulation phase for the light in the reference band, and set the topological charge number according to the reference pixel.

18. The design method of the image sensor according to claim 14, characterized in that, Each metasurface unit includes a target part and a reference part. In the target part, reference phases for modulating the corresponding light in the reference band are arranged, and in the reference part, target phases for modulating the corresponding light in the target band are arranged.

19. The design method of the image sensor according to claim 14, characterized in that, Perform Bloch boundary scanning on the superstructure unit to obtain the transmittance curves and phase curves of nanostructures with different characteristic sizes under different target band lights, and construct a characteristic size-phase library with high transmittance corresponding to each target band.

20. The design method of the image sensor according to claim 13, characterized in that, Determine at least four pixel units that make up the color pixel group, and the bands of the light sensed by two of the at least four pixel units are the same.

Images