Hyperspectral element, hyperspectral sensor comprising a hyperspectral element, and hyperspectral image generation apparatus

Abstract

A hyperspectral element includes: (1) a multi-filter including a first sub-filter that passes first wavelength light having a first wavelength and a second sub-filter that passes second wavelength light having a second wavelength different from the first wavelength; and (2) a multi-detector configured to detect the first wavelength light and the second wavelength light, wherein the first sub-filter and the second sub-filter can be arranged in series in an optical path of incident light incident on the multi-filter.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to Korean Patent Application No. 10-2020-0132137 filed with the Korean Intellectual Property Office on October 13, 2020, and Korean Patent Application No. 10-2021-0118543 filed with the Korean Intellectual Property Office on September 6, 2021, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] The apparatus and methods consistent with the example embodiments relate to hyperspectral elements, hyperspectral sensors, and hyperspectral image generation apparatus. Background Technology

[0004] Image sensors using filters are important optical devices in the field of optics. Because image sensors of related technologies incorporate various optical components, they are often large and bulky. Recently, in response to the miniaturization of image sensors, research has been conducted on simultaneously implementing integrated circuits and optical devices on a single semiconductor chip.

[0005] Hyperspectral imaging technology involves methods for simultaneously acquiring image and spectral information. Methods for generating hyperspectral images are mainly classified into scanning methods and non-scanning snapshot methods. Scanning methods are achieved by combining a scanning device with an image sensor for spectral analysis. Non-scanning snapshot methods involve directly performing hyperspectral measurements using different filters on the image pixels.

[0006] Scanning hyperspectral measurement methods are advantageous for obtaining high-resolution hyperspectral images, but the measurement time is long and it is difficult to miniaturize the scanning equipment. Similar to a conventional camera, non-scanning snapshot methods can perform measurements in a short time and have the advantage of miniaturization, but the spectral resolution and image pixel space are limited, and the image resolution deteriorates. Summary of the Invention

[0007] The exemplary embodiments at least address the problems and / or disadvantages described above, as well as other disadvantages not described above. Furthermore, the exemplary embodiments do not need to overcome the aforementioned disadvantages, and may not need to overcome any of the aforementioned problems.

[0008] One or more example embodiments provide hyperspectral elements with high resolution and broadband characteristics.

[0009] In addition, one or more example embodiments provide hyperspectral sensors with high resolution and broadband characteristics.

[0010] In addition, one or more example embodiments provide a hyperspectral image generation apparatus with high resolution and broadband characteristics.

[0011] According to one aspect of the embodiments, a hyperspectral element is provided, comprising: (1) a multi-filter, including: a first sub-filter for passing through first wavelength light having a first wavelength, and a second sub-filter for passing through second wavelength light having a second wavelength different from the first wavelength; and (2) a multi-detector configured to detect the first wavelength light and the second wavelength light, wherein the first sub-filter and the second sub-filter are arranged in series in the optical path of incident light incident on the multi-filter.

[0012] The transmission spectrum of the first sub-filter may include a first usage segment and a first sub-transmittance region outside the first usage segment. The transmission spectrum of the second sub-filter may include a second usage segment and a second sub-transmittance region outside the second usage segment. A first wavelength is included in the first usage segment and the second sub-transmittance region, and a second wavelength is included in the second usage segment and the first sub-transmittance region. The first sub-filter may be configured to filter out a first residual wavelength other than the first wavelength from the first usage segment, and the second sub-filter may be configured to filter out a second residual wavelength other than the second wavelength from the second usage segment.

[0013] The first and second usage segments can partially overlap.

[0014] The multiple detectors may include: a first sub-detector, including a first optical detection segment overlapping with a first wavelength and a second wavelength; and a second sub-detector, including a second optical detection segment overlapping with either the first wavelength or the second wavelength, wherein the first sub-detector and the second sub-detector are arranged in series.

[0015] The hyperspectral element may further include a spectral processor, a first sub-detector configured to generate a first channel signal about a first wavelength light and a second wavelength light, and to provide the first channel signal to the spectral processor, a second sub-detector configured to generate a second channel signal about a second wavelength light, and to provide the second channel signal to the spectral processor, and the spectral processor configured to generate information related to the intensity of the first wavelength light and the second wavelength light based on the first channel signal and the second channel signal.

[0016] The multi-detector may also include an ultraviolet detector, and the ultraviolet detector is arranged in series with the first sub-detector and the second sub-detector in the optical path of the incident light.

[0017] The multi-detector may also include an infrared detector, and the infrared detector may be arranged in series with the first sub-detector and the second sub-detector in the optical path of the incident light.

[0018] Each of the first sub-filter and the second sub-filter may include an alternately stacked first refractive index film and a second refractive index film, and the first refractive index film may have a different refractive index than the second refractive index film.

[0019] The first refractive index film of the first sub-filter and the first refractive index film of the second sub-filter can have different thicknesses.

[0020] Each of the first sub-filter and the second sub-filter may include: a first reflective layer; a second reflective layer; and a nanostructure layer disposed between the first reflective layer and the second reflective layer, wherein the nanostructure layer may include a plurality of nanorods, and the plurality of nanorods may be arranged asymmetrically.

[0021] Each of the first sub-filter and the second sub-filter may include: a first reflective layer; a second reflective layer; and a nanostructure layer disposed between the first reflective layer and the second reflective layer, the nanostructure layer including a plurality of nanopores, and the plurality of nanopores may be arranged asymmetrically.

[0022] The multiple detector may include alternately stacked n-type and p-type films, which may form a first photodiode and a second photodiode, respectively, and the second photodiode may be arranged further away from the multiple filters than the first photodiode.

[0023] Multiple filters can be set on multiple detectors, with a first photodiode configured to detect a first wavelength light and a second wavelength light, and a second photodiode configured to detect the longer wavelength of the first wavelength light and the second wavelength light.

[0024] The hyperspectral element may further include a spectral processor; a first photodiode is configured to generate a first channel signal relating to a first wavelength light and a second wavelength light, and to provide the first channel signal to the spectral processor; a second photodiode is configured to generate a second channel signal relating to the second wavelength light, and to provide the second channel signal to the spectral processor; and the spectral processor is configured to generate information relating to the intensity of the first wavelength light and the second wavelength light based on the first channel signal and the second channel signal.

[0025] The hyperspectral element may also include a microlens, and the microlens may be arranged in series with a multi-filter and collect incident light incident on the multi-filter.

[0026] According to one aspect of another embodiment, a hyperspectral sensor includes: a plurality of pixels configured to sense the transmission spectrum of incident light; a spectral processor; and a main processor, wherein each of the plurality of pixels may include: a first multi-filter for passing through a first wavelength light having a first wavelength and a second wavelength light having a second wavelength in the incident light, the second wavelength being longer than the first wavelength; a first multi-detector configured to generate a first channel signal and a second channel signal regarding the first wavelength light and the second wavelength light, and to provide the first channel signal and the second channel signal to the spectral processor; a second multi-filter for passing through a third wavelength light having a third wavelength and a fourth wavelength light having a fourth wavelength in the incident light, the fourth wavelength being longer than the third wavelength; and a second multi-detector configured to generate a third channel signal and a fourth channel signal regarding the third wavelength light and the fourth wavelength light, and to provide the third channel signal and the fourth channel signal to the spectral processor, and the spectral processor is configured to: generate information related to the intensity of the first wavelength light to the fourth wavelength light based on the first channel signal to the fourth channel signal, and to provide the information to the main processor.

[0027] The first multi-filter may include a first sub-filter and a second sub-filter. The transmission spectrum of the first sub-filter may include a first usage segment and a first sub-transmittance region outside the first usage segment. The transmission spectrum of the second sub-filter may include a second usage segment and a second sub-transmittance region outside the second usage segment. A first wavelength is included in the first usage segment and the second sub-transmittance region, and a second wavelength is included in the second usage segment and the first sub-transmittance region.

[0028] The first and second usage segments can partially overlap.

[0029] The second multi-filter may include a third sub-filter and a fourth sub-filter. The transmission spectrum of the third sub-filter may include a third usage segment and a third sub-transmission region. The transmission spectrum of the fourth sub-filter may include a fourth usage segment and a fourth sub-transmission region. A third wavelength may be included in the third usage segment and the fourth sub-transmission region, and a fourth wavelength may be included in the fourth usage segment and the third sub-transmission region.

[0030] The third and fourth usage segments can partially overlap.

[0031] The first and second multi-filters can be arranged in parallel, the first and second sub-filters can be arranged in series in the optical path of the incident light, and the third and fourth sub-filters can be arranged in series in the optical path of the incident light.

[0032] The first multi-detector may include: a first sub-detector configured to receive a first wavelength light and a second wavelength light and generate a first channel signal; and a second sub-detector configured to receive a second wavelength light and generate a second channel signal; and the second multi-detector includes: a third sub-detector configured to receive a third wavelength light and a fourth wavelength light and generate a third channel signal; and a fourth sub-detector configured to receive a fourth wavelength light and generate a fourth channel signal.

[0033] In each of the first multiple detectors, the first sub-detector and the second sub-detector can be arranged in series in the optical path of the incident light.

[0034] According to one aspect of another embodiment, a hyperspectral image generation apparatus includes: a hyperspectral element configured to receive incident light provided from a measurement target to generate a first channel signal relating to light having a first wavelength and light having a second wavelength, and a second channel signal relating to light having the first wavelength; a spectral processor configured to generate, based on the first channel signal and the second channel signal, first hyperspectral information relating to the intensity of light having the first wavelength at each location of the measurement target, and second hyperspectral information relating to the intensity of light having the second wavelength at each location of the measurement target; a main processor configured to generate hyperspectral image information based on the first hyperspectral information and the second hyperspectral information; and a display for displaying a hyperspectral image based on the hyperspectral image information. Attached Figure Description

[0035] The above and / or other aspects will become clearer by describing certain exemplary embodiments with reference to the accompanying drawings, in which:

[0036] Figure 1 This is a schematic block diagram of an image sensor according to an embodiment;

[0037] Figure 2 This is a block diagram of a hyperspectral element according to an embodiment;

[0038] Figure 3 It shows the use Figure 2 A flowchart of a method for measuring light using hyperspectral elements;

[0039] Figure 4 Show Figure 2 The transmission spectra of the first to third sub-filters and the transmission spectra of multiple filters;

[0040] Figure 5 Show Figure 2 The transmission spectrum of the multi-filter and the optical detection section from the first sub-detector to the third sub-detector;

[0041] Figure 6 This is a cross-sectional view of the hyperspectral element according to an embodiment;

[0042] Figure 7 yes Figure 6 Transmission spectrum curves from the first sub-filter to the third sub-filter;

[0043] Figure 8 yes Figure 6 The transmission spectrum of the multi-filter;

[0044] Figure 9 This is a cross-sectional view of the hyperspectral element according to an embodiment;

[0045] Figures 10 to 13 It is based on Figure 9 A perspective view of a multi-filter in an embodiment of a hyperspectral element;

[0046] Figure 14 yes Figure 9 The transmission spectrum of the multi-filter;

[0047] Figure 15 This is a cross-sectional view of the hyperspectral element according to an embodiment;

[0048] Figure 16 This is a cross-sectional view of the hyperspectral element according to an embodiment;

[0049] Figure 17 This is a block diagram of a hyperspectral sensor according to an embodiment;

[0050] Figure 18 yes Figure 17 A conceptual block diagram of pixels;

[0051] Figure 19 It shows the use Figure 17 A flowchart of a method for measuring light using a hyperspectral sensor;

[0052] Figure 20 yes Figure 18 The transmission spectrum curves of the multiple filters;

[0053] Figure 21 It is based on Figure 17 A cross-sectional view of the pixels of an embodiment of a hyperspectral sensor; and

[0054] Figure 22 This is a block diagram of a hyperspectral image generation apparatus including a hyperspectral sensor according to an embodiment;

[0055] Figure 23 This is a block diagram illustrating an example of an electronic device including an image sensor;

[0056] Figure 24 It is shown Figure 23 A schematic block diagram of the camera module; and

[0057] Figures 25 to 34 Various examples of electronic devices using image sensors according to embodiments are shown. Detailed Implementation

[0058] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein similar elements are indicated by similar reference numerals throughout the drawings. In this respect, the embodiments may take different forms and should not be construed as limited to the description set forth herein. Therefore, the embodiments are described below only by reference to the accompanying drawings to illustrate various aspects of this specification. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Throughout this disclosure, the expression “at least one of a, b, or c” means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0059] Embodiments will now be described in detail below with reference to the accompanying drawings. The embodiments to be described are merely exemplary and various modifications can be made according to the embodiments. In the following drawings, similar reference numerals denote similar elements, and the dimensions of each element in the drawings may be enlarged for clarity and ease of explanation.

[0060] In the following text, being described as “upper” or “above” can include not only being directly above and in contact with it, but also being above without being in contact with it.

[0061] Unless the context clearly indicates otherwise, a singular expression includes a plural expression. Furthermore, unless explicitly stated to the contrary, when a part “includes” a particular element, this indicates that other elements may also be included, rather than excluded.

[0062] In the following text, "arranged in series" means arranged on a single optical path.

[0063] In the following text, "arranged in parallel" means arranged on different optical paths.

[0064] Figure 1 This is a schematic block diagram of an image sensor 4000 according to an embodiment. (Reference) Figure 1 The image sensor 4000 may include a pixel array 4100, a timing controller (T / C) 4010, a line decoder 4020, and an output circuit 4030. The image sensor 4000 may include, but is not limited to, a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0065] The pixel array 4100 may include a multi-filter array 4110 and a multi-detector array 4120. The multi-filter array 4110 allows light of different wavelengths to pass through and includes multiple multi-filters arranged in a two-dimensional manner.

[0066] The multi-detector array 4120 includes multiple detectors for detecting light of different wavelengths that have been transmitted through the multiple multiple filters. Specifically, the multi-detector array 4120 includes multiple detectors arranged in a two-dimensional arrangement along multiple rows and columns.

[0067] The pixel array 4100 may include a plurality of pixels arranged in a two-dimensional manner. Each of the plurality of pixels may include a plurality of sub-pixels. Each of the plurality of sub-pixels may include a plurality of filters and a plurality of detectors corresponding to each other. The arrangement of the plurality of pixels can be implemented in various ways.

[0068] The row decoder 4020 selects one of the rows in the multi-detector array 4120 in response to the row address signal output from the timing controller 4010. The output circuit 4030 outputs the optical detection signals from the multiple multi-detectors arranged in the selected row, column by column. For this purpose, the output circuit 4030 may include a column decoder and an analog-to-digital converter (ADC). For example, the output circuit 4030 may include multiple ADCs arranged column by column between the column decoder and the multi-detector array 4120, or a single ADC arranged at the output of the column decoder. The timing controller 4010, the row decoder 4020, and the output circuit 4030 may be implemented as a single chip or as separate chips. A processor for processing the image signal output from the output circuit 4030 may be implemented as a single chip having the timing controller 4010, the row decoder 4020, and the output circuit 4030.

[0069] Figure 2 This is a block diagram of the hyperspectral element 10 according to an embodiment. Figure 3 It shows the use Figure 2 A flowchart of a method for measuring light using a hyperspectral element 10. Figure 4 Show Figure 2 Transmission spectra of the first to third sub-filters and transmission spectra of multiple filters. Figure 5 Show Figure 2 The transmission spectrum of the multi-filter and the optical detection section from the first sub-detector to the third sub-detector.

[0070] refer to Figure 2 A hyperspectral element 10 can be provided. The hyperspectral element 10 may include: a multi-filter 100, a multi-detector 200, and a spectral processor 300. The multi-filter 100 and the multi-detector 200 may be respectively included in a reference... Figure 1 The described multi-filter array ( Figure 1 4100) and multi-detector array ( Figure 1 In 4120). In other words, the multi-filter 100 can be a component of a multi-filter array ( Figure 1 (4110) is one of a plurality of multi-filters, and the multi-detector 200 may be one of a plurality of detector arrays ( Figure 1 One of multiple detectors (4120). The spectral processor 300 can be used with a reference... Figure 1 The processors described are essentially the same. The multiple filters 100 and multiple detectors 200 can be arranged in series. In this specification, "arranged in series" means arranged along a single optical path. The multiple filters 100 and multiple detectors 200 can be arranged along the optical path of the incident light IL. For example, the incident light IL can pass through the multiple filters 100 and multiple detectors 200 sequentially.

[0071] The multi-filter 100 may include a plurality of sub-filters 102, 104, and 106. The plurality of sub-filters 102, 104, and 106 may have different transmission properties. The transmission spectra of the plurality of sub-filters 102, 104, and 106 may be different from each other. Although the multi-filter 100 is shown as including a first sub-filter 102, a second sub-filter 104, and a third sub-filter 106, the number of sub-filters is not limited. In another embodiment, the multi-filter 100 may include fewer or more than three sub-filters. The multi-filter 100 may emit filtered light FL. The filtered light FL may be incident light IL filtered by the multi-filter 100. The filtered light FL may have multiple bands determined according to the different transmission properties of the plurality of sub-filters 102, 104, and 106. The transmission spectra of the first sub-filter 102, the second sub-filter 104, the third sub-filter 106, and the multiple filters 100 will be described below.

[0072] The multi-detector 200 can detect filtered light FL. The multi-detector 200 may include multiple sub-detectors 202, 204, and 206. The multiple sub-detectors 202, 204, and 206 may have different light detection properties. The light detection properties of the multiple sub-detectors 202, 204, and 206 will be described below. Although the multi-detector 200 is shown as including a first sub-detector 202, a second sub-detector 204, and a third sub-detector 206, the number of sub-detectors is not limited. In another embodiment, the multi-detector 200 may include fewer or more than three sub-detectors. The first sub-detector 202, the second sub-detector 204, and the third sub-detector 206 may be arranged in series. For example, the first sub-detector 202, the second sub-detector 204, and the third sub-detector 206 may be arranged sequentially in the optical path of the filtered light FL.

[0073] The multiple filters 100 and multiple detectors 200 of this disclosure can be configured in a sub-pixel, as described below. The first to third sub-filters 102, 104, and 106, and the first to third sub-detectors 202, 204, and 206 can be combined to measure multiple bands desired in a sub-pixel. A method for measuring multiple bands, performed by the hyperspectral element 10, is described below.

[0074] refer to Figure 3 and Figure 4 The incident light IL can be filtered by the multi-filter 100 to generate filtered light FL having a first band, a second band, and a third band (operation S110). The incident light IL can propagate from the target being measured and can be incident on the multi-filter 100. The filtered light FL can be obtained by allowing the first to third bands of the incident light IL to pass through and filtering out the remaining wavelengths that do not include the first to third bands from the incident light IL. Figure 3 The transmittance spectrum T① of the first sub-filter 102, T② of the second sub-filter 104, T③ of the third sub-filter 106, and T④ of the multiple filters 100 are shown. For ease of description, the transmittance of the first to third sub-filters 102, 104, and 106, and the transmittance of the multiple filters 100 are represented as 0 or 1.

[0075] The transmission spectrum T① of the first sub-filter 102 may include: a first transmission wavelength PB1, a first usage segment UB1, a first a sub-transmittance region STR1a, and a first a sub-transmittance region STR1b. The first usage segment UB1 may be located between the first a sub-transmittance region STR1a and the first a sub-transmittance region STR1b. In other words, the first a sub-transmittance region STR1a and the first a sub-transmittance region STR1b may be regions other than the first usage segment UB1. The first transmission wavelength PB1 may be adjusted within the first usage segment UB1. Light having a residual wavelength in the first usage segment UB1 that does not include the first transmission wavelength PB1 may be blocked by the first sub-filter 102. The first usage segment UB1 may be a band in which the first sub-filter 102 can be used as a transmission filter that allows light with a specific wavelength to pass through.

[0076] The transmission spectrum T② of the second sub-filter 104 may include a second transmission wavelength PB2, a second a sub-transmission region STR2a, and a second b sub-transmission region STR2b. A second usage segment UB2 may be disposed between the second a sub-transmission region STR2a and the second b sub-transmission region STR2b. In other words, the second a sub-transmission region STR2a and the second b sub-transmission region STR2b may be regions other than the second usage segment UB2. The second transmission wavelength PB2 can be adjusted within the second usage segment UB2. Light with a residual wavelength in the second usage segment UB2 that does not include the second transmission wavelength PB2 may be blocked by the second sub-filter 104. The second usage segment UB2 may be a band in which the second sub-filter 104 can function as a transmission filter that allows light with a specific wavelength to pass through.

[0077] The transmission spectrum T③ of the third sub-filter 106 may include a third transmission wavelength PB3, a third sub-transmission region STR3a, and a third sub-transmission region STR3b. A third usage segment UB3 may be disposed between the third sub-transmission region STR3a and the third sub-transmission region STR3b. In other words, the third sub-transmission regions STR3a and STR3b may be regions other than the third usage segment UB3. The third transmission wavelength PB3 may be adjusted within the third usage segment UB3. Light with wavelengths other than the third transmission wavelength PB3 within the third usage segment UB3 may be blocked by the third sub-filter 106. The third usage segment UB3 may be a band in which the third sub-filter 106 can function as a transmission filter that allows light with a specific wavelength to pass through.

[0078] The first segment UB1 can overlap with either the second sub-transmitting area STR2a or the second sub-transmitting area STR2b, and either the third sub-transmitting area STR3a or the third sub-transmitting area STR3b. For example, the first segment UB1 can overlap with both the second and third sub-transmitting areas STR2a and STR3a. The diagram shows the first segment UB1 partially overlapping the second sub-transmitting area STR2a and completely overlapping the third sub-transmitting area STR3a, but this is just an example. As another example, the first segment UB1 can completely overlap the second and third sub-transmitting areas STR2a and STR3a.

[0079] The first passing wavelength PB1 can be determined to completely overlap with any one of the second-a sub-transmitting regions STR2a and STR2b, which overlap with the first usage segment UB1, and any one of the third-a sub-transmitting regions STR3a and STR3b. For example, the first passing wavelength PB1 can completely overlap with the second-a sub-transmitting region STR2a and the third-a sub-transmitting region STR3a. The second usage segment UB2 can overlap with any one of the first-a sub-transmitting regions STR1a and STR1b, and any one of the third-a sub-transmitting regions STR3a and STR3b. For example, the second usage segment UB2 can overlap with the first-b sub-transmitting region STR1b and the third-a sub-transmitting region STR3a. The overlap of the second usage segment UB2 with each portion of the first-b sub-transmitting region STR1b and the third-a sub-transmitting region STR3a is shown, but this is only an example. As another example, the second segment UB2 can completely overlap with the 1b sub-transparent area STR1b and the 3a sub-transparent area STR3a.

[0080] The second transmission wavelength PB2 can be determined to completely overlap with any one of the first sub-transmitting region STR1a and the first sub-transmitting region STR1b that overlaps with the second usage segment UB2, as well as any one of the third sub-transmitting regions STR3a and STR3b. For example, the second transmission wavelength PB2 can completely overlap with the first sub-transmitting region STR1b and the third sub-transmitting region STR3a.

[0081] The third segment UB3 can overlap with either the first sub-transmitting area STR1a or the first sub-transmitting area STR1b, and either the second sub-transmitting area STR2a or the second sub-transmitting area STR2b. For example, the third segment UB3 can overlap with both the second sub-transmitting area STR2b and the first sub-transmitting area STR1b. The diagram shows the third segment UB3 completely overlapping the first sub-transmitting area STR1b and partially overlapping the second sub-transmitting area STR2b, but this is only an example. As another example, the third segment UB3 can completely overlap with both the second sub-transmitting area STR2b and the first sub-transmitting area STR1b.

[0082] The third transmission wavelength PB3 can be determined to completely overlap with any one of the following: the first sub-transmitting region STR1a, the first sub-transmitting region STR1b, and the second sub-transmitting region STR2a, which overlap with the first sub-transmitting region STR1a and the third usage segment UB3; and any one of the second sub-transmitting regions STR2a and STR2b. For example, the third transmission wavelength PB3 can completely overlap with the first sub-transmitting region STR1b and the second sub-transmitting region STR2b.

[0083] Sub-transmitting areas STR1a, STR2a, and STR3a (sub-transmitting areas 1a, 2a, and 3a) may include overlapping regions. For example, the overlapping areas of sub-transmitting areas STR1a, STR2a, and STR3a may be substantially the same as sub-transmitting area STR1a. Sub-transmitting areas STR1b, STR2b, and STR3b (sub-transmitting areas 1b, 2b, and 3b) may also include overlapping regions. For example, the overlapping areas of sub-transmitting areas STR1b, STR2b, and STR3b may be substantially the same as sub-transmitting area STR3b.

[0084] The transmission spectrum T④ of the multi-filter 100 can be determined based on the transmission spectrum T① of the first sub-filter 102, the transmission spectrum T② of the second sub-filter 104, and the transmission spectrum T③ of the third sub-filter 106. The wavelength range passing through the multi-filter 100 can be the wavelength range passing through the first sub-filter 102, the second sub-filter 104, and the third sub-filter 106. The wavelength range not passing through the multi-filter 100 can be the wavelength range not passing through at least one of the first sub-filter 102, the second sub-filter 104, or the third sub-filter 106.

[0085] The transmission spectrum T④ of the multi-filter 100 may include a first final passing wavelength TPB1, a second final passing wavelength TPB2, and a third final passing wavelength TPB3. The first final passing wavelength TPB1, the second final passing wavelength TPB2, and the third final passing wavelength TPB3 may be substantially the same as the first passing wavelength PB1, the second passing wavelength PB2, and the third passing wavelength PB3, respectively.

[0086] The transmission spectrum T④ of the multi-filter 100 may include: a first final sub-transmittance region TSTR1 and a second final sub-transmittance region TSTR2. The first final sub-transmittance region TSTR1 may have a substantially identical overlapping area with the first a sub-transmittance region STR1a, the second a sub-transmittance region STR2a, and the third a sub-transmittance region STR3a. For example, the first final sub-transmittance region TSTR1 may be substantially identical to the first a sub-transmittance region STR1a. The second final sub-transmittance region TSTR2 may have a substantially identical overlapping area with the first b sub-transmittance region STR1b, the second b sub-transmittance region STR2b, and the third b sub-transmittance region STR3b. For example, the second final sub-transmittance region TSTR2 may be substantially identical to the third b sub-transmittance region STR3b.

[0087] The band between the first final sub-transmittance region TSTR1 and the second final sub-transmittance region TSTR2 can be referred to as the final use band TUB. The final use band TUB can be a band in which the multi-filter 100 can be used as a transmission filter to allow light with a specific wavelength to pass through. The final use band TUB can be the sum of the first use band UB1, the second use band UB2, and the third use band UB3. The final use band TUB can be wider than each of the first use band UB1, the second use band UB2, and the third use band UB3.

[0088] This disclosure can provide a multi-filter 100 having a wider usage range than one of the sub-filters 102, 104 or 106.

[0089] refer to Figure 3 and Figure 5The system can provide the transmission spectrum T④ of the multi-filter 100, the first optical detection segment DB1 of the first sub-detector 202, the second optical detection segment DB2 of the second sub-detector 204, and the third optical detection segment DB3 of the third sub-detector 206. The first to third sub-detectors 202, 204, and 206 can respectively detect the filtered light FL and generate a first channel signal, a second channel signal, and a third channel signal (operation S120). The first to third optical detection segments DB1, DB2, and DB3 can be bands detectable by the first sub-detector 202, bands detectable by the second sub-detector 204, and bands detectable by the third sub-detector 206, respectively. The first to third optical detection segments DB1, DB2, and DB3 can overlap with each other. For example, the second optical detection segment DB2 and the third optical detection segment DB3 can be included in the first optical detection segment DB1, and the third optical detection segment DB3 can be included in the second optical detection segment DB2. In one example, first optical detection segments to third optical detection segments DB1, DB2, and DB3 can be positioned between a first final sub-transmittance region TSTR1 and a second final sub-transmittance region TSTR2. First optical detection segments to third optical detection segments DB1, DB2, and DB3 will not detect light with wavelengths present in the first final sub-transmittance region TSTR1 and the second final sub-transmittance region TSTR2. First optical detection segment DB1 may include a first final transmission wavelength TPB1, a second final transmission wavelength TPB2, and a third final transmission wavelength TPB3. The first final transmission wavelengths to the third final transmission wavelengths TPB1, TPB2, and TPB3 can be detected by a first sub-detector 202. The first sub-detector 202 can generate a first channel signal relating to the first final transmission wavelengths to the third final transmission wavelengths TPB1, TPB2, and TPB3. The first channel signal can be an electrical signal relating to the first final transmission wavelength TPB1, the second final transmission wavelength TPB2, and the third final transmission wavelength TPB3. The first sub-detector 202 can provide the first channel signal to the spectral processor 300.

[0090] The second optical detection segment DB2 may include a second final pass wavelength TPB2 and a third final pass wavelength TBP3, but may not include a first final pass wavelength TBP1. The second final pass wavelength TPB2 and the third final pass wavelength TPB3 can be detected by a second sub-detector 204. The second sub-detector 204 can generate a second channel signal relating to the second final pass wavelength TPB2 and the third final pass wavelength TBP3. The second channel signal can be an electrical signal relating to the second final pass wavelength TBP2 and the third final pass wavelength TBP3. The second sub-detector 204 can provide the second channel signal to the spectral processor 300.

[0091] The third optical detection segment DB3 may include a third final passing wavelength TPB3, but may not include the first final passing wavelength TBP1 and the second final passing wavelength TPB2. The third final passing wavelength TPB3 may be detected by the third sub-detector 206. The third sub-detector 206 may generate a third channel signal with respect to the third passing wavelength. The third channel signal may be an electrical signal with respect to the third passing wavelength. The third sub-detector 206 may provide the third channel signal to the spectral processor 300.

[0092] The spectral processor 300 can generate information related to the intensity of light having a first final passing wavelength TBP1, light having a second final passing wavelength TBP2, and light having a third final passing wavelength TBP3 based on the first channel signal to the third channel signal (operation S130).

[0093] This disclosure provides a multi-filter 100 comprising a plurality of sub-filters 102, 104, and 106 arranged in series. The final usable segment TUB of the multi-filter 100 may be equal to the sum of the first usable segments UB1, the second usable segment UB2, and the third usable segment UB3 of the plurality of sub-filters 102, 104, and 106. The multi-filter 100 may have a wider usable bandwidth than each of the sub-filters 102, 104, and 106. The multi-detector 200 of this disclosure may include a plurality of sub-detectors 202, 204, and 206 to detect multiple lights having multiple wavelengths. The plurality of sub-detectors 202, 204, and 206 may be arranged in a sub-pixel. Therefore, a sub-pixel may have a large area. Therefore, the hyperspectral element 10 may have high resolution.

[0094] Figure 6 This is a cross-sectional view of the hyperspectral element 12 according to an embodiment.

[0095] refer to Figure 6 A hyperspectral element 12 can be provided. The hyperspectral element 12 may include a multi-detector 210, a light-blocking layer ST, a multi-filter 110, a microlens ML, and a spectral processor 300. The multi-detector 210 and the multi-filter 110 can be connected to a reference... Figures 2 to 5 Those described have essentially the same properties.

[0096] Substrate 1 may include a semiconductor material. For example, substrate 1 may be a silicon (Si) substrate, a germanium (Ge) substrate, or a silicon-germanium (SiGe) substrate. Substrate 1 may include electronic devices. The electronic devices may form a processor that controls the multiple detectors 210 and receives channel signals from the multiple detectors 210. For example, the multiple detectors 210 may be electrically connected to the electronic devices.

[0097] The optical detection properties of the multi-detector 210 can be compared with those of the reference. Figures 2 to 5 The optical detection properties of the described multi-detector 200 are substantially the same. The multi-detector 210 may include multiple p-type layers PL and multiple n-type layers NL stacked alternately. Figure 6 The diagram shows two p-type layers PL and two n-type layers NL as an example. The two p-type layers PL and the two n-type layers NL can be stacked alternately to form three photodiodes PD1, PD2, and PD3. The three photodiodes PD1, PD2, and PD3, arranged in order of their distance from the multi-filter 110 from closest to farthest, can be referred to as the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3.

[0098] The photodetector properties of the first to third photodiodes PD1, PD2, and PD3 are substantially the same as those of the first to third sub-detectors 202, 204, and 206. The first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 can generate a first-channel signal, a second-channel signal, and a third-channel signal, respectively. The first to third photodiodes PD1, PD2, and PD3 can provide the first to third-channel signals to the spectral processor 300. The spectral processor 300 can generate information related to the intensity of light with different wavelengths based on the first to third-channel signals.

[0099] A light-blocking layer ST can be disposed on the side of the multi-detector 210. The light-blocking layer ST prevents the multi-detector 210 from detecting unwanted light. The light-blocking layer ST can include a light-blocking material. For example, the light-blocking layer ST can include amorphous silicon or amorphous germanium.

[0100] Multiple filters 110 can be disposed on multiple detectors 210. The light transmission properties of multiple filters 110 can be compared with those of a reference detector. Figures 2 to 5 The light transmission properties of the described multi-filter 110 are substantially the same. The multi-filter 110 may include multiple sub-filters 112, 114, and 116. A first sub-filter 112, a second sub-filter 114, and a third sub-filter 116 are shown as examples. The light transmission properties of the first to third sub-filters 112, 114, and 116 can be substantially the same as the light detection properties of the first to third sub-detectors 202, 204, and 206, respectively. The first to third sub-filters 112, 114, and 116 can be arranged in series. For example, the third sub-filter 116, the second sub-filter 114, and the first sub-filter 112 can be sequentially stacked from the side of the multi-detector 210 opposite to the substrate 1 side. However, the stacking order of the first to third sub-filters 112, 114, and 116 is not limited. The positions of the first sub-filter 112, the second sub-filter 114, and the third sub-filter 116 can be interchanged.

[0101] The first to third sub-filters 112, 114, and 116 may each comprise a distributed Bragg reflector filter (hereinafter referred to as a DBR filter). For example, each of the first to third sub-filters 112, 114, and 116 may comprise alternately stacked first and second refractive index layers. The first and second refractive index layers may have different refractive indices. For example, both the first and second refractive index layers may comprise SiO2 and TiO2. For example, both the first and second refractive index layers may comprise SiO2 and Si3N4. The DBR filter can adjust its light transmission properties according to the thickness of the first and second refractive index layers. The thicknesses of the first and second refractive index layers of the first sub-filter 112, the second sub-filter 114, and the third sub-filter 116 may be different from each other. For example, the thicknesses of the first and second refractive index layers of the first sub-filter 112, the first and second refractive index layers of the second sub-filter 114, and the first and second refractive index layers of the third sub-filter 116 can be determined to have the transmission spectra required by the sub-filters 112, 114, and 116, respectively.

[0102] A microlens ML can be disposed on the multi-filter 110. The microlens ML can receive the incident light IL and collect the incident light IL into the multi-filter 110. For example, the microlens ML can refract the incident light IL.

[0103] This disclosure can provide a hyperspectral element 12 with high resolution and broadband properties.

[0104] Figure 7 yes Figure 6 Transmission spectrum curves of the first sub-filter 112 to the third sub-filter 116. Figure 8 yes Figure 6 The transmission spectrum of the multi-filter 110.

[0105] refer to Figure 7The device can provide a first transmission spectrum DBR1 for a first sub-filter 112, a second transmission spectrum DBR2 for a second sub-filter 114, and a third transmission spectrum DBR3 for a third sub-filter 116. Referring to the first transmission spectrum DBR1, the transmitted wavelength can be approximately 570 nanometers (nm), and the range of use is from approximately 490 nanometers (nm) to approximately 470 nanometers (nm) to approximately 720 nanometers (nm). Referring to the second transmission spectrum DBR2, the transmitted wavelength can be approximately 450 nanometers (nm), and the range of use is from approximately 350 nanometers (nm) to approximately 550 nanometers (nm). Referring to the third transmission spectrum DBR3, the transmitted wavelength can be approximately 760 nanometers (nm), and the range of use is from approximately 600 nanometers (nm) to approximately 900 nanometers (nm).

[0106] refer to Figure 8 The transmission spectrum SS of the multi-filter 110, configured through a combination of the first to third sub-filters 112, 114, and 116, can be provided. The transmission spectrum SS of the multi-filter 110 can be determined based on the transmission spectra DBR1, DBR2, and DBR3 of the first to third sub-filters 112, 114, and 116. For example, for a specific wavelength where the transmittance value in the first transmittance spectrum DBR1 is 0, the transmittance value in the transmission spectrum SS of the multi-filter 110 can be 0 even if the transmittance values ​​in the second and third transmittance spectra DBR2 and DBR3 are not 0.

[0107] The bands used for the first transmission spectrum DBR1, the bands used for the second transmission spectrum DBR2, and the bands used for the third transmission spectrum DBR3 can be successive bands. Therefore, the final usable segment of the multi-filter 110 ranges from approximately 350 nanometers (nm) at the lower limit of the usable segment of the second transmission spectrum DBR2 to approximately 900 nanometers (nm) at the upper limit of the usable segment of the third transmission spectrum DBR3. The transmission wavelengths of the transmission spectrum SS of the multi-filter 110 can be approximately 450 nanometers (nm), approximately 570 nanometers (nm), and approximately 760 nanometers (nm).

[0108] Figure 9 This is a cross-sectional view of the hyperspectral element 14 according to an embodiment. Figures 10 to 13 It is based on Figure 9 A perspective view of the multi-filter 120 of an embodiment of the hyperspectral element 14. For the sake of brevity, the references are omitted. Figure 6 The given descriptions are essentially the same.

[0109] refer to Figure 9 A hyperspectral element 14 can be provided. The hyperspectral element 14 may include a substrate 1, a multi-detector 210, a light-blocking layer ST, a multi-filter 120, a microlens ML, and a spectral processor 300. The substrate 1, multi-detector 210, light-blocking layer ST, microlens ML, and spectral processor 300 can be connected to a reference... Figure 6 The corresponding components described are basically the same.

[0110] The light transmission properties of the multi-filter 120 can be compared with those of the reference. Figures 2 to 5 The light transmission properties of the described multi-filter 100 are substantially the same. The multi-filter 120 may include a first reflective layer RL1, a second reflective layer RL2, and an asymmetric nanostructure layer ANL disposed between the first reflective layer RL1 and the second reflective layer RL2.

[0111] Each of the first reflective layer RL1 and the second reflective layer RL2 can be a DBR layer. For example, the first reflective layer RL1 and the second reflective layer RL2 can be substantially the same DBR layer. The first reflective layer RL1 and the second reflective layer RL2 can have substantially the same light reflection properties.

[0112] The asymmetric nanostructure layer ANL can include multiple nanostructures arranged asymmetrically. The asymmetric nanostructure layer ANL can transmit light with multiple wavelengths in the use segment of the multi-filter 120 defined by the first reflective layer RL1 and the second reflective layer RL2.

[0113] In the following text, reference will be made to Figures 10 to 13 The description includes an example of a multi-filter that includes an asymmetric nanostructure layer (ANL).

[0114] refer to Figure 10 It can provide a multi-filter 120a including a first reflective layer RL1, a second reflective layer RL2, and an asymmetric nanostructure layer ANLa. The asymmetric nanostructure layer ANLa may include a base layer BL, multiple nanorods NR, and a capping layer CL. The base layer BL may be disposed on the first reflective layer RL1. For example, the base layer BL may include any of the following: polymers such as PC, PS, and PMMA, or SiO2.

[0115] Multiple nanorods NR can be asymmetrically arranged on the substrate BL. For example, based on a virtual line extending parallel to the upper surface of the substrate BL but passing through the midpoint of the upper surface of the substrate BL, the arrangement shape and / or number of nanorods NR arranged on both sides of the virtual line can be different from each other. The width of the multiple nanorods NR can be smaller than the wavelength of the incident light incident on the asymmetric nanostructure layer ANL. The distance between the multiple nanorods NR and the width of the multiple nanorods NR can be determined to have the transmission spectrum required by the multiple filters 120a. Although it is disclosed that the multiple nanorods NR have a rectangular column shape, the shape of the multiple nanorods NR is not limited to this. In another example, the multiple nanorods NR can have a circular column shape, a polygonal column shape, a hemispherical shape other than a polygonal column shape, or a polyhedral shape. The multiple nanorods NR can include a material with a refractive index greater than that of the substrate BL. For example, multiple nanorods NR can include any one of monocrystalline silicon, polycrystalline silicon (polycrystalline Si), amorphous silicon (amorphous Si), Si3N4, GaP, TiO2, AlSb, AlAs, AlGaAs, AlGaInP, BP, and ZnGeP2.

[0116] The capping layer CL can be disposed on the base layer BL and the multiple nanorods NR. The capping layer CL can include a material with a refractive index lower than that of the multiple nanorods NR. For example, the capping layer CL can include any of polymers such as PC, PS, and PMMA, and SiO2.

[0117] refer to Figure 11 It can provide a multi-filter 120b including a first reflective layer RL1, a second reflective layer RL2, and an asymmetric nanostructure layer ANLb. The asymmetric nanostructure layer ANLb may include a base layer BL, multiple nanopores NH, and a capping layer CL. (Refer to reference...) Figure 9 Different descriptions exist, but multiple nanopores NH can be disposed within the substrate BL. These nanopores NH can penetrate the substrate BL. For example, the multiple nanopores NH can extend along a direction perpendicular to the upper surface of the substrate BL. The multiple nanopores NH can be arranged asymmetrically. For example, based on a virtual line extending parallel to the upper surface of the substrate BL but passing through the midpoint of the upper surface of the substrate BL, the arrangement shape and / or number of nanopores NH arranged on either side of the virtual line can differ from each other. The width of the multiple nanopores NH can be smaller than the wavelength of incident light incident on the asymmetric nanostructure layer ANLb. The distance between the multiple nanopores NH and the width of the multiple nanopores NH can be determined to have the transmission spectrum required by the multi-filter 120b. Multiple nanopores NH having a cylindrical shape are disclosed, but the shape of the multiple nanopores NH is not limited to this. In another example, the multiple nanopores NH can have a polygonal cylindrical shape, a hemispherical shape other than a polygonal cylindrical shape, or a polyhedral shape.

[0118] The capping layer CL can be set on the base layer BL. The materials of the base layer BL and the capping layer CL can be referenced. Figure 9 The descriptions are essentially the same.

[0119] refer to Figure 12 It can provide a multi-filter 120c including a first reflective layer RL1, a second reflective layer RL2, and an asymmetric nanostructure layer ANLc. The asymmetric nanostructure layer ANLc may include a base layer BL, multiple nanorods NR, multiple nanopores NH, and a capping layer CL. (Refer to reference...) Figure 9 and Figure 10 Unlike those descriptions, multiple nanorods (NR) and multiple nanopores (NH) can be simultaneously arranged. The multiple nanorods (NR) and multiple nanopores (NH) can be arranged asymmetrically. The base layer (BL), multiple nanorods (NR), multiple nanopores (NH), and capping layer (CL) can be compared with the reference... Figure 9 and Figure 10 The descriptions are essentially the same.

[0120] refer to Figure 13 It can provide a multi-filter 120d including a first reflective layer RL1, a second reflective layer RL2, and an asymmetric nanostructure layer ANLd. The asymmetric nanostructure layer ANLd can include a base layer BL, multiple nanostrips NS, and a capping layer CL. The base layer BL and the capping layer CL can be used with a reference... Figure 8 The descriptions are essentially the same. Multiple nanostrips NS can be disposed on the substrate BL. Multiple nanostrips NS can be arranged asymmetrically on the substrate BL. For example, based on a virtual line extending along a direction parallel to the upper surface of the substrate BL but passing through the midpoint of the upper surface of the substrate BL, the arrangement shape and / or number of nanostrips NS arranged on either side of the virtual line can differ from each other. Multiple nanostrips NS can extend in a first direction DR1 parallel to the upper surface of the substrate BL. The width of the multiple nanostrips NS can be smaller than the wavelength of incident light incident on the asymmetric nanostructure layer ANLd. The hyperspectral element 14 of this disclosure can have high resolution.

[0121] Figure 14 yes Figure 9 The transmission spectrum of the multi-filter.

[0122] refer to Figure 14 The transmission spectrum of the multi-filter can have a first transmission wavelength λ1 and a second transmission wavelength λ2. The first transmission wavelength λ1 can be about 580 nanometers (nm), and the second transmission wavelength λ2 can be about 600 nanometers (nm). The range of the transmission spectrum of the multi-filter can be from about 490 nanometers (nm) to about 680 nanometers (nm).

[0123] Figure 15This is a cross-sectional view of the hyperspectral element 16 according to an embodiment. For the sake of brevity, references may be omitted. Figure 6 and Figure 9 The descriptions are essentially the same.

[0124] refer to Figure 15 A hyperspectral element 16 can be provided. The hyperspectral element 16 may include a substrate 1, a multi-detector 210, a light-blocking layer ST, a multi-filter 130, a microlens ML, and a spectral processor 300. The substrate 1, multi-detector 210, light-blocking layer ST, microlens ML, and spectral processor 300 can be connected to a reference... Figure 6 The corresponding components described are basically the same.

[0125] The multi-filter 130 may include a first sub-filter 132, a second sub-filter 134, and a third sub-filter 136. Each of the first to third sub-filters 132, 134, and 136 may be referenced. Figure 9 The described multi-filter system, comprising an asymmetric nanostructure layer ANL, is substantially identical. The first to third sub-filters 132, 134, and 136 can be arranged in series. The first to third sub-filters 132, 134, and 136 can have different heights. For example, the farther a sub-filter 132, 134, or 136 is positioned from the microlens ML, the higher its height. The first to third sub-filters 132, 134, and 136 can have different transmission spectra. For example, the transmission spectrum of the first sub-filter 132 may include a reference spectrum. Figure 4 The first sub-filter 112 described includes the first use segment UB1, which also includes, for example... Figure 14 The first segment UB1 shown contains two transmission wavelengths. The transmission spectrum of the second sub-filter 134 may include a reference wavelength. Figure 4 The second sub-filter 114 described includes the second usage segment UB2, which also includes, for example... Figure 14 The second used segment UB2 is shown to have two transmitted wavelengths. The transmission spectrum of the third sub-filter 136 may include: reference... Figure 4 The third sub-filter 116 described includes the third use segment UB3, which also includes, for example... Figure 14 The third segment UB3 shown contains two wavelengths.

[0126] The hyperspectral element 16 disclosed herein can have broadband and high-resolution properties.

[0127] Figure 16 This is a cross-sectional view of the hyperspectral element 18 according to an embodiment. For the sake of brevity, references may be omitted. Figure 6 and Figure 15 The descriptions are essentially the same.

[0128] refer to Figure 16 A hyperspectral element 18 can be provided. The hyperspectral element 18 may include a substrate 1, a multi-detector 220, a light-blocking layer ST, a multi-filter 130, a microlens ML, and a spectral processor 300. The substrate 1, the light-blocking layer ST, the microlens ML, and the spectral processor 300 can be connected to a reference... Figure 6 The substrate 1, light-blocking layer ST, microlens ML, and spectral processor 300 described respectively are essentially the same. The multi-filter 130 can be compared with the reference... Figure 15 The multi-filter 130 described is essentially the same.

[0129] Reference Figure 6 Depending on the description, the multi-detector 220 may also include an ultraviolet detector UD and an infrared detector IRD. For example, the ultraviolet detector UD may be disposed between the stacked structure of p-layer PL and n-layer NL and the multi-filter 110, and the infrared detector IRD may be disposed between the stacked structure of p-layer PL and n-layer NL and the substrate 1.

[0130] This disclosure can measure light in the visible, ultraviolet and infrared regions, and provides a hyperspectral element 18 with broadband and high-resolution properties.

[0131] Figure 17 This is a block diagram of the hyperspectral sensor 20 according to an embodiment. Figure 18 yes Figure 17 A conceptual block diagram of pixels. Figure 19 It shows the use Figure 17 A flowchart of a method for measuring light using a hyperspectral sensor 20. Figure 20 Show Figure 18 The transmission spectrum curves of the multiple filters are shown. For simplicity, references are omitted. Figures 2 to 5 The descriptions are essentially the same.

[0132] refer to Figures 17 to 20 A hyperspectral sensor 20 can be provided. The hyperspectral sensor 20 may include a pixel array PA, a spectral processor 300, and a main processor 1000. The pixel array PA may include multiple pixels PX. The multiple pixels PX can be arranged in the pixel array PA. The multiple pixels PX can be substantially identical to each other.

[0133] like Figure 18As shown, each of the plurality of pixels PX may include a plurality of subpixels SPX1, SPX2, SPX3, and SPX4. For example, the plurality of subpixels SPX1, SPX2, SPX3, and SPX4 may include a first subpixel SPX1, a second subpixel SPX2, a third subpixel SPX3, and a fourth subpixel SPX4. However, the number of subpixels SPX1, SPX2, SPX3, and SPX4 is not limited. In another example, a pixel PX may include fewer than or more than four subpixels.

[0134] A first hyperspectral element SS1, a second hyperspectral element SS2, a third hyperspectral element SS3, and a fourth hyperspectral element SS4 can be configured, respectively used for the first to fourth sub-pixels SPX1, SPX2, SPX3, and SPX4. The first to fourth hyperspectral elements SS1, SS2, SS3, and SS4 can each include a first multi-detector MD1, a second multi-detector MD2, a third multi-detector MD3, and a fourth multi-detector MD4. Each of the first to fourth multi-detectors MD1, MD2, MD3, and MD4 can correspond to a reference. Figures 2 to 5 Description Figure 2 The multi-detector 200. The first to fourth multi-detectors MD1, MD2, MD3, and MD4 can be substantially identical to each other. The first to fourth multi-detectors MD1, MD2, MD3, and MD4 can have the same optical detection properties. For example, each of the first to fourth multi-detectors MD1, MD2, MD3, and MD4 can be compared with a reference... Figure 2 The described multi-detector 200 is substantially the same. Each of the first to fourth multi-detectors MD1, MD2, MD3, and MD4 may include: a first sub-detector for measuring light having a wavelength in a first optical detection segment DB1, a second sub-detector for measuring light having a wavelength in a second optical detection segment DB2, and a third sub-detector for measuring light having a wavelength in a third optical detection segment DB3. The first to third optical detection segments DB1, DB2, and DB3 may be referenced. Figures 2 to 5 The descriptions are essentially the same.

[0135] The first to fourth hyperspectral elements SS1, SS2, SS3, and SS4 may each include a first multi-filter MF1, a second multi-filter MF2, a third multi-filter MF3, and a fourth multi-filter MF4. The first to fourth multi-filters MF1, MF2, MF3, and MF4 may be substantially identical to each other. The first to fourth multi-filters MF1, MF2, MF3, and MF4 may have different light transmission properties. For example, each of the first to fourth multi-filters MF1, MF2, MF3, and MF4 may include a reference. Figure 2The three sub-filters described are: the first multi-filter to the fourth multi-filter MF1, MF2, MF3, and MF4, which can filter the incident light IL to generate a filtered first light FL1, a filtered second light FL2, a filtered third light FL3, and a filtered fourth light FL4, respectively (operation S210). The filtered first light to the filtered fourth light FL1, FL2, FL3, and FL4 can have different spectra. The light transmission properties of the first multi-filter to the fourth multi-filter MF1, MF2, MF3, and MF4 are described below.

[0136] Figure 20 Curves T(1) to T(4) can represent the transmission spectra of the first to fourth multi-filters MF1, MF2, MF3 and MF4, respectively. Referring to curve T(1), light with the final passing wavelength TPB1a (1a), light with the final passing wavelength TPB1b (1b), light with the final passing wavelength TPB1c (1c), light with wavelengths in the first sub-transmittance region TSTR1a (1a), and light with wavelengths in the first sub-transmittance region TSTR1b (1b) can pass through the first multi-filter MF1.

[0137] Referring to curve T(2), light with the final passing wavelength TPB2a of the 2ath, light with the final passing wavelength TPB2b of the 2bth, light with the final passing wavelength TPB2c of the 2cth, light with the wavelength in the 2ath subtransmittance region TSTR2a, and light with the wavelength in the 2bth subtransmittance region TSTR2b can pass through the second multi-filter MF2.

[0138] Referring to curve T(3), light with the final passing wavelength TPB3a of the 3a, light with the final passing wavelength TPB3b of the 3b, light with the final passing wavelength TPB3c of the 3c, light with the wavelength in the 3a sub-transmittance region TSTR3a, and light with the wavelength in the 3b sub-transmittance region TSTR3b can pass through the third multi-filter MF3.

[0139] Referring to curve T(4), light with the final passing wavelength TPB4a of the 4a, light with the final passing wavelength TPB4b of the 4b, light with the final passing wavelength TPB4c of the 4c, light with the wavelength in the 4a sub-transmittance region TSTR4a, and light with the wavelength in the 4b sub-transmittance region TSTR4b can pass through the fourth multi-filter MF4.

[0140] The final passing wavelengths TPB1a (1a), TPB2a (2a), TPB3a (3a), and TPB4a (4a), the final passing wavelengths TPB1b (1b), TPB2b (2b), TPB3b (3b), and TPB4b (4b), the final passing wavelengths TPB1c (1c), TPB2c (2c), TPB3c (3c), and TPB4c (4c) can be different wavelengths located in the first optical detection segment DB1. The final passing wavelengths TPB1b (1b), TPB2b (2b), TPB3b (3b), and TPB4b (4b), the final passing wavelengths TPB1c (1c), TPB2c (2c), TPB3c (3c), and TPB4c (4c) can be different wavelengths located in the second optical detection segment DB2. The final passing wavelengths TPB1c (1c), TPB2c (2c), TPB3c (3c), and TPB4c (4c) can be different wavelengths located in the third optical detection segment DB3.

[0141] The first to fourth multi-detectors MD1, MD2, MD3, and MD4 can detect filtered light to generate channel signals (operation S220). The first sub-detector MD1a of the first multi-detector MD1 can detect the final passing wavelengths from 1a to 1c (TBP1a, TPB1b, and TPB1c) to generate the 1a channel signal. The first sub-detector MD1a of the first multi-detector MD1 can provide the 1a channel signal to the spectral processor 300. The second sub-detector MD1b of the first multi-detector MD1 can detect the final passing wavelengths TPB1b and TPB1c (1b and 1c) to generate the 1b channel signal. The second sub-detector MD1b of the first multi-detector MD1 can provide the 1b channel signal to the spectral processor 300. The third sub-detector MD1c of the first multi-detector MD1 can sense the final passing wavelength TBP1c (1c) to generate the 1c channel signal. The third sub-detector MD1c of the first multi-detector MD1 can provide the 1c channel signal to the spectral processor 300.

[0142] The first sub-detector MD2a of the second multi-detector MD2 can detect wavelengths from 2a to 2c (final passing wavelengths TPB2a, TPB2b, and TPB2c) to generate the 2a channel signal. The first sub-detector MD2a of the second multi-detector MD2 can provide the 2a channel signal to the spectral processor 300. The second sub-detector MD2b of the second multi-detector MD2 can detect wavelengths 2b and 2c (final passing wavelengths TPB2b and TPB2c) to generate the 2b channel signal. The second sub-detector MD2b of the second multi-detector MD2 can provide the 2b channel signal to the spectral processor 300. The third sub-detector MD2c of the second multi-detector MD2 can detect wavelength 2c (final passing wavelength TPB2c) to generate the 2c channel signal. The third sub-detector MD2c of the second multi-detector MD2 can provide the 2c channel signal to the spectral processor 300.

[0143] The first sub-detector MD3a of the third multi-detector MD3 can detect wavelengths from 3a to 3c (final passing wavelengths TPB3a, TPB3b, and TPB3c) to generate the 3a channel signal. The first sub-detector MD3a of the third multi-detector MD3 can provide the 3a channel signal to the spectral processor 300. The second sub-detector MD3b of the third multi-detector MD3 can detect wavelengths 3b and 3c (final passing wavelengths) to generate the 3b channel signal. The second sub-detector MD3b of the third multi-detector MD3 can provide the 3b channel signal to the spectral processor 300. The third sub-detector MD3c of the third multi-detector MD3 can detect wavelength 3c (final passing wavelength TPB3c) to generate the 3c channel signal. The third sub-detector MD3c of the third multi-detector MD3 can provide the 3c channel signal to the spectral processor 300.

[0144] The first sub-detector MD4a of the fourth multi-detector MD4 can detect the final passing wavelengths from 4a to 4c (TPB4a, TPB4b, and TPB4c) to generate the 4a channel signal. The first sub-detector MD4a of the fourth multi-detector MD4 can provide the 4a channel signal to the spectral processor 300. The second sub-detector MD4b of the fourth multi-detector MD4 can detect the final passing wavelengths TPB4b and TPB4c of the fourth multi-detector MD4 to generate the 4b channel signal. The second sub-detector MD4b of the fourth multi-detector MD4 can provide the 4b channel signal to the spectral processor 300. The third sub-detector MD4c of the fourth multi-detector MD4 can sense the final passing wavelength TPB4c of the fourth multi-detector MD4 to generate the 4c ​​channel signal. The third sub-detector MD4c of the fourth multi-detector MD4 can provide the 4c ​​channel signal to the spectral processor 300.

[0145] The spectral processor 300 can generate information related to the intensity of light having final passing wavelengths from channel 1a to channel 4c, namely TBP1a, TBP1b, TBP1c, TBP2a, TBP2b, TBP2c, TBP3a, TBP3b, TBP3c, TBP4a, TBP4b, and TBP4c (operation S230). The spectral processor 300 can provide the main processor 1000 with the information related to the intensity of light having final passing wavelengths from channel 1a to channel 4c, namely TBP1a, TBP1b, TBP1c, TBP2a, TBP2b, TBP2c, TBP3a, TBP3b, TBP3c, TBP4a, TBP4b, and TBP4c.

[0146] The main processor 1000 can generate hyperspectral information based on information related to light intensity. For example, the main processor 1000 can generate hyperspectral image information.

[0147] Light with three different wavelengths can be detected by sub-pixels SPX1 through SPX2, SPX3, and SPX4, respectively. Light with wavelengths from twelve bands can be detected by a single pixel PX, which includes sub-pixels SPX1 through SPX4.

[0148] When a subpixel detects light with only one wavelength, n subpixels are needed to detect light with n wavelengths. In scenarios where hyperspectral sensors need to detect light with as many wavelengths as possible, as the number of subpixels increases, the area of ​​each subpixel decreases, and therefore, the resolution may decrease.

[0149] This disclosure provides a hyperspectral sensor 20 that includes subpixels that detect multiple beams of light having multiple wavelengths. Because this disclosure uses fewer subpixels than the number of wavelengths to be detected, it is possible to provide subpixels with a larger area than a single subpixel that only detects light having one wavelength. Therefore, a hyperspectral sensor 20 with high resolution can be provided.

[0150] This disclosure provides a hyperspectral sensor 20 with broadband and high-resolution properties.

[0151] Figure 21 It is based on Figure 17 A cross-sectional view of the pixels of an embodiment of the hyperspectral sensor 20. For the sake of brevity, the references are omitted. Figure 6 as well as Figures 17 to 20 The descriptions are essentially the same.

[0152] refer to Figure 21This can provide a pixel PX for the hyperspectral sensor 20. The pixel PX may include: a substrate 1; first to fourth multi-detectors MD11, MD12, MD13, and MD14; a light-blocking layer ST; first to fourth multi-filters MF11, MF12, MF13, and MF14; and first to fourth microlenses ML. The substrate 1 can be connected to a reference... Figure 6 The substrates described are essentially the same.

[0153] The first to fourth multi-detectors MD11, MD12, MD13, and MD14 can be disposed on the substrate 1. Each of the first to fourth multi-detectors MD11, MD12, MD13, and MD14 can be connected to a reference. Figure 6 The described multi-detector 200 is essentially the same. The optical detection properties of the first to fourth multi-detectors MD11, MD12, MD13, and MD14 can be compared with those in the reference. Figures 18 to 20 The optical detection properties of the first to fourth multi-detectors MD11, MD12, MD13 and MD14, which are described respectively, are basically the same.

[0154] A light-blocking layer ST can be disposed on the side surfaces of the first to fourth multi-detectors MD11, MD12, MD13, and MD14. The light-blocking layer ST can isolate the first to fourth multi-detectors MD11, MD12, MD13, and MD14 from each other, thus preventing them from detecting unwanted light.

[0155] The first to fourth multi-filters MF11, MF12, MF13 and MF14 can be respectively set on the first to fourth multi-detectors MD11, MD12, MD13 and MD14. Figure 20 Each of the first to fourth multi-filters MF11, MF12, MF13 and MF14 shown can have the same characteristics as the reference. Figure 14 The described multi-filters have essentially the same structure, but this is not a limitation. For example, the first to fourth multi-filters MF11, MF12, MF13, and MF14 can have different heights, and the asymmetric nanostructure layers ANL of the first to fourth multi-filters MF11, MF12, MF13, and MF14 can have different heights. In another example, each of the first to fourth multi-filters MF11, MF12, MF13, and MF14 can be... Figure 5 Multi-filter 110 or Figure 8The multi-filter 120. The light transmission properties of the first to fourth multi-filters MF11, MF12, MF13 and MF14 can be compared with those of the reference filter. Figures 16 to 19 The light transmission properties of the first to fourth multi-filters MF1, MF2, MF3 and MF4 are basically the same.

[0156] The planarization layer 400 can be disposed on the first to fourth multi-filters MF11, MF12, MF13, and MF14. The upper surface of the planarization layer 400 can be parallel to the upper surface of the substrate 1. For example, the planarization layer 400 can include SiO2.

[0157] The first to fourth microlenses ML11, ML12, ML13, and ML14 can be disposed on the planarization layer 400. The first to fourth microlenses ML11, ML12, ML13, and ML14 can be arranged to correspond to the first to fourth multi-filters MF11, MF12, MF13, and MF14, respectively. The first to fourth microlenses ML11, ML12, ML13, and ML14 can collect light into the first to fourth multi-filters MF11, MF12, MF13, and MF14, respectively.

[0158] This disclosure can provide a pixel PX including multiple sub-pixels SPX1, SPX2, SPX3 and SPX4 that detect light having multiple wavelengths.

[0159] Figure 22 This is a block diagram of a hyperspectral image generation apparatus 30 including a hyperspectral sensor according to an embodiment. For the sake of brevity, it may be omitted from the reference. Figures 17 to 21 The descriptions are essentially the same.

[0160] refer to Figure 22 A hyperspectral image generating apparatus 30 can be provided. For example, the hyperspectral image generating apparatus 30 can be an image sensor. The hyperspectral image generating apparatus 30 may include a pixel array PA, a spectral processor 300, a main processor 1000, a memory 2000, and a display 3000. The pixel array PA and the spectral processor 300 may be referred to as a hyperspectral sensor unit 22. The pixel array PA and the spectral processor 300 can be respectively connected to a reference... Figures 16 to 20The pixel array PA and the spectral processor 300 described are essentially the same. The pixel array PA can receive incident light provided by the measurement target. The spectral processor 300 can generate information related to the position of the measurement target and the intensity of light at each wavelength. This information related to the position of the measurement target and the intensity of light at each wavelength can be referred to as hyperspectral information. The spectral processor 300 can provide the hyperspectral information to the main processor 1000. The main processor 1000 can generate hyperspectral image information based on the hyperspectral information obtained from the spectral processor 300. The main processor 300 provides the hyperspectral image information to the display 2000.

[0161] The display 2000 can output various types of information, including hyperspectral image information. For example, the display 2000 can display hyperspectral image information, the operating status of the hyperspectral image generating device 30, and a user interface window for selecting functions and options chosen by the user.

[0162] The memory 3000 stores various data used for the operation of the hyperspectral image generating apparatus 30, such as programs used to process or control the main processor 1000. For example, the memory 3000 may store multiple application programs driven by the hyperspectral image generating apparatus 30, as well as data and commands used for the operation of the hyperspectral image generating apparatus 30. The memory 3000 can be accessed by the main processor 1000, and the main processor 1000 can perform data reading / writing / modification / deletion / updating. The memory 3000 can be implemented not only as a storage medium within the hyperspectral image generating apparatus 30, but also as an external storage medium, including a removable disk such as a USB flash drive, and a web server using a network.

[0163] This disclosure provides a hyperspectral image generation apparatus 30 including a hyperspectral sensor unit 22 with high optical resolution and a wide operating range.

[0164] Figure 23 This is a block diagram illustrating an example of an electronic device ED01 that includes an image sensor.

[0165] refer to Figure 23In the network environment ED00, electronic device ED01 can communicate with another electronic device ED02 via a first network ED98 (short-range wireless communication network, etc.), or with another electronic device ED04 and / or server ED08 via a second network ED99 (long-range wireless communication network, etc.). Electronic device ED01 can communicate with electronic device ED04 via server ED08. Electronic device ED01 may include a processor ED20, a memory ED30, an input device ED50, a sound output device ED55, a display device ED60, an audio module ED70, a sensor module ED76, an interface ED77, a haptic module ED79, a camera module ED80, a power management module ED88, a battery ED89, a communication module ED90, a user identification module ED96, and / or an antenna module ED97. Electronic device ED01 may omit some components (such as the display device ED60), or may include other components. One or more of the components may be implemented as integrated circuits. For example, the sensor module ED76 (fingerprint sensor, iris sensor, illuminance sensor, etc.) can be embedded in the display device ED60 (display, etc.). Furthermore, when the image sensor 4000 includes spectral functionality, some functions of the sensor module ED76 (color sensor and illuminance sensor) can be implemented in the image sensor 4000 itself rather than in a separate sensor module.

[0166] Processor ED20 can be configured to execute software (program ED40, etc.) to control one or more components (hardware or software components) of electronic device ED01, components connected to processor ED20, and perform various data processing or calculations. As part of the data processing or calculations, processor ED20 can be configured to load commands and / or data received from other components (sensor module ED76, communication module ED90, etc.) into volatile memory ED32, process the commands and / or data stored in volatile memory ED32, and store the result data in non-volatile memory ED34. Processor ED20 may include a main processor ED21 (central processing unit (CPU), application processor (AP), etc.) and an auxiliary processor ED23 (graphics processing unit (GPU), image signal processor, sensor hub processor, communication processor, etc.) that can operate independently of or with the main processor ED21. The auxiliary processor ED23 can use less power than the main processor ED21 and can perform specified functions.

[0167] When the main processor ED21 is inactive (sleep state), the auxiliary processor ED23 can be responsible for controlling the operation of functions and / or states related to one or more components of the electronic device ED01 (display device ED60, sensor module ED76, communication module ED90, etc.). Alternatively, when the main processor ED21 is active (application execution state), the auxiliary processor ED23 can perform unified operations together with the main processor ED21. The auxiliary processor ED23 (image signal processor, communication processor, etc.) can be implemented as part of other functionally related components (camera module ED80, communication module ED90, etc.).

[0168] The memory ED30 can store various data required by the components of the electronic device ED01 (processor ED20, sensor module ED76, etc.). This data may include, for example, software (program ED40, etc.) and input and / or output data of software-related commands. The memory ED30 may include volatile memory ED32 and / or non-volatile memory ED34. The non-volatile memory ED32 may include internal memory ED36 fixedly installed in the electronic device ED01 and removable external memory ED38.

[0169] The program ED40 can be stored as software in the memory ED30, and may include the operating system ED42, middleware ED44, and / or application ED46.

[0170] Input device ED50 can receive commands and / or data from outside electronic device ED01 for use by components of electronic device ED01 (processor ED20, etc.). Input device ED50 may include a microphone, mouse, keyboard, and / or digital pen (stylus, etc.).

[0171] The sound output device ED55 can output sound signals to the external device ED01. The sound output device ED55 may include a speaker and / or a handset. The speaker can be used for general purposes such as multimedia playback or recording, and the handset can be used to receive incoming calls. The handset can be coupled to the speaker as part of the speaker or can be implemented as a stand-alone device.

[0172] Display device ED60 can visually provide information to the user of electronic device ED01. Display device ED60 may include a display, a holographic device, or control circuitry for controlling a projector and related devices. Display device ED60 may include touch circuitry configured to detect touch operations and / or sensor circuitry (pressure sensor, etc.) configured to measure the intensity of the force generated by the touch operation.

[0173] The audio module ED70 can convert sound into electrical signals, or vice versa. The audio module ED70 can acquire sound via the input device ED50, or output sound via the sound output device ED55 and / or the speakers and / or headphones of the electronic device (electronic device ED02, etc.) directly or wirelessly connected to the electronic device ED01.

[0174] The sensor module ED76 can detect the operating status (power, temperature, etc.) or external environmental status (user status, etc.) of the electronic device ED01, and can generate electrical signals and / or data values ​​corresponding to the detected status. The sensor module ED76 may include gesture sensors, gyroscope sensors, atmospheric sensors, magnetic sensors, accelerometers, grip sensors, proximity sensors, color sensors, infrared (IR) sensors, biometric sensors, temperature sensors, humidity sensors, and / or illuminance sensors.

[0175] Interface ED77 may support one or more specified protocols for electronic device ED01 to connect directly or wirelessly to another electronic device (electronic device ED02, etc.). Interface ED77 may include a High Definition Multimedia Interface (HDMI) interface, a Universal Serial Bus (USB) interface, an SD card interface, and / or an audio interface.

[0176] The connection terminal ED78 may include a connector through which electronic device ED01 can be physically connected to another electronic device (electronic device ED02, etc.). The connection terminal ED78 may include an HDMI connector, a USB connector, an SD card connector, and / or an audio connector (headphone connector, etc.).

[0177] The haptic module ED79 can convert electrical signals into mechanical stimuli (vibration, motion, etc.) or electrical stimuli that can be recognized by the user through touch or motion sensation. The haptic module ED79 may include a motor, piezoelectric devices, and / or electrical stimulation devices.

[0178] The ED80 camera module can capture still images and video. The ED80 camera module may include: a lens assembly comprising one or more lenses; Figure 1 The camera module ED80 includes an image sensor 1000, an image signal processor, and / or a flash. The lens assembly included in the camera module ED80 can collect light emitted from the target whose image is to be captured.

[0179] The power management module EDg8 manages the power supplied to the electronic device ED01. The power management module EDg8 can be implemented as part of a power management integrated circuit (PMIC).

[0180] Battery ED89 can power components of electronic device ED01. Battery ED89 may include a non-rechargeable main battery, a rechargeable dual battery, and / or a fuel cell.

[0181] Communication module ED90 can support the establishment of (wired) and / or wireless communication channels between electronic device ED01 and other electronic devices (electronic device ED02, electronic device ED04, server ED08, etc.), and perform communication through the established communication channels. Communication module ED90 may include one or more communication processors, which operate independently of processor ED20 (application processor, etc.) and support direct and / or wireless communication. Communication module ED90 may include wireless communication module ED92 (cellular communication module, short-range wireless communication module, Global Navigation Satellite System (GNSS) communication module, etc.) and / or wired communication module ED94 (local area network (LAN) communication module, power line communication module, etc.). Corresponding communication modules among these can communicate with other electronic devices through a first network ED98 (a short-range wireless communication network such as Bluetooth, WiFi Direct, or Infrared Data Association (IrDA)) or a second network ED99 (a long-range communication network such as a cellular network, the Internet, or computer network (LAN, WAN, etc.). The various types of communication modules described above can be integrated into a single component (single chip, etc.) or implemented as multiple components (multiple chips). The wireless communication module ED92 can identify and authenticate the electronic device ED01 in the first network ED98 and / or the second network ED99 by using user information (International Mobile Subscriber Identity (IMSI) etc.) stored in the user identification module ED96.

[0182] Antenna module ED97 can transmit signals and / or power to or from external sources (other electronic devices, etc.). The antenna may include a transmitter comprising conductive patterns formed on a substrate (printed circuit board (PCB), etc.). Antenna module ED97 may include one or more antennas. When antenna module ED97 includes multiple antennas, appropriate antennas suitable for communication methods used in communication networks such as first network ED98 and / or second network ED99 can be selected. Through the selected antennas, signals and / or power can be transmitted or received between communication module ED90 and other electronic devices. In addition to the antennas, another component (radio frequency integrated circuit (RFIC), etc.) may be included in antenna module ED97.

[0183] One or more components of the electronic device ED01 can connect to each other and exchange signals (commands, data, etc.) through communication methods (bus, general purpose input and output (GPIO), serial peripheral bus (SPI), mobile industrial processor interface (MIPI), etc.) executed in peripheral devices.

[0184] Commands or data can be sent or received between electronic device ED01 and another external electronic device ED04 via server ED08 connected to the second network ED99. Other electronic devices ED02 and ED04 can be of the same or different type as electronic device ED01. All or some operations performed in electronic device ED01 can be performed by one or more of the other electronic devices ED02, ED04, and ED08. For example, when electronic device ED01 must perform a function or service, instead of directly performing that function or service, it can request one or more other electronic devices to perform part or all of that function or service. The one or more electronic devices receiving the request can perform additional functions or services related to the request and can send the results of the execution back to electronic device ED01. For this purpose, cloud computing, distributed computing, and / or client-server computing technologies can be used.

[0185] Figure 24 It is shown Figure 23 A schematic block diagram of the ED80 camera module.

[0186] refer to Figure 24 The camera module ED80 may include a lens assembly CM10, a flash CM20, and an image sensor 4000. Figure 1 The camera module ED80 may include an image stabilizer CM40, a memory CM50 (buffer memory, etc.), and / or an image signal processor CM60. A lens assembly CM10 can collect light emitted from the subject, which is the target of image capture. The camera module ED80 may include multiple lens assemblies CM10, and in this case, the camera module ED80 may include a dual-camera system, a 360-degree camera, or a spherical camera. Some of the multiple lens assemblies CM10 may have the same lens properties (angle of view, focal length, AF, F-number, optical zoom, etc.) or may have different lens properties. The lens assembly CM10 may include a wide-angle lens or a telephoto lens.

[0187] The flash CM20 can emit light to enhance light emitted or reflected from the subject. The flash CM20 may include one or more light-emitting diodes (red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, ultraviolet LEDs, etc.) and / or a xenon lamp. The image sensor 1000 may be... Figure 1The image sensor 1000 described herein can acquire an image corresponding to a subject by converting light emitted or reflected from a subject and transmitted through the lens assembly CM10 into an electrical signal. The image sensor 1000 may include one or more sensors selected from image sensors with different properties (e.g., RGB sensors, black-and-white (BW) sensors, IR sensors, or UV sensors). Each sensor included in the image sensor 1000 may be implemented as a charge-coupled device (CCD) sensor and / or a complementary metal-oxide-semiconductor (CMOS) sensor.

[0188] Image stabilizer CM40 can respond to movement of camera module ED80 or electronic device ED01 including camera module ED80 by moving lens assembly CM10 or one or more lenses included in image sensor 4000 in a specific direction, or by controlling the operating characteristics of image sensor 4000 (adjusting readout timing, etc.) to compensate for negative effects caused by movement. Image stabilizer CM40 can detect movement of camera module ED80 or electronic device ED01 using a gyroscope sensor (not shown) or accelerometer sensor (not shown) disposed inside or outside camera module ED80. Image stabilizer CM40 can be implemented in an optical form.

[0189] The memory CM50 can store part or all of the image data acquired by the image sensor 4000 for subsequent image processing operations. For example, when multiple images are acquired at high speed, the acquired raw data (Bayer pattern data, high-resolution data, etc.) can be stored in the memory CM50, and only the low-resolution image can be displayed. Then, the raw data of the selected image (user-selected, etc.) can be sent to the image signal processor CM60. The memory CM50 can be integrated into the memory ED30 of the electronic device ED01 or can be configured as a separate memory that operates independently.

[0190] The image signal processor CM60 can perform image processing operations on images acquired by the image sensor 4000 or image data stored in the memory CM50. The image processing may include depth map generation, 3D modeling, panorama generation, feature point extraction, image compositing, and / or image compensation (noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, softening, etc.). The image signal processor CM60 can perform control (exposure time control or readout timing control, etc.) on components included in the camera module ED80 (image sensor 4000, etc.). Images processed by the image signal processor CM60 can be stored again in the memory CM50 for further processing, or can be provided to external components of the camera module ED80 (memory ED30, display device ED60, electronic device ED02, electronic device ED04, server ED08, etc.). The image signal processor CM60 can be integrated into the processor ED20 or configured as a separate processor operating independently of the processor ED20. When the image signal processor CM60 is configured as a processor separate from the processor ED20, the image processed by the image signal processor CM60 can undergo additional image processing by the processor ED20 and then be displayed by the display device ED60.

[0191] Electronic device ED01 may include multiple camera modules ED80 with different attributes or functions. In this case, one of the multiple camera modules ED80 may be a wide-angle camera, and the other camera modules may be telephoto cameras. Similarly, one of the multiple camera modules ED80 may be a front-facing camera, and the other camera modules may be rear-facing cameras.

[0192] Figures 25 to 34 Various examples of electronic devices using an image sensor 4000 according to embodiments are shown.

[0193] The image sensor 4000 according to the embodiment can be applied to Figure 25 The mobile phone or smartphone shown is 5100m. Figure 26 The tablet computer or smart tablet computer 5200 shown is shown. Figure 27 The digital camera or video recorder 5300 shown Figure 28 The laptop computer shown is 5400 or Figure 29 The television or smart TV 5500 shown is an example. For instance, a smartphone 5100m or a smart tablet computer 5200 may include multiple high-resolution cameras, each including a high-resolution image sensor. By using high-resolution cameras, depth information of targets in an image can be extracted, image defocusing can be adjusted, or objects in an image can be automatically identified.

[0194] In addition, the image sensor 4000 can be applied to Figure 30 The smart refrigerator 5600 shown Figure 31 The surveillance camera 5700 shown Figure 32 The robot 5800 shown Figure 33 Examples include the medical camera 5900. For instance, the smart refrigerator 5600 can automatically identify food items in the refrigerator using an image sensor and can notify the user via a smartphone about the presence of specific types of food, the types of food placed or removed, etc. Furthermore, the surveillance camera 5700 can provide ultra-high-resolution images using high sensitivity, allowing users to identify objects or people even in dark environments. The robot 5800 can enter disaster or industrial locations inaccessible to humans to provide users with high-resolution images. The medical camera 5900 can provide high-resolution images for diagnosis or surgery and can dynamically adjust its field of view.

[0195] In addition, the image sensor 4000 can be applied to Figure 34 The vehicle 6000 is shown. The vehicle 6000 may include a plurality of vehicle cameras 6010, 6020, 6030, and 6040 arranged in various locations. Each of the vehicle cameras 6010, 6020, 6030, and 6040 may include an image sensor according to an embodiment. The vehicle 6000 can provide the driver with various information about the interior of the vehicle 6000 or the surroundings of the vehicle 6000 by using the plurality of vehicle cameras 6010, 6020, 6030, and 6040, and can provide the driver with information necessary for autonomous driving by automatically recognizing objects or people in the images.

[0196] This disclosure can provide hyperspectral elements with high resolution and broadband characteristics.

[0197] This disclosure can provide a hyperspectral sensor with high resolution and broadband characteristics.

[0198] This disclosure provides a hyperspectral image generation apparatus with high resolution and broadband characteristics.

[0199] However, the effects of the embodiments are not limited to those disclosed above.

[0200] The exemplary embodiments described above are merely examples and should not be construed as limiting. This teaching can be readily applied to other types of devices. Furthermore, the description of the exemplary embodiments is intended to be illustrative and not to limit the scope of the claims, and various alternatives, modifications, and variations will be apparent to those skilled in the art.

Claims

1. A hyperspectral element, comprising: Multiple filters, including: The first sub-filter allows light of the first wavelength to pass through, and A second sub-filter allows light of a second wavelength, different from the first wavelength, to pass through; and Multiple detectors are configured to detect the first wavelength light and the second wavelength light. The first sub-filter and the second sub-filter are arranged in series in the optical path of the incident light incident on the multiple filters. The transmission spectrum of the first sub-filter includes a first usable segment and a first sub-transmittance region outside the first usable segment. The transmission spectrum of the second sub-filter includes a second usable segment and a second sub-transmittance region outside the second usable segment. The first wavelength is included in both the first usage segment and the second sub-transmitting region. Wherein, the second wavelength is included in the second usage segment and the first sub-transmitting region, and The first sub-filter is configured to filter out a first remaining wavelength other than the first wavelength from the first usage segment, and the second sub-filter is configured to filter out a second remaining wavelength other than the second wavelength from the second usage segment.

2. The hyperspectral element according to claim 1, wherein, The first usage segment and the second usage segment partially overlap.

3. The hyperspectral element according to claim 1, wherein, The multiple detectors include: The first sub-detector includes a first optical detection segment overlapping the first wavelength and the second wavelength; and The second sub-detector includes a second optical detection segment that overlaps with either the first wavelength or the second wavelength, and The first sub-detector and the second sub-detector are arranged in series.

4. The hyperspectral element according to claim 3, further comprising: Spectrum processor; The first sub-detector is configured to generate a first channel signal relating to the first wavelength light and the second wavelength light, and to provide the first channel signal to the spectral processor. The second sub-detector is configured to generate a second channel signal for the second wavelength of light and provide the second channel signal to the spectral processor. The spectral processor is configured to generate information related to the intensity of the first wavelength light and the second wavelength light based on the first channel signal and the second channel signal.

5. The hyperspectral element according to claim 3, wherein, The multi-detector system also includes an ultraviolet detector, and The ultraviolet detector is arranged in series with the first sub-detector and the second sub-detector in the optical path of the incident light.

6. The hyperspectral element according to claim 3, wherein, The multi-detector also includes an infrared detector, and The infrared detector is arranged in series with the first sub-detector and the second sub-detector in the optical path of the incident light.

7. The hyperspectral element according to claim 1, wherein, Each of the first sub-filter and the second sub-filter includes: alternately stacked first and second refractive index films, and The first refractive index film has a different refractive index than the second refractive index film.

8. The hyperspectral element according to claim 7, wherein, The first refractive index film of the first sub-filter and the first refractive index film of the second sub-filter have different thicknesses.

9. The hyperspectral element according to claim 1, wherein, Each of the first sub-filter and the second sub-filter includes: First reflective layer; Second reflective layer; and A nanostructure layer is disposed between the first reflective layer and the second reflective layer. The nanostructure layer comprises multiple nanorods, and The plurality of nanobars are arranged asymmetrically.

10. The hyperspectral element according to claim 1, wherein, Each of the first sub-filter and the second sub-filter includes: First reflective layer; Second reflective layer; and A nanostructure layer is disposed between the first reflective layer and the second reflective layer. The nanostructure layer includes multiple nanopores, and The multiple nanopores are arranged asymmetrically.

11. The hyperspectral element according to claim 1, wherein, The multi-detector comprises: alternately stacked n-type and p-type films, and The n-type film and the p-type film respectively form a first photodiode and a second photodiode, and the second photodiode is arranged further away from the multi-filter than the first photodiode.

12. The hyperspectral element according to claim 11, wherein, The multiple filters are disposed on the multiple detectors. The first photodiode is configured to detect the first wavelength light and the second wavelength light, and The second photodiode is configured to detect the light with a relatively longer wavelength between the first wavelength light and the second wavelength light.

13. The hyperspectral element according to claim 12, further comprising: Spectrum processor; The first photodiode is configured to generate a first channel signal relating to the first wavelength light and the second wavelength light, and to provide the first channel signal to the spectral processor. The second photodiode is configured to generate a second-channel signal relating to the second wavelength of light and to provide the second-channel signal to the spectral processor. The spectral processor is configured to generate information related to the intensity of the first wavelength light and the second wavelength light based on the first channel signal and the second channel signal.

14. The hyperspectral element according to claim 1, further comprising: Microlenses; The microlens and the multiple filters are arranged in series and collect incident light incident on the multiple filters.

15. A hyperspectral sensor, comprising: Multiple pixels are configured to sense the transmission spectrum of incident light; Spectrum processor; as well as Main processor Each of the plurality of pixels includes: A first multi-filter allows light of a first wavelength and light of a second wavelength from the incident light to pass through, wherein the second wavelength is longer than the first wavelength. A first multi-detector is configured to generate a first-channel signal with respect to the first wavelength light and the second wavelength light, and a second-channel signal with respect to the second wavelength light, and to provide the first-channel signal and the second-channel signal to the spectral processor. The second multi-filter allows light of a third wavelength and light of a fourth wavelength to pass through the incident light, wherein the fourth wavelength is longer than the third wavelength. The second multi-detector is configured to generate a third-channel signal with respect to the third wavelength light and the fourth wavelength light, and a fourth-channel signal with respect to the fourth wavelength light, and to provide the third-channel signal and the fourth-channel signal to the spectral processor. The spectral processor is configured to generate information related to the intensity of the first wavelength light, the second wavelength light, the third wavelength light, and the fourth wavelength light based on the first channel signal to the fourth channel signal, and to provide the information to the main processor.

16. The hyperspectral sensor according to claim 15, wherein, The first multi-filter includes a first sub-filter and a second sub-filter. The transmission spectrum of the first sub-filter includes a first usable segment and a first sub-transmittance region outside the first usable segment. The transmission spectrum of the second sub-filter includes a second usable segment and a second sub-transmittance region outside the second usable segment. Wherein, the first wavelength is included in the first usage segment and the second sub-transmitting region, and The second wavelength is included in the second use segment and the first sub-transmittance area.

17. The hyperspectral sensor according to claim 16, wherein, The first usage segment and the second usage segment partially overlap.

18. The hyperspectral sensor according to claim 16, wherein, The second multi-filter includes a third sub-filter and a fourth sub-filter. The transmission spectrum of the third sub-filter includes a third usable segment and a third sub-transmittance region. The transmission spectrum of the fourth sub-filter includes a fourth usable segment and a fourth sub-transmittance region. The third wavelength is included in the third usage segment and the fourth sub-transmittance region, and The fourth wavelength is included in the fourth usage segment and the third sub-transmittance region.

19. The hyperspectral sensor according to claim 18, wherein, The third usage segment partially overlaps with the fourth usage segment.

20. The hyperspectral sensor according to claim 18, wherein, The first multi-filter and the second multi-filter are arranged in parallel. The first sub-filter and the second sub-filter are arranged in series in the optical path of the incident light, and The third sub-filter and the fourth sub-filter are arranged in series in the optical path of the incident light.

21. The hyperspectral sensor according to claim 15, wherein, The first multi-detector includes: A first sub-detector is configured to receive the first wavelength light and the second wavelength light, and generate the first channel signal; and The second sub-detector is configured to receive the second wavelength light and generate the second channel signal, and The second multi-detector includes: A third sub-detector is configured to receive the third wavelength light and the fourth wavelength light, and to generate the third channel signal; and The fourth sub-detector is configured to receive the fourth wavelength light and generate the fourth channel signal.

22. The hyperspectral sensor according to claim 21, wherein, In the first multi-detector, the first sub-detector and the second sub-detector are arranged in series in the optical path of the incident light.

23. A hyperspectral image generation apparatus, comprising: The hyperspectral element of claim 1 is configured to receive incident light provided from a measurement target to generate a first channel signal with respect to light having a first wavelength and light having a second wavelength, and a second channel signal with respect to light having the first wavelength; A spectral processor is configured to generate, based on the first channel signal and the second channel signal, first hyperspectral information relating to the intensity of light having the first wavelength at each location of the measurement target, and second hyperspectral information relating to the intensity of light having the second wavelength at each location of the measurement target; The main processor is configured to generate hyperspectral image information based on the first hyperspectral information and the second hyperspectral information; as well as The display shows a hyperspectral image based on the hyperspectral image information.

Images