Spectroscopic chip

Abstract

The present application relates to a kind of spectral chips.The spectral chip includes: image sensor;And, the light modulation layer above the image sensor, wherein the light modulation layer includes the multiple filter units formed by different kinds of filter materials, and the different kinds of filter materials have different transmission spectrum.In this way, the light modulation layer formed by the filter unit arrangement of different colors of filter materials with different transmittance corresponding to different colors of filter materials is arranged, and the anti-crosstalk effect can be improved.

Description

Spectral chip Technical Field

[0001] The present application relates to the field of spectral technology, and more specifically, to a spectral chip. Background Art

[0002] A sensor array typically consists of multiple pixels, each of which is centered around a photodiode. Typically, the photoelectric conversion unit of a sensor array includes multiple photodiodes arranged in an array. These photodiodes convert captured photons into electrons, which are then converted into electrical signals through other auxiliary circuit structures for output.

[0003] However, crosstalk prevention in sensor arrays is a major issue affecting sensor array reliability. Crosstalk signals typically originate from two main sources: incident light passing through a pixel and refracting or reflecting into adjacent pixels; and photoelectrons generated by the long-wavelength portion of light deep within the substrate diffuse into adjacent pixels. As sensor array pixels become increasingly smaller, crosstalk caused by electronic coupling can impact sensor array performance. Severe crosstalk can degrade sensor array resolution and accuracy.

[0004] Currently, there are various methods for reducing spectral crosstalk. For example, some other types of crosstalk can be reduced by controlling the processing profile between pixels. However, spectral crosstalk is a natural material property that cannot be reduced through processing. Another approach to reducing spectral crosstalk is to use thicker color filter materials; this can significantly reduce crosstalk, but it also reduces the signal intensity and quantum efficiency (QE) of the pixels. Another approach to reducing spectral crosstalk is to increase the filter material concentration in the filter, but this increases the material's refractive index and leads to higher optical crosstalk.

[0005] Interactions between light and matter, such as absorption, scattering, fluorescence, and Raman spectroscopy, produce specific spectra, each unique to each substance. Therefore, spectral information can be considered the "fingerprint" of all things. Consequently, the market has developed various spectral chips corresponding to spectral information acquisition methods. However, each spectral chip has its own crosstalk issue, which can degrade the chip's recovery performance.

[0006] Based on this, it is desired to provide an improved spectral chip solution. Summary of the Invention

[0007] An embodiment of the present application provides a spectral chip that modulates light entering a light sensor through a light modulation layer composed of filter units corresponding to filter materials of different colors with different transmittances, so as to improve the anti-crosstalk effect.

[0008] According to one aspect of the present application, a spectral chip is provided, comprising: an image sensor; and a light modulation layer located above the image sensor, wherein the light modulation layer comprises a plurality of filter units formed by different types of filter materials, and the different types of filter materials have different transmission spectra.

[0009] In the above-mentioned spectral chip, one filter material corresponds to one filter unit, and one filter material corresponds to one or more physical pixels.

[0010] In the above-mentioned spectral chip, a plurality of filter materials correspond to one spectral pixel, and the light modulation layer is formed by a periodic array arrangement of the plurality of filter materials corresponding to the spectral pixel.

[0011] In the above-mentioned spectral chip, the number of filter units in the spectral pixel is greater than the number of types of filter materials.

[0012] In the above-mentioned spectral chip, the first number of filter units corresponding to the first filter material with high transmittance is smaller than the second number of filter units corresponding to the second filter material with low transmittance.

[0013] In the above-mentioned spectral chip, the filter units corresponding to the various filter materials constituting the spectral pixels include a central filter unit and m adjacent filter units, and DN0 is the central filter unit to measure the true DN values ​​of different bands, which can be expressed as: DN0 = dn0 - m * α * dn0 + α * (dn1 + dn2 + ... + dn m )

[0014] where dn1, dn2, ..., dn m is the output digital quantization vector of the physical pixels corresponding to the m adjacent filtering units at different wavelengths, ɑ is the crosstalk coefficient, and dn0 is the output digital quantization vector of the physical pixels corresponding to the central filtering unit at different wavelengths.

[0015] In the above spectrum chip, the total crosstalk x0 of the central filter unit is: x0 = sum(abs(ɑ*(dn1+dn2+…+dn m )-m*ɑ*dn0)).

[0016] In the above-mentioned spectral chip, the m adjacent filter units are four filter units adjacent to the central filter unit in four directions of up, down, left, and right, or eight filter units adjacent in four directions of up, down, left, and right and four diagonal directions.

[0017] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral chip includes: determining any one filter unit as a reference filter unit; determining a candidate filter unit whose transmission spectrum curve difference with the reference filter unit is less than a predetermined threshold, wherein the transmission spectrum curve difference refers to the sum of the absolute values ​​of the transmittance differences of the transmission spectrum curve in different bands; and arranging the reference filter unit and the candidate filter unit adjacent to each other or diagonally.

[0018] In the above-mentioned spectral chip, the reference filter unit is a red filter unit.

[0019] In the above spectral chip, the predetermined threshold is a value of a middle range of differences between all filter units or differences between other filter units and a reference filter unit, arranged in numerical order.

[0020] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral chip includes: arranging the infrared filter units away from the red filter units.

[0021] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral chip further includes: arranging the red light filter units close to each other.

[0022] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral chip includes:

[0023] The infrared filter unit is set in the center area of ​​the spectrum pixel, and the red light filter unit is then arranged at intervals; or the blue light filter unit is set in the center area of ​​the spectrum pixel, and the red light filter unit is then arranged at intervals.

[0024] In the above-mentioned spectrum chip, determining the arrangement of the filter units in the spectrum chip further includes: arranging the blue light filter unit and the red light filter unit adjacent to each other.

[0025] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral pixels includes: calculating the integral value of each filter material of the spectral chip in the 600-780nm band; determining one of the filter units corresponding to the top three filter materials with the largest integral values ​​as a reference filter unit, and arranging the other two filter units adjacent to the reference filter unit; and arranging the filter unit corresponding to the filter material with the smallest integral value and the filter units corresponding to the top three filter materials with the largest integral values ​​at intervals.

[0026] In the above-mentioned spectral chip, determining the arrangement of the filter units in the spectral pixels includes: determining any one filter unit as a reference filter unit; determining a specific wavelength band region of a transmission spectrum curve of the reference filter unit, wherein the specific wavelength band region includes at least two peaks; determining a candidate filter unit whose transmission spectrum curve is a smooth curve or includes at least one peak in the specific wavelength band region, and a central wavelength band corresponding to the at least one peak is close to a central wavelength band of any peak of the reference filter unit; and arranging the reference filter unit and the candidate filter unit adjacent to each other.

[0027] In the above-mentioned spectral chip, each filter unit corresponds to one filter material or at least two filter materials.

[0028] In the above-mentioned spectral chip, the at least two filter materials are stacked in sequence from top to bottom above the physical pixels in the filter unit.

[0029] In the above-mentioned spectral chip, different filter units in one spectral pixel use the same filter material in a predetermined light modulation layer, and use different filter materials in other light modulation layers.

[0030] In the above-mentioned spectral chip, the at least two filter materials are arranged in parallel above the physical pixels in the filter unit.

[0031] The spectral chip provided in the embodiment of the present application can modulate the light entering the photoelectric detection layer of the sensor through a light modulation layer composed of filter units corresponding to filter materials of different colors with different transmittances, so as to improve the anti-crosstalk effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Various other advantages and benefits of the present application will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The drawings in the specification are intended only to illustrate preferred embodiments and are not to be construed as limiting the present application. Obviously, the drawings described below are merely examples of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. Throughout the drawings, the same reference numerals denote the same components.

[0033] FIG1 is a schematic diagram of a light modulation layer including nine filter materials of a spectral chip according to an embodiment of the present application.

[0034] FIG2 is a schematic diagram illustrating an example of disposing filter units of different colors of a spectral chip according to an embodiment of the present application.

[0035] FIG3 is a schematic diagram illustrating transmission spectrum curves corresponding to a plurality of filter units in a spectral chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0037] In the embodiment of the present application, the spectral chip is primarily composed of an image sensor, such as a CMOS image sensor (CIS), and a light modulation layer located above the image sensor. The light modulation layer includes different types of filter materials. These different types of filter materials primarily refer to having different corresponding transmission spectra, i.e., different types of filter materials have different transmittances. The filter materials can be implemented as dyes, pigments, and other materials.

[0038] Specifically, by applying filter materials to a certain area of ​​the image sensor, the different filter materials will bring different modulation effects to light of different wavelengths, that is, different transmittances. Moreover, the greater the difference in transmittance between different filter materials for different wavelengths, the better the recovery performance of the spectral chip. That is, the spectral chip can be formed by applying n types of filter materials to the upper end of the photodiode (PD) of the image sensor. One filter material can correspond to one physical pixel or multiple physical pixels. For example, one filter material corresponds to one physical pixel, one filter material corresponds to four physical pixels, or nine pixels, etc. The filter materials and corresponding physical pixels constitute a filter unit. Multiple (at least two) filter units constitute a spectral pixel, which is the minimum unit for restoring the spectral curve. The spectral chip includes at least one spectral pixel. For example, if the spectral chip is used to restore the spectral curve, the spectral pixel can be one, while if the spectral chip is used for spectral imaging, the number of spectral pixels is at least two. It should be noted that, in some embodiments, the number of physical pixels corresponding to different filter materials is different. For example, there are at least two filter materials, and the transmittance of one filter material is significantly greater than that of the other filter material. At this time, the loss of incident light with a large transmittance is small, and the response value received by the corresponding physical pixel will be larger, and the corresponding signal-to-noise ratio will also be larger, and the overall effect will be better. However, due to the large loss of incident light, the same number of physical pixels may not give a good effect. Therefore, in this embodiment, the number of physical pixels corresponding to the filter material with a larger transmittance is smaller than the number of physical pixels corresponding to the filter material with a smaller transmittance. For the transmittance, the transmittance curve can be integrated within a specific band (generally understood as the working band of the spectral chip, such as one or more bands of visible light, infrared, and / or ultraviolet). The larger the value, the greater the transmittance.

[0039] It should be noted that if the spectrum needs to be restored, the spectral pixel requires more filter units with different transmission spectra. In principle, the number should be at least greater than two. However, for higher precision, the number of filter units required will also be greater. If the combination of filter materials to form different filter units is not considered, then in the technical solution according to the embodiment of the present application, the number of filter materials should be greater than 2, preferably greater than or equal to 3. As shown in Figure 1, Figure 1 illustrates a schematic diagram of a light modulation layer of a spectral chip according to an embodiment of the present application, including nine filter materials, namely filter materials A, B, C, D, E, F, G, H, and I, wherein the filter materials are formed above the physical pixels to form corresponding filter units A, B, ..., I. It can be considered that the filter units A, B, ..., I constitute spectral pixels.

[0040] Therefore, an embodiment of the present application provides a spectral chip, comprising: an image sensor; and a light modulation layer located above the image sensor, wherein the light modulation layer includes a plurality of filter units formed by different types of filter materials, and the different types of filter materials have different transmission spectra.

[0041] Furthermore, in the spectral chip according to the embodiment of the present application, one filter material corresponds to one filter unit, and one filter material corresponds to one or more physical pixels.

[0042] The following describes the operating principle of a spectral chip according to an embodiment of the present application. The intensity signal of the incident light at different wavelengths λ is denoted as x(λ), and the transmission spectrum curve of the optical modulation layer is denoted as T(λ). The optical modulation layer comprises filter units composed of n types of filter materials, each of which has a different transmission spectrum. Overall, the optical modulation layer can be denoted as Ti(λ) (i = 1, 2, 3, ..., n). The corresponding physical pixel beneath each filter material detects the light intensity bi adjusted by the filter material.

[0043] The relationship between the spectral distribution of the incident light and the measurement value of the image sensor can be expressed by the following formula: bi = ∫x(λ)*Ti(λ)*R(λ)dλ

[0044] Discretize again and get: bi=Σ(x(λ)*Ti(λ)*R(λ))

[0045] Where R(λ) is the response of the image sensor, which can be expressed as: Ai(λ)=Ti(λ)*R(λ),

[0046] The above formula can be expanded into matrix form:

[0047] Among them, bi (i = 1, 2, 3, ..., n) is the response of the image sensor after the light to be measured passes through the light modulation layer, which corresponds to the light intensity measurement value of the photoelectric detection layer corresponding to n filter units. When one physical pixel corresponds to one structural unit, it can be understood as the light intensity measurement value corresponding to n physical pixels, which is a vector of length n. A is the system's response to light of different wavelengths, which is determined by two factors: the transmittance of the filter structure and the quantum efficiency of the image sensor. A is a matrix, and each row vector corresponds to the response of the filter unit to incident light of different wavelengths. Here, the incident light is discretely and uniformly sampled, with a total of m sampling points. The number of columns of A is the same as the number of sampling points of the incident light. Here, x(λ) is the intensity of the incident light at different wavelengths λ, that is, the spectrum of the incident light to be measured.

[0048] On the basis of the above implementation, by arraying the spectral pixels, a snapshot spectral imaging device can be realized.

[0049] In the embodiment of the present application, the nine filter materials shown in FIG1 can be understood as constituting a spectral pixel, and the corresponding n is 9. By arranging the spectral pixels in a periodic array, the corresponding spectral chip for imaging can be obtained.

[0050] That is, in the spectral chip according to the embodiment of the present application, a plurality of filter materials correspond to one spectral pixel, and the light modulation layer is formed by a periodic array arrangement of the plurality of filter materials corresponding to the spectral pixel.

[0051] In some embodiments, the number of filter units in a spectral pixel is greater than the number of types of filter materials, that is, the same filter units exist.

[0052] That is, in the spectral chip according to the embodiment of the present application, the number of filter units in the spectral pixel is greater than the number of types of filter materials.

[0053] Furthermore, in the above-mentioned spectral chip, the number of filter units corresponding to the filter material with a larger transmittance is smaller than the number of filter units corresponding to the filter material with a smaller transmittance.

[0054] Due to limitations in filter material area and material properties, crosstalk can occur where different filter materials meet: light entering one filter element will partially propagate to another filter element. This phenomenon causes the final measured transmittance of the filter elements to vary (understandably, there will be errors). If the multiple filter materials are randomly arranged, crosstalk can reduce the transmittance differences between the different filter materials, and this effect is uncontrollable. For example, the filter element E shown in Figure 1 may experience crosstalk with other surrounding filter elements.

[0055] In the present embodiment, crosstalk refers to the phenomenon in which, after light passes through a filter material in the light modulation layer, some light that theoretically falls on the projection area of ​​the image sensor corresponding to that filter material may enter the projection area of ​​the image sensor corresponding to another filter material due to diffraction or other reasons. This light ultimately falls on the projection area of ​​the other filter material on the image sensor, causing the final digital quantization (DN) value of the other filter material to change.

[0056] Furthermore, in order to provide a spectral chip with improved anti-crosstalk effect, a definition of the crosstalk coefficient is provided in an embodiment of the present application. Taking FIG1 as an example, the filter material E coated on the image sensor and the eight filter materials around it are used as the minimum repeating unit to obtain a filter material arrangement covering the entire image sensor. Light of different wavelengths passes through the filter material and enters the photoelectric detection layer below, and is converted into an electrical signal and output in the form of a DN value. The following relationship exists: DN E =dn E -4*ɑ*dn E +ɑ*(dn B +dn D +dn F +dn H )

[0057] In the above formula, DN E is measured, which is represented by the actual DN value of the filter unit E in different bands (a vector), dn E The theoretical crosstalk-free DN value of the filter unit in different bands (a vector). B 、dn D 、dn F 、dn H is the theoretical crosstalk-free DN value (respectively a vector) for the surrounding filter units B, D, F, and H at different wavelengths. α is the crosstalk coefficient, representing the proportion of light that is crosstalked from filter unit E to a surrounding filter unit. The DN value can be understood as the light intensity measurement value obtained by the physical pixel after the light enters the filter material and reaches the physical pixel. * is the dot product, that is, the element-wise multiplication between vectors. In this example, it can be understood that crosstalk is generally mainly generated between adjacent filter units, so when calculating the crosstalk coefficient, the four adjacent filter units are mainly considered. Individual embodiments may also consider the eight adjacent filter units. For example, if filter unit E is considered, filter units A to I must also be taken into account. It should be noted that the present invention is based on the crosstalk coefficient between filter units being fixed and consistent. There may be certain differences in practice, but this does not deviate from the scope of the present invention. That is, regardless of whether there are differences, a corresponding crosstalk coefficient can be calculated, and then the crosstalk coefficient can be used to calculate the crosstalk amount.

[0058] Furthermore, the present invention provides a definition of the crosstalk amount x. For example, taking filter unit E as an example, when the light under filter unit E crosstalks to the surrounding filter units, the light from the surrounding filter units will also crosstalk back to filter unit E. Therefore, after the crosstalk ends, the total amount of crosstalk under filter unit E can be calculated as x e =sum(abs(α*(dn B +dn D +dn F +dn H )-4*α*dn E ))

[0059] Among them, sum() means summing all elements, so x e is a rational number.

[0060] It can be understood that the crosstalk amount x of any filter unit can be calculated in the above manner, that is, the crosstalk amount x represents the sum of the absolute values ​​of the differences between the theoretical crosstalk-free DN value and the measured DN value of the filter unit in different bands.

[0061] That is, in the spectral chip according to the embodiment of the present application, the filter units corresponding to the multiple filter materials constituting the spectral pixels include a central filter unit and m adjacent filter units, and DN0 is the central filter unit measured to obtain the true DN value of different bands as follows: DN0 = dn0 - m*ɑ*dn0 + ɑ*(dn1 + dn2 + ... + dn m )

[0062] where dn1, dn2, ..., dn m is the output digital quantization vector of the physical pixels corresponding to the m adjacent filtering units at different wavelengths, α is the crosstalk coefficient, and dn0 is the output digital quantization vector of the physical pixels corresponding to the central filtering unit at different wavelengths.

[0063] Furthermore, in the spectral chip, the total crosstalk x0 of the central filter unit is: x0=sum(abs(α*(dn1+dn2+…+dn m )-m*α*dn0))

[0064] Here, the m adjacent filter units may be four filter units adjacent to the central filter unit in four directions of up, down, left, and right, or eight filter units adjacent in four directions of up, down, left, and right and four diagonal directions.

[0065] It should be noted that the spectral chip involved in the present invention can be understood as a periodic array of multiple spectral pixels. Therefore, it can be assumed that there will be crosstalk caused by adjacent spectral pixels at the boundaries. Therefore, the boundary problem can be ignored, and only the overall crosstalk amount of the spectral pixels needs to be calculated. Of course, in principle, there must be filter units at the boundaries of the spectral chip where the number of adjacent filter units is less than four. However, for the present invention, the filter units at the boundaries can be temporarily ignored. Therefore, for a solution with only one spectral pixel, the crosstalk amount of all filter units in a certain spectral pixel can also be calculated according to the periodic array. Furthermore, the present invention defines adjacent and spaced arrangements between filter units. Adjacent filter units as referred to in the present invention means that at least one side of the filter units is adjacent. If the filter units are rectangular, at least one side is adjacent. If the filter units are circular or irregular in shape, it can be assumed that the filter units are periodically arranged, and there will be corresponding boundaries, and the boundaries are adjacent. Spaced arrangements refer to filter units having no adjacent boundaries, for example, diagonal arrangements (diagonal arrangements can be understood as assuming that the filter units are regular rectangles, and the two filter units have no adjacent sides but have a common corner) or separated arrangements. The specific method is as follows:

[0066] Each filter unit can calculate a crosstalk amount x n , after adding up the crosstalk amounts of all filter units, the sum of all crosstalk amounts can be obtained as the total crosstalk amount of the spectral pixel, for example, recorded as x total , that is, assuming that the spectral pixel is composed of n filter units, the crosstalk amount x of each filter unit composed of filter materials can be calculated n , then x total =x1+x2+…+x n For example, taking the nine filter materials shown in Figure 1 as an example, then x total =x A +x B +x C +x D +x E +x F +x G +x H +x I In some embodiments, the number of filter units in a spectral pixel is greater than the number of filter material types, that is, even if the same filter units exist, the total crosstalk amount is still the sum of the crosstalk amounts of each filter unit.

[0067] When x total When x is the smallest, it can be considered that the corresponding crosstalk of the n filter materials under this arrangement is the smallest, that is, it has a good anti-crosstalk capability. totalThe smaller it is, the smaller the corresponding crosstalk is. For those who need spectral imaging, on this basis, you can obtain the spectral pixels with the smallest total crosstalk, and then array the constructed spectral pixels, so that you can obtain a spectral chip with low crosstalk for spectral imaging. Back to the above-mentioned boundary problem, that is, the present invention determines the arrangement scheme by calculating the total crosstalk amount. When the total crosstalk amount is the smallest, the total spectral crosstalk amount corresponding to the periodic array of the spectral pixels is also the smallest, so there is no need to pay too much attention to the boundary problem. For the spectral chip with a single spectral pixel, it is necessary to assume that the spectral chip can be a plurality of spectral pixels arranged in a periodic manner during the arrangement, so that the crosstalk amount at the boundary can be calculated, and finally the spectral pixel with the smallest total crosstalk amount is obtained, which is the required arrangement scheme; it is also possible to calculate the corresponding crosstalk amount for the boundary filter unit separately. The overall calculation principle is the same as above, the difference is that the number of filter units that form crosstalk is reduced.

[0068] Further, provide x total The minimum calculation method, when there are fewer types of filter materials, can be screened by exhaustive method to find the minimum x total That is, the total crosstalk value under each arrangement is calculated, and the one with the smallest value is the optimal solution.

[0069] When there are many types of filter materials, exhaustive enumeration is less feasible. Using the above-mentioned method, manual arrangement can be performed according to certain rules, or computer-assisted arrangement can be used to obtain the lowest total crosstalk possible. The rules proposed herein for manual arrangement include: filter materials with similar transmission spectrum curves are placed as close together as possible. Similar transmission spectrum curves refer to the sum of the absolute values ​​of the transmittance differences between the two transmission spectrum curves in different wavelength bands being smaller, which can be defined as the smaller the difference (the difference can be defined as a numerical value). That is, the spectral chip contains n types of filter materials, corresponding to n types of filter units. The transmission spectrum curves of each filter unit are different, but the transmission spectrum curves are obtainable, i.e., the transmission spectrum curves are known. By determining the differences between the filter units, if the two are similar, they are arranged as close together as possible. Any filter unit can be selected as the reference filter unit. For example, a red filter unit is preferably selected as the reference filter unit, and filter units with smaller corresponding differences are arranged around it in sequence. In this case, a reference filter unit exists in the spectral chip, and the difference between it and an adjacent filter unit is less than a predetermined threshold. For example, the difference is preferably minimized, meaning that the difference between the reference filter unit and the other filter units is greater than the reference filter unit. Preferably, the red filter unit is the reference filter unit, and the distance between the filter unit centers is the distance between the filter units. It is worth noting that the selection of the predetermined threshold in the present invention is affected by the filter material. It can be understood that once the filter material is determined, the differences between all filter units, or between other filter units and the reference filter unit, can be calculated. All differences are ranked, and the middle difference value (i.e., the differences can be ranked numerically and the median taken) can be the predetermined threshold. Filter units less than the predetermined threshold are preferably positioned adjacent to or diagonally opposite the reference filter unit. Furthermore, in certain embodiments, unlike the above embodiments, considering that the spectral chip is comprised of multiple spectral pixel arrays, it is also necessary to consider that individual filter materials cannot be arranged adjacent to each other. Therefore, it is necessary to consider a reasonable arrangement to minimize crosstalk. Therefore, in the spectral chip, the difference between the reference filter unit and the diagonally opposite filter units is minimized. It should be noted that special scenarios need to be considered here. In certain embodiments, the crosstalk coefficient α may be 0 or approaching 0 (a 0 vector, which can be understood as the crosstalk coefficient between any filter unit being 0 or approaching 0, or equivalently, the overall crosstalk coefficient being 0 or approaching 0). This means that the crosstalk between the filter units is insignificant, and in this case, the filter materials can be arranged arbitrarily. For example, as shown in Figure 1, if the mutual crosstalk between filter unit A and filter unit B is 0, the transmission spectra of filter unit A and filter unit B can be similar or completely different. If mutual crosstalk exists between the two, the transmission spectra of filter unit A and filter unit B must be as similar as possible.

[0070] Therefore, in the spectral chip according to the embodiment of the present application, determining the arrangement of the filter units in the spectral pixels includes: determining any one filter unit as a reference filter unit; determining a candidate filter unit whose transmission spectrum curve difference with the reference filter unit is less than a predetermined threshold, wherein the transmission spectrum curve difference refers to the sum of the absolute values ​​of the transmittance differences of the transmission spectrum curve in different wavelength bands; and arranging the reference filter unit and the candidate filter unit adjacent to each other.

[0071] Furthermore, in the above-mentioned spectral chip, the reference filter unit is preferably a red filter unit.

[0072] Specifically, in the prior art, common filter materials are generally composed of R, G, B, IR, etc. For example, RGB forms a more common visible light sensor, and RGB-IR can constitute a common visible light-infrared sensor, that is, R, G, B, and IR all have their own transmission spectrum curves. For example, the transmission band of R is basically 600-780nm, the transmission band of B is basically 350-450nm, and the IR band is greater than 780nm. In this regard, the present invention can classify the filter materials that need to be arranged. The ones with higher transmittance in the 600-780nm band can be defined as red light filter materials, and the ones with higher transmittance in the 350-450nm band are defined as blue light filter materials. Materials with higher transmittance in the band greater than 780nm are defined as infrared filter materials. It should be noted that this definition refers to the specific type and quantity of filter materials. The transmission spectrum curve of each filter material can be displayed in the same coordinate system (here we follow the conventional coordinate system, with the horizontal axis being the band and the vertical axis being the transmittance). The 600-780nm band is intercepted. If the proportion of the integral value in the 600-780nm band to the total value is greater than 50%, the transmittance is considered to be higher; similarly, if the proportion of the integral value in the 350-450nm band is greater than 50%, it is defined as a blue light filter material, and if the proportion of the integral value in the band greater than 780nm is greater than 50%, it is defined as an infrared filter material. For the spectral chip according to the embodiment of the present application, the red light filter units of the spectral chip are arranged as close as possible, while the infrared filter units are arranged as far away from the red light filter units as possible. Specifically, in the spectral chip, the filter units belonging to the same red light category are arranged adjacent to each other, and the filter units belonging to the infrared category are arranged at intervals from the red light filter units. To a certain extent, different needs may affect different arrangement methods. Therefore, it can be considered that in order to achieve better results, the red light filter units should be arranged as close as possible, while the infrared filter units should be arranged away from the red light filter units. It can also be considered that there is at least one red light filter unit and an infrared filter unit arranged in intervals.

[0073] It should be noted that the spectral chips protected by this invention are based on a holistic approach, aiming to minimize crosstalk across the entire spectral chip. Therefore, the arrangement not only considers a specific spectral pixel but also all spectral pixels. This means that the red and infrared filter units of adjacent spectral pixels are spaced apart. In other words, this invention should be understood as the arrangement of the entire spectral chip following a corresponding pattern, such as the arrangement of at least one spectral pixel or the periodic arrangement of a particular spectral pixel.

[0074] Therefore, in the spectral chip according to the embodiment of the present application, determining the arrangement of the filter units in the spectral chip includes: arranging the infrared filter units away from the red filter units.

[0075] Furthermore, in the above-mentioned spectral chip, the red light filter units are arranged close to each other.

[0076] In one example, for a spectral chip requiring a periodic arrangement of spectral pixels, if an infrared filter unit is set at a boundary of the spectral pixel and not at a corner, then a red filter unit should not be set at the corresponding position corresponding to the other non-adjacent boundary. Furthermore, if an infrared filter unit is set at a corner of the spectral pixel, then red filter units should not be set at the two adjacent corners. Regardless of whether the red filter unit is set at a corner or boundary, it will be more difficult to set the red filter unit. Therefore, preferably, for spectral pixels with 9 or more filter units, the infrared filter unit is set at the center of the spectral pixel (it should be noted that due to the periodic arrangement of the spectral chip, there may not be a defined center area for the entire spectral chip), and the red filter units are then set at intervals.

[0077] In another embodiment, if the spectral pixel does not have an infrared filter unit but has a blue light filter unit, the blue light filter unit needs to be arranged away from the red light filter unit. The specific arrangement is similar to that of the infrared type. However, since the crosstalk between the blue light filter unit and the red light filter unit will not be too large, partial adjacent arrangements can be tolerated.

[0078] In another embodiment, when a red light filter unit, a blue light filter unit, and an infrared light filter unit are all present, priority is given to spacing the red light filter unit and the infrared light filter unit. Otherwise, the arrangement is the same as that of the infrared light filter unit and the red light filter unit.

[0079] Therefore, in the spectral chip according to the embodiment of the present application, the arrangement of the filter units in the spectral pixels is determined by: setting the infrared filter units in the central area of ​​the spectral pixels and then arranging the red light filter units at intervals; or setting the blue light filter units in the central area of ​​the spectral pixels and then arranging the red light filter units at intervals, wherein, in some embodiments, the blue light filter units are partially arranged adjacent to the red light filter units.

[0080] By way of example, but not limitation, a spectral pixel may be composed of 3*3 filter units, with red filter units R0, R1, and R2, green filter units G0, G1, and G2, blue filter units B0 and B1, and infrared filter unit IR. Arranged according to the above arrangement, the IR filter unit can be placed in the center, the red filter units can be spaced apart in the corners, and the green and blue filter units can be ignored, as shown in FIG2 . FIG2 illustrates a schematic diagram of an example arrangement of filter units of different colors in a spectral chip according to an embodiment of the present application.

[0081] In another embodiment, the integral value of each filter material constituting the spectral chip in the 600-780 nm band may be calculated, and all of the filter materials may be sorted from largest to smallest according to the integral value. One of the filter units formed from the three filter materials with the highest integral values ​​is selected as a reference filter unit, and the other two filter units are positioned adjacent to the reference filter unit. The filter unit formed from the filter material with the lowest integral value in the 600-780 nm band is positioned spaced apart from the filter units formed from the three filter materials with the highest integral values. For example, as shown in FIG2 , filter units R0, R1, and R2 have relatively large integral values ​​in the 600-780 nm band, while filter unit IR has the lowest integral value in the 600-780 nm band. In this case, the arrangement shown in FIG2 may be considered, and the spectral pixels shown in FIG2 may be periodically arranged to produce a spectral chip. In this case, filter units R0 and R2 in the spectral chip are positioned adjacent to filter unit R1, while filter unit IR is spaced apart from the three filter units. It should be noted that the spectral chips protected by this invention are considered as a whole, i.e., the overall crosstalk of the spectral chip is minimized. Therefore, the arrangement is not just about individual spectral pixels, but the entire spectral chip itself. It should be noted that while this invention is preferably described using a periodic arrangement, individual embodiments may also utilize a non-periodic arrangement. However, the overall principle and arrangement remain unchanged; for example, the entire chip can be considered a single spectral pixel.

[0082] Therefore, in the spectral chip according to the embodiment of the present application, determining the arrangement of the filter units in the spectral pixels includes: calculating the integral value of each filter material of the spectral chip in the 600-780nm band; determining one of the filter units corresponding to the top three filter materials with the largest integral values ​​as a reference filter unit, and arranging the other two filter units adjacent to the reference filter unit; and arranging the filter unit corresponding to the filter material with the smallest integral value at intervals from the filter units corresponding to the top three filter materials with the largest integral values.

[0083] In addition, in the embodiments of the present application, the transmission spectrum curve of the filter unit corresponding to each filter material has at least two peaks in a specific wavelength range. The presence of these peaks helps to improve the transmission spectrum curve difference and enhance the spectral recovery capability. When it is desired to preserve the transmission spectrum curve difference, as shown in Figure 3, taking filter unit E as an example, the transmission spectrum curve of filter unit E has multiple peaks (with some fluctuations) in the wavelength range from λ1 to λ2. In this case, the filter unit disposed adjacent to filter unit E preferably has a smooth transmission spectrum curve in the corresponding wavelength range or at least two peaks in the corresponding wavelength range, and the central wavelength range corresponding to the peaks is close to the central wavelength range of filter unit E. This ensures that the transmission spectrum curve of the filter unit has fluctuations in the wavelength range and is not offset by crosstalk, thereby ensuring the recovery capability. Here, the specific wavelength range can be manually set according to needs. For example, for different application scenarios, different transmission spectrum curve differences are required in different wavelength ranges to achieve better spectral recovery capability in the specific wavelength range. Typically, specific wavelength regions are associated with filter units, meaning that different filter units exhibit different transmission spectrum curves. When at least two peaks appear within a specific wavelength region, these fluctuations are preserved as much as possible. Figure 3 illustrates a schematic diagram of transmission spectrum curves corresponding to multiple filter units in a spectral chip according to an embodiment of the present application.

[0084] That is, in the spectral chip according to an embodiment of the present application, determining the arrangement of the filter units in the spectral pixels includes: determining any one filter unit as a reference filter unit; determining a specific wavelength region of the transmission spectrum curve of the reference filter unit, wherein the specific wavelength region includes at least two peaks; determining a candidate filter unit whose transmission spectrum curve is a smooth curve or includes at least one peak in the specific wavelength region, and whose central wavelength corresponding to the at least one peak is close to the central wavelength of any peak of the reference filter unit; and arranging the reference filter unit and the candidate filter unit adjacent to each other. A smooth curve can be understood as a curve having no peaks or valleys in the specific wavelength region, or a transmission spectrum curve having a small difference between the maximum and minimum values ​​in the specific wavelength region.

[0085] Furthermore, in the spectral chip according to the embodiment of the present application, to achieve better spectral restoration accuracy, the filter units need to have greater diversity and be as numerous as possible. This requires a wide variety of filter materials, and the filter materials must have good performance to ensure restoration accuracy. Therefore, at least two filter materials can be placed above a physical pixel to form a filter unit. As described above, filter material A corresponds to filter unit A, and filter material B corresponds to filter unit B. In this embodiment, filter material A and filter material B are simultaneously applied above the physical pixels (which can be one or multiple physical pixels) that constitute a filter unit. The filter units are formed by the combined action of filter material A and filter material B. It should be noted that in this embodiment, the use of at least two filter materials to form a filter unit does not involve mixing various filter materials to form a new filter material. Instead, multiple filter materials are arranged together above the physical pixels to form a filter unit while maintaining the properties of the various filter materials. The multiple filter materials work together to form a corresponding transmission spectrum curve. This solution is conducive to manufacturing more filter units with different transmission spectrum curves under specific conditions of the amount and type of filter materials, thereby improving the restoration accuracy.

[0086] That is, in the spectral chip according to the embodiment of the present application, each filter unit corresponds to one filter material or at least two filter materials.

[0087] For example, when the filter unit is composed of physical pixels and at least two filter materials, the at least two filter materials can be stacked sequentially from top to bottom and formed above the physical pixels. Specifically, the filter unit can be composed of one physical pixel or at least two physical pixels (for example, four physical pixels), and the filter materials can be two or more. In this embodiment, at least two filter materials are sequentially formed above the physical pixels during the manufacturing process. For example, two filter materials form two layers, three filter materials form three layers, and multiple filter materials form multiple layers. For example, two filter materials form a first light modulation layer and a second light modulation layer, respectively, and three filter materials form a first light modulation layer, a second light modulation layer, and a third light modulation layer, respectively.

[0088] That is, in the above-mentioned spectral chip, the at least two filter materials are stacked in sequence from top to bottom above the physical pixels in the filter unit.

[0089] In addition, in some embodiments, in order to reduce process costs, different filter units in a spectral pixel can use the same filter material in a certain light modulation layer, and use different filter materials in other light modulation layers, so that different filter units also have different modulation effects, but the process cost can be reduced (the process steps can be reduced to a certain extent).

[0090] That is, in the above-mentioned spectral chip, different filter units in one spectral pixel use the same filter material in a predetermined light modulation layer, and use different filter materials in other light modulation layers.

[0091] Furthermore, in another variant embodiment, based on the above embodiment, the coverage area of ​​the upper filter material can be reduced, that is, it does not completely cover the lower filter material. In other words, the area of ​​the filter material corresponding to the filter material further away from the photodetection layer is smaller. In this way, the light transmittance can be maximized to a certain extent without affecting the modulation effect, thereby improving the overall signal-to-noise ratio and enhancing accuracy.

[0092] That is, in the above-mentioned spectral chip, the coverage area of ​​the filter material farther from the image sensor among the at least two filter materials is smaller than the coverage area of ​​the filter material closer to the image sensor.

[0093] Furthermore, in another variant embodiment, based on the above embodiment, to address the low transmittance issue associated with a stacking scheme, at least two filter materials are arranged in parallel on the light modulation layer. Specifically, different filter materials within the same filter unit are not stacked, but rather located on the same layer. For example, these materials may be arranged side by side or in an enclosed arrangement.

[0094] That is, in the above-mentioned spectral chip, the at least two filter materials are arranged in parallel above the physical pixels in the filter unit.

[0095] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0096] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0097] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0098] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0099] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A spectral chip, comprising: Image sensor; as well as, A light modulation layer is located above the image sensor, wherein the light modulation layer includes a plurality of filter units formed by different types of filter materials, and the different types of filter materials have different transmission spectra.

2. The spectral chip according to claim 1, wherein: One filter material corresponds to one filter unit, and one filter material corresponds to one or more physical pixels.

3. The spectral chip according to claim 1, wherein: A plurality of filter materials correspond to one spectral pixel, and the light modulation layer is formed by a periodic array arrangement of the plurality of filter materials corresponding to the spectral pixel.

4. The spectral chip according to claim 3, wherein: The number of filter units in the spectral pixel is greater than the number of filter material types.

5. The spectral chip according to claim 4, wherein: The first number of filter units corresponding to the first filter material with high transmittance is less than the second number of filter units corresponding to the second filter material with low transmittance.

6. The spectral chip according to claim 3, wherein: The filter units corresponding to the various filter materials constituting the spectral pixel include a central filter unit and m adjacent filter units, and DN0 is the central filter unit. The real DN values ​​of different bands are expressed as: DN0 = dn0-m*α*dn0+α*(dn1+dn2+...+dn m ) where dn1, dn2, ..., dn m is the output digital quantization vector of the physical pixels corresponding to the m adjacent filtering units at different wavelengths, α is the crosstalk coefficient, and dn0 is the output digital quantization vector of the physical pixels corresponding to the central filtering unit at different wavelengths.

7. The spectral chip according to claim 6, wherein: The total amount of crosstalk x0 of the central filter unit is: x0=sum(abs(α*(dn1+dn2+…+dn m )-m*ɑ*dn0)).

8. The spectral chip according to claim 6, wherein: The m adjacent filter units are four filter units adjacent to the central filter unit in four directions of up, down, left, and right, or eight filter units adjacent to the four directions of up, down, left, and right and four diagonal directions.

9. The spectral chip according to claim 1 or 3, wherein: Determining the arrangement of the filter units in the spectral chip includes: Determine any one filter unit as a reference filter unit; Determine a candidate filter unit whose transmission spectrum curve difference with the reference filter unit is less than a predetermined threshold, wherein the transmission spectrum curve difference refers to the sum of the absolute values ​​of the transmittance differences of the transmission spectrum curve in different bands; and arrange the reference filter unit and the candidate filter unit adjacent to each other or diagonally.

10. The spectral chip according to claim 9, wherein: The reference filter unit is a red filter unit.

11. The spectral chip according to claim 9, wherein: The predetermined threshold is a value of a difference between all the filter units or a difference between other filter units and a reference filter unit, which is arranged in an intermediate range according to numerical values.

12. The spectral chip according to claim 9, wherein: Determining the arrangement of the filter units in the spectral chip includes: arranging the infrared filter units away from the red filter units.

13. The spectral chip according to claim 12, wherein: Determining the arrangement of the filter units in the spectral chip further includes: arranging the red light filter units close to each other.

14. The spectral chip according to claim 9, wherein: Determining the arrangement of the filter units in the spectral chip includes: The infrared filter unit is arranged in the center area of ​​the spectral pixel, and the red light filter units are arranged at intervals; or, The blue light filter units are arranged in the center area of ​​the spectrum pixel, and the red light filter units are arranged at intervals.

15. The spectral chip according to claim 14, wherein: Determining the arrangement of the filter units in the spectrum chip further includes: arranging the blue light filter unit and the red light filter unit partially adjacent to each other.

16. The spectral chip according to claim 3, wherein: Determining the arrangement of the filter units in the spectral pixel includes: Calculating the integral value of each filter material of the spectrum chip in the 600-780nm band; Determine one of the filter units corresponding to the first three filter materials with the largest integral values ​​as a reference filter unit, and the other two filter units are arranged adjacent to the reference filter unit; and The filter unit corresponding to the filter material with the smallest integral value is arranged at intervals from the filter units corresponding to the first three filter materials with the largest integral values.

17. The spectral chip according to claim 3, wherein: Determining the arrangement of the filter units in the spectral pixel includes: Determine any one filter unit as a reference filter unit; Determine a specific wavelength region of the transmission spectrum curve of the reference filter unit, wherein the specific wavelength region includes at least two peaks; Determine a candidate filter unit whose transmission spectrum curve is a smooth curve or includes at least one peak in the specific waveband region, and the central waveband corresponding to the at least one peak is close to the central waveband of any peak of the reference filter unit; and The reference filter unit and the candidate filter unit are arranged adjacent to each other.

18. The spectral chip according to claim 1, wherein: Each filter unit corresponds to one filter material or at least two filter materials.

19. The spectral chip according to claim 18, wherein: The at least two filter materials are stacked sequentially from top to bottom in the filter unit and formed above the physical pixels.

20. The spectral chip according to claim 19, wherein: Different filter units in the one spectral pixel use the same filter material in a predetermined light modulation layer, and use different filter materials in other light modulation layers.

21. The spectral chip according to claim 18, wherein: The at least two filter materials are arranged in parallel above the physical pixels in the filter unit.

Images