Spectroscopic chip and spectroscopic camera

Abstract

This application relates to a spectral chip and a spectral camera. The spectral chip of this application includes: a silicon substrate including an image sensing layer; and a spectral modulation layer disposed on the image sensing layer, wherein the spectral modulation layer includes: a broadband modulation region for acquiring first image data of a target; and a narrowband modulation region for acquiring second image data of the target, wherein the first image data is used to perform white balance processing on the second image data.

Description

Technical Field

[0001] This application relates to the field of spectral modulation technology, specifically to a spectral chip and a spectral camera. Background Technology

[0002] In traditional imaging technology, RGB images are acquired by sensors, and white balance algorithms are used to restore the image colors. However, because RGB images contain little information, resulting in significant signal loss, the white balance algorithm is not accurate enough when restoring image colors. Utility Model Content

[0003] In view of the problems existing in the prior art, the purpose of this application is to provide a spectral chip and a spectral camera.

[0004] In a first aspect of this application, a spectral chip is provided, comprising:

[0005] Silicon-based substrate, including the image sensing layer;

[0006] A spectral modulation layer is disposed on the image sensing layer, wherein the spectral modulation layer comprises:

[0007] Broadband modulation region, to acquire the first image data of the target;

[0008] Narrow-spectrum modulation region, to acquire second image data of the target;

[0009] The first image data is used to perform white balance processing on the second image data.

[0010] Optionally, the broadband modulation region is disposed around the periphery of the spectral chip, surrounding the narrowband modulation region, wherein the global color temperature value of the ambient light of the environment in which the target is located is obtained based on the first image data, and the global color temperature value is used to perform global white balance correction on the second image data.

[0011] Optionally, the broadband modulation region includes multiple broadband modulation units arranged in an array along the row and column directions, respectively, each broadband modulation unit being composed of broadband modulation sub-units with different transmittance curves; the narrowband modulation region includes multiple narrowband modulation units arranged in an array along the row and column directions, each narrowband modulation unit including a narrowband modulation sub-unit composed of at least two of green filter materials, red filter materials, blue filter materials, yellow filter materials, and near-infrared filter materials.

[0012] Optionally, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; the narrowband modulation unit is a 2*2 narrowband modulation sub-structure made of green filter material, red filter material and blue filter material, wherein the two materials on one diagonal of the 2*2 structure are both green filter materials.

[0013] Optionally, the broadband modulation region is distributed across the spectral chip, and the remaining area is the narrowband modulation region, wherein...

[0014] Each broadband modulation region includes a broadband modulation unit, and each broadband modulation unit consists of broadband modulation subunits with different transmittance curves;

[0015] The narrowband modulation region includes multiple narrowband modulation units arranged in arrays along the row and column directions, respectively. Each narrowband modulation unit includes a narrowband modulation subunit composed of at least two of the following: green filter material, red filter material, blue filter material, yellow filter material, and near-infrared filter material.

[0016] The local color temperature value of the ambient light of the target's environment is obtained based on the first image data of each broadband modulation zone, and the local color temperature value is used to perform local white balance correction on the second image data obtained by the narrowband modulation unit within a preset range of the broadband modulation zone.

[0017] Optionally, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; the narrowband modulation unit is a 2*2 narrowband modulation sub-unit structure made of green filter material and red filter material, wherein the two materials on one diagonal of the 2*2 structure are both green filter materials and the two materials on the other diagonal are both red filter materials.

[0018] Optionally, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; the narrowband modulation unit includes a first narrowband modulation sub-unit and a second narrowband modulation sub-unit, wherein the first narrowband modulation sub-unit is a 2*2 narrowband modulation sub-unit structure made of red filter material, and the second narrowband sub-unit is a 2*2 narrowband modulation sub-unit structure made of green filter material.

[0019] Optionally, the spectral modulation layer includes periodic units arranged in an array along the row and column directions, respectively. The periodic units include at least two of green, red, and blue filter materials, as well as multiple spectral modulation materials with different transmittance curves. The green, red, and blue filter materials constitute the narrow-spectrum modulation region, and the spectral modulation materials constitute the broadband modulation region.

[0020] Optionally, the periodic unit is a 4*4 structure, wherein,

[0021] The first row, along the first direction, consists of red filter material, green filter material, first spectral modulation material C1, and green filter material in sequence;

[0022] The second row, along the first direction, consists of green filter material, blue filter material, green filter material, and second spectral modulation material C2;

[0023] The third row, along the first direction, consists of the third spectral modulation material C3, the green filter material, the fourth spectral modulation material C4, and the green filter material.

[0024] The fourth row, along the first direction, consists of green filter material, fifth spectral modulation material C5, green filter material, and sixth spectral modulation material C6.

[0025] Optionally, the broadband modulation region is a colloid composed of resin material, photoinitiator material, pigment, and solvent material.

[0026] Optionally, the resin material is a soluble resin, including one or more of phenolic resins, polyurethane resins, polyvinyl alcohol resins, and maleic anhydride resins;

[0027] Optionally, the resin material is a light-curing resin, including one or more of polyimide resins, polyvinyl alcohol resins, epoxy resins, and styrene resins.

[0028] Optionally, the photoinitiator material includes one or more of benzophenones, alkylphenyl ketones, benzoin and its derivatives, iodonium salts, and iron aromatics.

[0029] Optionally, the solvent material includes one or more of ethylene glycol methyl ethers, propylene glycol methyl ether acetates, triethylene glycol methyl ethers, and ethyl 3-ethoxypropionate.

[0030] Optionally, the pigment is one or more of phthalocyanine blue, phthalocyanine green, phthalocyanine yellow, pyrrolopyrrole red, pyrrolopyrrole orange, pyrrolopyrrole yellow, azo red, azo yellow, and azo orange.

[0031] Optionally, the spectral chip further includes:

[0032] A base material layer is disposed on the image sensing layer, wherein light incident on the broadband modulation region of the spectral chip is modulated by the spectral modulation layer and the base material layer.

[0033] Optionally, the base material layer has a first transmittance curve, wherein the shape of the first transmittance curve is designed such that the transmittance curve of the light incident on the broadband modulation region after being modulated by the spectral modulation layer and the base material layer is more sensitive in the preset wavelength range than in the outside of the preset wavelength range.

[0034] Optionally, the transmittance of the first transmittance curve within a preset wavelength range is higher than the transmittance outside the preset wavelength range.

[0035] Optionally, if the waveform of the broadband modulation subunit within the preset wavelength range is plateau-shaped, the modulated transmittance curve within the preset wavelength range exhibits a peak; or

[0036] If the waveform of the broadband modulation subunit has a peak within the preset wavelength range, and the transmitted transmittance curve after modulation still shows a peak within the preset wavelength range, the full width at half maximum (FWHM) of the peak after modulation is smaller than that of the peak before modulation; or

[0037] If the waveform of the broadband modulation subunit is concave within the preset wavelength range, the waveform of the modulated transmittance curve within the preset wavelength range will show a peak.

[0038] Optionally, the base material layer is a colloidal cured film composed of a mixture of resin material, photoinitiator material, and solvent material.

[0039] Optionally, the base material layer is a single-layer structure.

[0040] Optionally, in the broadband modulation region, the thickness of the base material layer corresponding to different broadband modulation subunits is stepped.

[0041] Optionally, the base material layer includes:

[0042] A mesh structure formed of a dielectric material, wherein the orthographic projection of each mesh of the mesh structure onto the image sensing layer is aligned with the orthographic projection of one or more spectral modulation materials onto the image sensing layer;

[0043] The base material is filled into the mesh, wherein the base material is the same in each mesh.

[0044] Optionally, the base material in each mesh of the grid structure corresponds to a broadband modulation sub-unit, wherein, in the broadband modulation region, the thickness of the base material layer corresponding to different spectral modulation materials is stepped.

[0045] Optionally, the thickness of the spectral modulation layer ranges from 500 to 1000 nm, and the thickness of the base material layer ranges from 50 to 500 nm.

[0046] Optionally,

[0047] The preset wavelength range is 500-650nm;

[0048] The first transmittance curve satisfies the following constraints: peak transmittance wavelength is 500-640nm, T(400nm-490nm)≥70%, T(585nm-620nm)≥90%, T(650nm-900nm)≥70%;

[0049] The filter unit includes 3*3 broadband modulation sub-units C1-C9, and the transmittance curves of the broadband modulation sub-units satisfy the following constraints:

[0050] TC1: Valley transmittance wavelength is 420-465nm, T(420nm-465nm)≤20%, T(515nm-900nm)≥80%;

[0051] TC2: Valley transmittance wavelength is 470-500nm, T(470nm-500nm)≤10%, T(525nm-900nm)≥90%;

[0052] TC3: Valley transmittance wavelength is 510-560nm, T(400nm-450nm)≥80%, T(510nm-560nm)≤20%; T(580nm-900nm)≥85%;

[0053] TC4: Valley transmittance wavelength is 585-615nm, T(415nm-480nm)≥80%, T(585nm-615nm)≤10%; T(650nm-900nm)≥75%;

[0054] TC5: Valley transmittance wavelength is 620-645nm, T(300nm-540nm) ≥80%, T(620nm-645nm) ≤10%; T(675nm-900nm) ≥85%;

[0055] TC6: Valley transmittance wavelength is 625-660nm, T(400nm-535nm) ≥80%, T(625nm-660nm) ≤10%; T(690nm-900nm) ≥85%;

[0056] TC7: Valley transmittance wavelength is 655-700nm; T(400nm-545nm) ≥80%; T(655nm-700nm) ≤20%; T(740nm-900nm) ≥85%;

[0057] TC8: Valley transmittance wavelength is 685-730nm, T(400nm-565nm) ≥80%, T(685nm-730nm) ≤10%; T(780nm-900nm) ≥85%;

[0058] TC9: Valley transmittance wavelength is 700-745nm, T(470nm-590nm) ≥80%, T(700nm-745nm) ≤40%; T(790nm-900nm) ≥80%;

[0059] The transmittance curves of the light incident on the spectral chip, after being modulated by the spectral modulation layer and the base material layer, respectively satisfy the following constraints:

[0060] TA1: Peak transmittance wavelength is 500nm-650nm, T(430nm-460nm)≤10%, T(500nm-650nm)≥60%, T(650nm-900nm)≥55%;

[0061] TA2: Peak transmittance wavelength is 550-620nm, T(460nm-500nm)≤10%, T(550nm-620nm)≥80%, T(650nm-900nm)≥65%;

[0062] TA3: Peak transmittance wavelength is 575-635nm, T(510nm-550nm)≤15%, T(575nm-635nm)≥70%, T(650nm-900nm)≥60%;

[0063] TA4: Valley transmittance wavelength is 550-620nm, T(400nm-540nm)≥20%, T(550nm-620nm)≤20%, T(640nm-900nm)≥55%;

[0064] TA5: Valley transmittance wavelength is 615-650nm, T(300nm-565nm) ≥60%, T(615nm-650nm) ≤20%, T(675nm-900nm) ≥60%;

[0065] TA6: Valley transmittance wavelength is 600-670nm, T(400nm-560nm) ≥60%, T(600nm-670nm) ≤20%, T(700nm-900nm) ≥60%;

[0066] TA7: Valley transmittance wavelength is 630-700nm, T(400nm-575nm) ≥60%, T(630nm-700nm) ≤20%, T(735nm-900nm) ≥60%;

[0067] TA8: Valley transmittance wavelength is 635-735nm, T(400nm-590nm) ≥60%, T(635nm-735nm) ≤20%, T(780nm-900nm) ≥60%;

[0068] TA9: Peak transmittance wavelength is 500-625nm, T(300nm-480nm)≤60%, T(500nm-625nm)≥60%, T(685nm-750nm)≤40%, T(810nm-900nm)≥60%.

[0069] Optionally, there is at least one inclined overlap region between adjacent broadband modulation subunits that is tilted relative to the silicon substrate, and the orthographic projection of the inclined overlap region on the silicon substrate spans two adjacent pixels on the image sensing layer.

[0070] Optionally, the spectral modulation material is frustum-shaped.

[0071] Optionally, two adjacent broadband modulation subunits are respectively a regular square frustum and an inverted square frustum.

[0072] Optionally, the angle between the beveled overlap area and the silicon substrate is between 60 degrees and 90 degrees.

[0073] Optionally, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively. The first row along the first direction is C1-C3, the second row along the first direction is C4-C6, and the third row along the first direction is C7-C9. C2, C4, C6 and C8 are regular square frustums, and C1, C3, C5, C7 and C9 are inverted square frustums.

[0074] Optionally, the preset wavelength range is 500-650nm;

[0075] The first transmittance curve satisfies the following constraints: peak transmittance wavelength is 500-640nm, T(400nm-490nm)≥70%, T(585nm-620nm)≥90%, T(650nm-900nm)≥70%;

[0076] The transmittance curve of light incident on the broadband modulation region of the spectral chip, modulated by the spectral modulation layer and the base material layer, satisfies the following constraint:

[0077] TA1: Peak transmittance wavelength is 520-640nm; T(420nm-495nm)≤10%; T(580nm-625nm)≥55%; T(670nm-900nm)≥50%;

[0078] TA2: Peak transmittance wavelength is 570-645nm; T(425nm-550nm) ≤10%; T(600nm-630nm) ≥70%; T(670nm-900nm) ≥55%;

[0079] TA3: Peak transmittance wavelength is 575-645nm; T(460nm-555nm)≤15%; T(580nm-620nm)≥70%; T(660nm-900nm)≥55%;

[0080] TA4: Valley transmittance wavelength is 550-650nm; T (400nm-500nm) ≥ 45%; T (580nm-645nm) ≤ 10%; T (700nm-900nm) ≥ 50%;

[0081] TA5: Valley transmittance wavelength is 550-675nm; T(425nm-500nm) ≥35%; T(550nm-665nm) ≤10%; T(700nm-900nm) ≥45%;

[0082] TA6: Valley transmittance wavelength is 500-650nm; T(350nm-450nm)≤40%; T(485nm-615nm)≥50%; T(800nm-900nm)≥60%;

[0083] TA7: Valley transmittance wavelength is 575-750nm; T(400nm-570nm) ≥45%; T(630nm-730nm) ≤10%; T(775nm-900nm) ≥50%;

[0084] TA8: Valley transmittance wavelength is 600-750nm; T(460nm-565nm)≥40%; T(620nm-745nm)≤10%; T(800nm-900nm)≥45%;

[0085] TA9: Valley transmittance wavelength is 600-760nm; T(460nm-580nm)≥50%; T(635nm-645nm)≤15%; T(800nm-900nm)≥50%.

[0086] In a second aspect of this application, a spectral camera is provided, comprising:

[0087] The spectral chip described in the first aspect;

[0088] The calculation unit is used to obtain the color temperature value of the ambient light of the environment where the target is located based on the first image data and to perform white balance processing on the second image data based on the color temperature value.

[0089] In this application, by setting broadband and narrowband modulation regions in the spectral modulation layer of the spectral chip, high spectral resolution image data (i.e., first image data) and high spatial resolution image data (i.e., second image data) can be obtained. The ambient light color temperature value obtained based on the high spectral resolution image data is then used to perform white balance processing on the high spatial resolution image data, accurately restoring the image. Furthermore, by optimizing the layout of the broadband and narrowband modulation regions, global white balance correction and local white balance correction can be achieved respectively. Attached Figure Description

[0090] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0091] Figure 1 This is a schematic diagram of the structure of a spectral chip according to an embodiment of this application;

[0092] Figure 2 This is a schematic diagram of the layout of a spectral modulation layer according to an embodiment of this application;

[0093] Figure 3 It is based on Figure 2 A schematic diagram of the algorithm flow corresponding to the layout of the spectral modulation layer shown;

[0094] Figure 4 This is a schematic diagram of the layout of the spectral modulation layer according to another embodiment of this application;

[0095] Figure 5It is based on Figure 4 A schematic diagram of the algorithm flow corresponding to the layout of the spectral modulation layer shown;

[0096] Figure 6 This is a schematic diagram of the layout of the spectral modulation layer according to yet another embodiment of this application;

[0097] Figure 7 This is a schematic diagram of the layout of the spectral modulation layer according to another embodiment of this application;

[0098] Figure 8 It is based on the light transmittance curve of the broadband modulation region 9 channels of the spectral chip of existing technology;

[0099] Figure 9 This is a schematic diagram of the structure of a spectral chip according to a preferred embodiment of this application;

[0100] Figure 10 This is a light transmittance curve of the base material layer according to an embodiment of this application;

[0101] Figure 11 This is a light transmittance curve of a structure having a base material layer according to an embodiment of this application after modulation;

[0102] Figure 12 (a) and (b) are schematic diagrams of a preferred structure of a broadband modulation unit according to an embodiment of this application;

[0103] Figure 13 Is Figure 8 The diagram shown illustrates the presence of an aliasing region in the light transmittance curve.

[0104] Figure 14 Is Figure 11 The diagram shown illustrates the presence of an aliasing region in the light transmittance curve.

[0105] Figure 15 (a) and (b) are schematic diagrams of a preferred structure of a broadband modulation unit according to another embodiment of this application;

[0106] Figure 16 It is based on Figure 15 A partial schematic diagram of the structure of the broadband modulation unit shown;

[0107] Figure 17 The light transmittance curve after modulation of a broadband modulation subunit having the shapes of a regular square truncated pyramid and an inverted square truncated pyramid, according to another embodiment of this application;

[0108] Figure 18 The light transmittance curve after modulation of a broadband modulation subunit having a base material layer and having the shapes of a regular square truncated pyramid and an inverted square truncated pyramid, according to another embodiment of this application;

[0109] Figure 19 This is a schematic flowchart of an image white balance processing method according to an embodiment of this application. Detailed Implementation

[0110] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0111] In the description of this application, for ease of description, spatial relative terms such as "below," "under," "below," "above," and "on" may be used to describe the relationship between one element and another. When an element or layer is referred to as "on," "adjacent to," or "connected to" other elements or layers, it may be directly on, adjacent to, or connected to other elements or layers, or there may be intervening elements or layers. Conversely, when an element is referred to as "directly on," "directly adjacent to," or "directly connected to" other elements or layers, there are no intervening elements or layers.

[0112] It should also be understood that, for ease of description, the term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and may include only A, only B, or A and B. The singular forms of the terms "a" or "this" may also include plural forms.

[0113] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0114] In traditional RGB cameras, RGB images are acquired by a sensor, and the colors are restored using a white balance algorithm. However, because RGB images are two-dimensional and lack three-dimensional spectral information, signal loss is severe, resulting in inaccurate white balance when restoring image colors.

[0115] Therefore, this application improves and optimizes the white balance algorithm for RGB images by modifying the structure of the spectral adjustment layer, resulting in a more accurate white balance effect.

[0116] like Figure 1 As shown, this application provides a spectral chip, comprising:

[0117] A silicon-based substrate, including an image sensing layer 10;

[0118] A spectral modulation layer 20 is disposed on the image sensing layer 10, wherein the spectral modulation layer includes: a broadband modulation region 250 (the area outside the dashed frame) for acquiring first image data of the target; and a narrowband modulation region 200 (the area inside the dashed frame) for acquiring second image data of the target.

[0119] The first image data is used to perform white balance processing on the second image data.

[0120] In one specific example, the first image data and the second image data are input to a white balance algorithm, which can be executed by a computing unit described later.

[0121] The image sensing layer 10 can be a CMOS image sensor or a CCD image sensor formed on a silicon substrate, wherein the image sensing layer 10 includes a plurality of pixels 100.

[0122] The broadband modulation region 250 is illustrated in the figure using nine spectral materials (broadband modulation subunits) with different transmittance curves as examples, denoted as C1, C2, C3, C4, C5, C6, C7, C8, and C9, as disclosed in CN113497065A and CN119437431A. The narrowband modulation region 200 is illustrated in the figure using a narrowband modulation subunit composed of RGB filter materials as an example. However, the materials in the narrowband modulation region are not limited to RGB; they can also be yellow filter materials and near-infrared filter materials.

[0123] The broadband modulation region is used to acquire high spectral resolution image data (i.e., the first image data), and the narrowband modulation region is used to acquire high spatial resolution image data (i.e., the second image data).

[0124] It should be noted that, in this application, the "wide" in broadband modulation is relative to the "narrow" in narrowband modulation. That is, the modulation wavelength range constituted by the modulation materials in the broadband modulation region is greater than the modulation wavelength range constituted by the modulation materials in the narrowband modulation region. For example, the modulation range constituted by C1-C9 can be between 350-1100nm, while the modulation range constituted by RGB filter units is only in the visible light band.

[0125] In the figure, each modulation sub-unit (such as C1, R) on the spectral modulation layer corresponds one-to-one with the pixel 100 below, that is, their orthogonal projections on the silicon substrate are aligned with each other. However, this application is not limited to this. In another specific example, each modulation sub-unit corresponds to multiple pixels 100 below. In this case, multiple pixels 200 corresponding to one modulation sub-unit together constitute a "superpixel". In this application, the terms "superpixel" and "pixel" are not distinguished.

[0126] In a specific example, as Figure 2 shown, the wide-spectrum modulation region 250 is disposed at the periphery of the spectral chip, surrounding the narrow-spectrum modulation region 200. In other words, in this example, the wide-spectrum modulation region forms a "hui" character shape on the spectral chip, enclosing the narrow-spectrum modulation region. The ellipsis in the figure represents the partially un-drawn modulation sub-units.

[0127] When a spectral camera constituted by a spectral chip with such a structure captures an object under ambient light, high spectral resolution image data can be obtained through the wide-spectrum modulation region, and then the global color temperature value of the ambient light can be obtained. Moreover, high spatial resolution image data can be obtained through the narrow-spectrum modulation region, and then the high spatial resolution image data can be white balanced by means of the global color temperature value to obtain a more accurately restored color image.

[0128] Specifically, Figure 3 as shown, for example, through the above-mentioned calculation unit, on the one hand, the high spectral resolution image data obtained by the wide-spectrum modulation region is spectrally inverted to obtain a spectral curve, and then the color temperature value of the ambient light is calculated through color temperature calculation. On the other hand, the high spatial resolution image data obtained by the narrow-spectrum adjustment region is de-mosaicked and then white balanced. When performing white balancing, the above-mentioned color temperature value is considered as a gain factor, so as to make the image restoration more accurate.

[0129] Among them, for the methods of spectral inversion, color temperature value calculation, and de-mosaicking, each manufacturer has its own specific methods, which are not limited in the present invention, and any existing method can be used to execute. The main contribution of this application lies in the layout manner of the wide-spectrum modulation region and the narrow-spectrum modulation region, and the white balancing correction of the high spatial resolution image data obtained through the narrow-spectrum modulation region by means of the color temperature value obtained through the wide-spectrum modulation region.

[0130] Still referring back to Figure 2 , the wide-spectrum modulation region 250 includes a plurality of wide-spectrum modulation units arranged in an array along the row direction and the column direction respectively, and each wide-spectrum modulation unit is composed of wide-spectrum modulation sub-units with different transmittance curves. In the specific example in the figure, the wide-spectrum modulation unit is a 3*3 wide-spectrum modulation sub-unit structure made of 9 different materials, and the 9 wide-spectrum modulation sub-units are respectively denoted as C1, C2, C3, C4, C5, C6, C7, C8, and C9.

[0131] The narrowband modulation region includes multiple narrowband modulation units arranged in an array along the row and column directions, respectively. Each narrowband modulation unit is a narrowband modulation subunit composed of at least two of the following: green filter material, red filter material, blue filter material, yellow filter material, and near-infrared filter material. In a specific example in the figure, the narrowband modulation unit is a 2*2 structure made of green filter material, red filter material, and blue filter material, wherein the two materials on one diagonal of the 2*2 structure are both green filter materials, and the other diagonal is composed of red filter material and blue filter material, respectively.

[0132] By selecting nine modulation materials as a spectral modulation unit and periodically repeating them in the outermost rows and columns of the spectral chip, the color temperature and white balance information of ambient light can be given more accurately, making the image color reproduction more accurate.

[0133] In addition, since these nine spectral modulation channels are broadband modulation channels, they have higher energy utilization, higher spectral resolution, and higher spectral accuracy and range.

[0134] In a preferred embodiment, Figure 3 In the "preprocessing" step shown, for the nine modulation materials, an interpolation algorithm can be introduced so that each channel in C1-C9 has the image information of the other eight channels, thereby restoring the image information of the nine spectral modulation channels. By interpolation, the complete image information of the spectral chip can be obtained without losing image spatial resolution because the wide-spectrum modulation area occupies the position of the narrow-spectrum modulation area (especially when the wide-spectrum modulation unit is a 4*4 structure or even larger).

[0135] Similarly, interpolation can be performed using interpolation algorithms known in the art, and this application is not limited to this.

[0136] The above-mentioned square-shaped layout can perform white balance correction on the entire narrow spectral modulation region surrounded by the wide spectral modulation region in the spectral chip. In other words, this layout is a "global" correction method.

[0137] In complex lighting conditions (such as multiple light sources), using the aforementioned U-shaped layout results in a weighted color temperature value from multiple light sources. The white balance calculated based on this weighted color temperature value is inaccurate, leading to inaccurate color reproduction in some areas of the image. Therefore, Figure 4 This illustrates a layout according to another embodiment of the present application.

[0138] like Figure 4As shown, broadband modulation regions are scattered across the spectral chip, while the remaining areas are narrowband modulation regions. Each broadband modulation region includes a broadband modulation unit, and each broadband modulation unit is composed of spectral modulation materials (broadband modulation sub-units) with different transmittance curves. The narrowband modulation region includes multiple narrowband modulation units arranged in arrays along the row and column directions, respectively. Each narrowband modulation unit includes a narrowband modulation sub-unit composed of at least two of the following: green filter material, red filter material, blue filter material, yellow filter material, and near-infrared filter material.

[0139] This layout allows for the individual acquisition of the local color temperature value of ambient light based on the high spectral resolution image data of each broadband modulation zone. Furthermore, this local color temperature value enables local white balance correction of the high spatial resolution image data obtained by the narrowband modulation unit (the narrowband modulation unit in the dashed box in the figure) within a preset range of the broadband adjustment zone.

[0140] Those skilled in the art will understand that this preset range can be selected according to actual needs, such as 4 rows and 4 columns of narrow spectrum modulation units around the wide spectrum modulation area, 6 rows and 6 columns of narrow spectrum modulation units, etc.

[0141] When a spectral camera with such a spectral chip captures an image of a target under ambient light, it can obtain the local color temperature value of the ambient light through each broadband modulation zone, and obtain high spatial resolution image data through the narrowband modulation zone. Furthermore, it uses the second image data obtained from the narrowband modulation unit within a preset range of the broadband modulation zone based on the local color temperature value to perform local white balance correction. In other words, this "spring-eye" layout method, where each broadband modulation zone obtains a local color temperature value, and the white balance of its nearby narrowband modulation zone is corrected using this local color temperature value, is significantly improved compared to... Figure 2 The layout shown provides more accurate color temperature and white balance information for the target, resulting in more accurate image color reproduction.

[0142] Specifically, such as Figure 5 As shown, the computing unit, on one hand, obtains a spectral curve from the broadband image data (low-resolution hyperspectral image) obtained from the broadband modulation region through spectral inversion, and then calculates the color temperature value of the ambient light. On the other hand, it performs white balance processing on the narrow-spectrum image data (high-resolution low-spectral image) obtained from the narrow-spectrum adjustment region after de-mosaicing. During white balance processing, the color temperature value of each region is used as a gain factor for regional white balance, thereby making the image restoration more accurate.

[0143] Among these methods, each manufacturer has its own specific approach for spectral inversion, color temperature calculation, and de-mosaicing. This invention does not limit the specific methods used, and any existing method can be employed. The main contribution of this application lies in the layout of the broadband and narrowband modulation regions, and in using the local color temperature value obtained from the broadband modulation region to perform white balance correction on the high spatial resolution image data obtained from the narrowband modulation region.

[0144] and Figure 3 The steps shown differ from those shown, because zone color temperature correction is required, therefore... Figure 5 The method steps shown also include: using the spectral curve obtained by spectral inversion to provide location information to perform semantic segmentation on the captured narrow-spectrum image to obtain the region for color temperature correction by partitioning the broadband modulation region.

[0145] Still turn back Figure 4 In the specific example shown in the figure, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, and the broadband modulation sub-units are respectively denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9; the narrowband modulation unit is a 2*2 narrowband modulation sub-unit structure made of green filter material and red filter material, wherein the two materials on one diagonal of the 2*2 structure are both green filter materials, and the two materials on the other diagonal are both red filter materials.

[0146] Because the 9-channel spectral modulation is a broadband modulation channel, it has higher energy utilization, higher spectral resolution, and higher spectral accuracy and range.

[0147] Nine broadband modulation subunits constitute a broadband modulation unit and are arranged periodically. In the "preprocessing" step, an interpolation algorithm can be introduced for the nine modulation materials, ensuring that each channel in C1-C9 contains image information from the other eight channels, thus reconstructing the image information of the nine spectral modulation channels. This layout obtains complete image information of the spectral chip without sacrificing image spatial resolution due to the broadband modulation region occupying the space of the narrowband modulation region (especially when the broadband modulation unit is a 4x4 structure or even larger). Interpolation can be performed using interpolation algorithms well-known in the art.

[0148] Figure 4 Another alternative implementation of the layout shown is, for example Figure 6As shown, the broadband modulation unit is a 3*3 broadband modulation sub-unit structure made of 9 different materials, and the broadband modulation sub-units are respectively denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9; the narrowband modulation unit includes a first narrowband modulation sub-unit and a second narrowband modulation sub-unit, wherein the first narrowband modulation sub-unit is a 2*2 narrowband modulation sub-unit structure made of red filter material, and the second narrowband sub-unit is a 2*2 narrowband modulation sub-unit structure made of green filter material.

[0149] This layout allows for white balance processing of image data obtained from adjacent narrow-spectrum modulation units (the narrow-spectrum modulation units in the dashed box in the figure) within a preset range, based on the color temperature value obtained from a wide-spectrum modulation zone.

[0150] Those skilled in the art will understand that this preset range can be selected according to actual needs, such as 4 rows and 4 columns of narrow spectrum modulation units around the wide spectrum modulation area, 6 rows and 6 columns of narrow spectrum modulation units, etc.

[0151] Figure 6 The effect of the layout shown is the same as Figure 4 The layout shown achieves the same effect.

[0152] Figure 7 The layout according to another embodiment of this application is shown. As shown, the spectral modulation layer includes periodic units (shown by dashed boxes) arranged in an array along the row and column directions, respectively. The periodic units include at least two of green, red, and blue filter materials, as well as multiple spectral modulation materials with different transmittance curves. The green, red, and blue filter materials constitute the narrow-spectrum modulation region, and the spectral modulation materials constitute the broadband modulation region.

[0153] Specifically, the periodic unit is a 4*4 structure, wherein the first row along the first direction (from left to right in the figure) consists of a red filter material, a green filter material, a first spectral modulation material C1, and a green filter material; the second row along the first direction consists of a green filter material, a blue filter material, a green filter material, and a second spectral modulation material C2; the third row along the first direction consists of a third spectral modulation material C3, a green filter material, a fourth spectral modulation material C4, and a green filter material; and the fourth row along the first direction consists of a green filter material, a fifth spectral modulation material C5, a green filter material, and a sixth spectral modulation material C6.

[0154] This layout method has the following beneficial effects:

[0155] Acquire spectral information while obtaining target image information;

[0156] R, G, B, and C1-C6 are arranged in a cycle as shown in the figure. Each cycle consists of one R, B, C1, C2, C3, C4, C5, C6 and eight G channels. The G channels are sensitive information channels for the human eye. The C1-C6 spectral modulation channels can provide more accurate color temperature and white balance information of the target, making the color reproduction of the image more accurate.

[0157] Because the 6-channel spectral modulation is a broadband modulation channel, it has higher energy utilization, higher spectral resolution, higher spectral accuracy, and higher spectral range.

[0158] By selecting 16 sub-units (rows and columns) of the chip as a spectral modulation period, the image information of the spectral modulation channel can be restored through interpolation algorithms. With this layout, the complete image information of the selected silicon-based chip can be obtained without losing the image spatial resolution.

[0159] As mentioned above, in order to achieve broadband modulation, the various broadband modulation subunits (modulation materials) have different transmittance curves, such as... Figure 8 As shown. However, due to limitations of the material itself, the transmittance curve of the modulation channel cannot achieve a better linear response in certain critical wavelength ranges (also known as the critical band, such as 500-650nm when used in cameras and other imaging tools for visible light imaging). Figure 8 As shown, some modulation channels exhibit plateau regions or even notch lines in this band; existing technology CN119437431A Figure 4 (As shown, the same problem exists.) When performing color reproduction, such a structure is not sensitive to the key band and will be affected by interference from bands outside the key band, especially under low light conditions.

[0160] In general, the conventional approach is to try to further adjust the material selection and ratio of each broadband modulation material. However, this method does not easily achieve satisfactory results because it requires consideration of the material selection of each broadband modulation sub-unit. Even if it is achieved, when the application scenario changes (which means the key band is different), it is necessary to try to select and match the materials of each broadband modulation sub-unit for the new key band, which is too complicated.

[0161] Therefore, in a preferred embodiment of this application, such as Figure 9 As shown, the spectral chip also includes a base material layer 30, which is disposed on the image sensing layer. The light incident on the broadband modulation region is modulated by the broadband modulation material and the base material layer.

[0162] Unlike existing technologies, this application adds a base material layer 30. Light incident on the broadband modulation region sequentially passes through the spectral modulation layer 20 and the base layer 30, and is detected by pixels in the image sensing layer 10, thereby forming an image. Therefore, in the broadband modulation region, the incident light is modulated jointly by each broadband modulation subunit and the base material layer. Of course, those skilled in the art will understand that light can also be incident on the narrow spectrum modulation region, but the current embodiment only addresses the aforementioned defects in the transmittance curve; therefore, unless otherwise specified, the situation regarding the narrow spectrum modulation region will not be described further.

[0163] The inventors introduced a base material layer 30 into the original structure and designed the shape of its transmittance curve (hereinafter referred to as the first transmittance curve). This makes the transmittance curve of the light incident on the spectral chip, after being modulated by the spectral modulation layer and the base material layer, more sensitive in the preset wavelength range than in the range outside the preset wavelength range. Essentially, this makes the spectral chip have a higher transmittance for light in the key band (which, from a design perspective, can also be called the preset wavelength range) than for light outside that range.

[0164] In a preferred embodiment, the transmittance of the first transmittance curve within a preset wavelength range is higher than the transmittance outside the preset wavelength range.

[0165] Figure 10 The figure shows the first transmittance curve designed for the visible light band 500-650nm. Specifically, the first transmittance curve satisfies the following constraints: peak transmittance wavelength is 500-640nm, T(400nm-490nm)≥70%, T(585nm-620nm)≥90%, and T(650nm-900nm)≥70%.

[0166] The process of modulating the transmittance curves of each broadband modulation subunit using the first transmittance curve is mathematically described as multiplying the transmittances of the two subunits at the same wavelength to obtain the new transmittance, thus yielding a wavelength-transmittance function. Alternatively, it can be described as using the transmittance on the first transmittance curve as a modulation factor to modulate the transmittance on the transmittance curves of each broadband modulation subunit. This design of the first transmittance curve allows for the following... Figure 8 The transmittance curve shown is modulated within a preset wavelength range, for example, to improve or alleviate situations where the curve shows a plateau or even a dip within that wavelength range.

[0167] Specifically, if the waveform of the broadband modulation subunit within the preset wavelength range is plateau-shaped, the modulated transmittance curve within the preset wavelength range will exhibit a peak; if the waveform of the broadband modulation subunit within the preset wavelength range has a peak, the modulated transmittance curve within the preset wavelength range will still exhibit a peak, and the half-width at half maximum (FWHM) of the modulated peak will be smaller than that of the peak before modulation; if the waveform of the broadband modulation subunit within the preset wavelength range is concave, the modulated transmittance curve within the preset wavelength range will exhibit a peak.

[0168] Taking C1-C9 in the figure as an example, the transmittance curves of each broadband modulation subunit satisfy the following constraints:

[0169] TC1: Valley transmittance wavelength is 420-465nm, T(420nm-465nm)≤20%, T(515nm-900nm)≥80%;

[0170] TC2: Valley transmittance wavelength is 470-500nm, T(470nm-500nm)≤10%, T(525nm-900nm)≥90%;

[0171] TC3: Valley transmittance wavelength is 510-560nm, T(400nm-450nm)≥80%, T(510nm-560nm)≤20%; T(580nm-900nm)≥85%;

[0172] TC4: Valley transmittance wavelength is 585-615nm, T(415nm-480nm)≥80%, T(585nm-615nm)≤10%; T(650nm-900nm)≥75%;

[0173] TC5: Valley transmittance wavelength is 620-645nm, T(300nm-540nm) ≥80%, T(620nm-645nm) ≤10%; T(675nm-900nm) ≥85%;

[0174] TC6: Valley transmittance wavelength is 625-660nm, T(400nm-535nm) ≥80%, T(625nm-660nm) ≤10%; T(690nm-900nm) ≥85%;

[0175] TC7: Valley transmittance wavelength is 655-700nm; T(400nm-545nm) ≥80%; T(655nm-700nm) ≤20%; T(740nm-900nm) ≥85%;

[0176] TC8: Valley transmittance wavelength is 685-730nm, T(400nm-565nm) ≥80%, T(685nm-730nm) ≤10%; T(780nm-900nm) ≥85%;

[0177] TC9: Valley transmittance wavelength is 700-745nm, T(470nm-590nm) ≥80%, T(700nm-745nm) ≤40%; T(790nm-900nm) ≥80%.

[0178] Sutra Figure 10 Modulation of the first transmittance curve shown yields the following result: Figure 11 The line shape shown:

[0179] The transmittance curves of light incident on the spectral chip, after being modulated by each broadband modulation subunit and the base material layer in the broadband modulation region, satisfy the following constraints:

[0180] TA1: Peak transmittance wavelength is 500nm-650nm, T(430nm-460nm)≤10%, T(500nm-650nm)≥60%, T(650nm-900nm)≥55%;

[0181] TA2: Peak transmittance wavelength is 550-620nm, T(460nm-500nm)≤10%, T(550nm-620nm)≥80%, T(650nm-900nm)≥65%;

[0182] TA3: Peak transmittance wavelength is 575-635nm, T(510nm-550nm)≤15%, T(575nm-635nm)≥70%, T(650nm-900nm)≥60%;

[0183] TA4: Valley transmittance wavelength is 550-620nm, T(400nm-540nm)≥20%, T(550nm-620nm)≤20%, T(640nm-900nm)≥55%;

[0184] TA5: Valley transmittance wavelength is 615-650nm, T(300nm-565nm) ≥60%, T(615nm-650nm) ≤20%, T(675nm-900nm) ≥60%;

[0185] TA6: Valley transmittance wavelength is 600-670nm, T(400nm-560nm) ≥60%, T(600nm-670nm) ≤20%, T(700nm-900nm) ≥60%;

[0186] TA7: Valley transmittance wavelength is 630-700nm, T(400nm-575nm) ≥60%, T(630nm-700nm) ≤20%, T(735nm-900nm) ≥60%;

[0187] TA8: Valley transmittance wavelength is 635-735nm, T(400nm-590nm) ≥60%, T(635nm-735nm) ≤20%, T(780nm-900nm) ≥60%;

[0188] TA9: Peak transmittance wavelength is 500-625nm, T(300nm-480nm)≤60%, T(500nm-625nm)≥60%, T(685nm-750nm)≤40%, T(810nm-900nm)≥60%.

[0189] In this application, by introducing a base material layer with a transmittance curve designed for a preset wavelength range, the transmittance curve of the original broadband modulation subunit is modulated. This optimizes the response of the spectroscopic camera to the preset wavelength range, improves image quality, reduces interference from other bands (the presence of the base material layer reduces the transmittance of light outside the preset wavelength range), and ensures accurate color reproduction during shooting. Even if the application scenario changes (the key wavelength changes), only the transmittance curve of a single base layer needs to be designed for the new target wavelength, rather than adjusting the transmittance curves of multiple broadband modulation subunits in a coordinated manner.

[0190] The following explains how to achieve the transmittance curve designed for the base material layer. The inventor's idea is to make it compatible with the materials and processes of the broadband modulation subunit.

[0191] As mentioned above, the material of the broadband modulation subunit is a colloidal curable film composed of resin materials, photoinitiator materials, pigments, and solvent materials.

[0192] The resin materials include soluble resins, such as phenolic resins, polyurethane resins, polyvinyl alcohol resins, and maleic anhydride resins; and photocurable resins, such as polyimide resins, polyvinyl alcohol resins, epoxy resins, and styrene resins.

[0193] Photoinitiator materials include benzophenones, alkyl phenyl ketones, benzoin and its derivatives, iodonium salts, and iron aromatics.

[0194] Pigments include aniline pigments, phthalocyanine pigments, azo pigments, and pyrrole pigments.

[0195] Solvent-based materials include ethylene glycol methyl ethers, propylene glycol methyl ether acetates, triethylene glycol methyl ethers, and ethyl 3-ethoxypropionate.

[0196] By selecting and proportioning these materials, the transmittance curves of each of the aforementioned broadband modulation subunits can be obtained.

[0197] Based on this, the base material layer is selected to be a colloidal curable film made of resin material, photoinitiator material and solvent material.

[0198] exist Figure 9 In the example shown, the base material layer 30 is a monolithic structure, meaning that the base material layer 30 is formed on the image sensing layer 10 in the form of a monolithic deposition, covering the image sensing layer 10. However, in this case, when light exits from a broadband modulation subunit and enters the base material layer, there will be scattering to areas outside the base material layer region corresponding to that broadband modulation subunit, thus forming crosstalk. This situation is particularly prominent when the base material layer is thick.

[0199] Therefore, this application provides another preferred structure. Specifically, the base material layer includes:

[0200] A mesh structure formed of a dielectric material, wherein the orthographic projection of each mesh of the mesh structure onto the image sensing layer is aligned with the orthographic projection of one or more broadband modulation subunits onto the image sensing layer;

[0201] The base material is filled into the mesh, wherein the base material is the same in each mesh.

[0202] The medium material is preferably a light-absorbing material, such as the black matrix commonly used in the display panel industry.

[0203] Those skilled in the art will understand that, for Figure 2 , 4 For the narrow-spectrum modulation region in the layouts shown in 6 and 7, the underlying base material layer can be made into a grid structure or a monolithic structure. From the perspective of process simplicity, it is preferable to make it into a uniform grid structure for both the narrow-spectrum modulation region and the wide-spectrum modulation region.

[0204] As those skilled in the art can anticipate, the addition of a base material layer reduces the energy utilization of light, a situation that is further highlighted in embodiments with a grid structure formed by light-absorbing materials.

[0205] Typically, the solution that comes to mind is to reduce the thickness of the base material layer. However, if the base layer is too thin, it will be impossible to effectively modulate the key bands.

[0206] Therefore, in one embodiment, experiments revealed that the thickness of the base layer ranges from 50 to 500 nm, and the thickness of the modulation layer ranges from 500 to 1000 nm. Within this range, the spectral modulation channels of the base layer and modulation layer are advantageous for fabrication and can achieve better spectral modulation performance. A base layer thickness exceeding 500 nm increases the optical path length of the modulation channel, leading to spectral crosstalk between adjacent pixels in the image sensing layer. For the modulation layer, if its thickness is less than 500 nm, it loses its broadband modulation capability and cannot achieve the ideal modulation spectral line shape; while if the modulation layer thickness exceeds 1000 nm, it makes the fabrication process difficult and, in addition to increasing spectral crosstalk, also reduces the transmittance of the modulation spectral lines.

[0207] However, this overall adjustment method does not take into account the differences between different broadband modulation sub-unit materials (e.g., different wavelengths of light scatter the same material to different degrees). A more preferred embodiment is given below.

[0208] In embodiments where the base material layer 30 is a monolithic structure, within the same broadband modulation unit (e.g. Figure 2 , Figure 4 , Figure 6 In a broadband modulation unit (as shown), the thickness of the base material layer corresponding to different broadband modulation subunits (different filter materials) is stepped. For example, as... Figure 12 As shown, where Figure 12 (a) is a top view of a broadband modulation unit, and (b) is a cross-sectional view along the dashed line in the figure. Figure 12 As shown in (b), the thickness of the base material region corresponding to C1 is greater than that corresponding to C2, C3, C4, C5, C6, C7, C8, and C9. The overall base material layer is stepped.

[0209] As will be discussed in the process section later, the thickness varies depending on which broadband modulation subunit is fabricated first; the thickness of the corresponding base material region fabricated first is greater than that of the corresponding base material region fabricated later. Of course, the overall thickness range of the base layer can still be controlled between 50-500 nm.

[0210] In the specific process, broadband modulation sub-units corresponding to wavelength ranges insensitive to scattering by the base layer material can be selected as C1 (processed first), and broadband modulation sub-units corresponding to wavelength ranges sensitive to scattering by the base layer material can be selected as C9 (processed later). This addresses the crosstalk issue. Simultaneously, for subsequent device flatness, the sum of the thicknesses of each broadband modulation sub-unit and its corresponding base material layer region in the projection relationship is equal. Of course, the overall thickness of the modulation layer can still be controlled within 500-1000 nm. Thus, with the same total thickness, the thickness ratio of each broadband modulation sub-unit to its corresponding base layer material can vary, equivalent to variations in the material ratio in the vertical direction across different channels. This increases the means to adjust the transmittance curve shape, allowing for more precise control over obtaining the desired curve shape.

[0211] Those skilled in the art will understand that, for example Figure 2 , Figure 4 , Figure 6 The thickness of the base material layer below the narrow-spectrum modulation region shown is not required and can be of the same thickness.

[0212] In the above-described mesh structure implementation, the above concept can also be realized. In the entire spectral chip, the sum of the thickness of the base material in all meshes and the thickness of the corresponding broadband modulation subunits are equal; and in the same filter unit, the thickness of the base material corresponding to different broadband modulation subunits is stepped.

[0213] On the other hand, regardless of the transmittance curve of existing spectral chips ( Figure 8 ) or the transmittance curve of the spectral chip after adding the aforementioned basic material layer of this application ( Figure 11 All of these exhibit aliasing in at least some channels within certain wavelength ranges. This aliasing becomes more severe as the number of channels increases.

[0214] Figure 13 and Figure 14 In respectively Figure 8 and Figure 11 Based on this, the corresponding aliasing region is shown (this phenomenon exists even in the aforementioned preset wavelength region after the addition of the base material layer), which leads to low spectral resolution, making it difficult to accurately identify and modulate the spectral signal, thus limiting the application of the spectral chip in complex environments.

[0215] Typically, when fabricating broadband modulation subunits, the desired ideal morphology is vertical or nearly vertical. Researchers have explored various methods for fabricating vertical or near-vertical broadband modulation subunits. However, the inventors of this application have discovered through experimentation that when the edges of the broadband modulation subunits are inclined, meaning that adjacent broadband modulation subunits have overlapping areas (inclined interfaces), Figure 13 or Figure 14 The waveform in the aliasing region shown will demix, which gives the inventors a new idea to further optimize the transmittance curve.

[0216] Therefore, in one embodiment of this application, in the broadband modulation unit of the broadband modulation region, there is at least one inclined overlapping area relative to the silicon substrate between each broadband modulation subunit and its adjacent broadband modulation subunit, and the orthographic projection of the inclined overlapping area on the silicon substrate crosses two adjacent pixels on the image sensing layer.

[0217] In a preferred embodiment Figure 15 (a) is a broadband modulation unit (e.g.) Figure 2 , Figure 4 , Figure 6 The figure shows a top view of a broadband modulation unit, and 15(b) is a cross-sectional view along the dashed line in the figure. Figure 15 As can be seen in (b), C2, C4, C6 and C8 are regular trapezoids (corresponding to a regular square frustum in the concept of solids), and C1, C3, C5, C7 and C9 are inverted trapezoids (corresponding to an inverted square frustum in the concept of solids).

[0218] Please note that "positive" and "inverted" are relative concepts. In this application, a "positive square truncated pyramid" refers to a pyramid whose surface area closer to the image sensing layer is larger than the surface area farther from the image sensing layer. That is, relative to the image sensing layer, the broadband modulation subunit is smaller at the top and larger at the bottom. An "inverted square truncated pyramid" refers to a pyramid whose surface area closer to the image sensing layer is smaller than the surface area farther from the image sensing layer. That is, relative to the image sensing layer, the broadband modulation subunit is larger at the top and smaller at the bottom.

[0219] Such a structure can achieve such a modulation effect, such as Figure 16 As shown, taking C1-C3 as an example, the light entering pixel P1 is modulated by C1 and C2 (the R1 area to the left of the dashed line L1) (and also by the underlying base material layer below, the same below); the light entering pixel P3 is modulated by C3 and C2 (the R2 area to the right of the dashed line L2); the light entering pixel P2 is modulated by C1 (the R3 area to the right of the dashed line L3), C2, and C3 (the R4 area to the left of the dashed line L4).

[0220] The above examples use regular and inverted square truncated pyramids as illustrations; however, this application is not limited to these. Any truncated pyramid structure is acceptable. That is, a broadband modulation subunit and its adjacent broadband modulation subunits need at least one overlapping area with a slope to achieve a co-modulation effect. For example, C1 and C2, C4 each have overlapping areas with slopes. Those skilled in the art, guided by the teachings of this application, can adjust the area and slope of the overlapping areas to ensure that the light entering each pixel is co-modulated by at least two different modulation subunits, thereby finely adjusting the linear shape of the light transmittance curve entering the pixel and alleviating aliasing at certain wavelengths.

[0221] Those skilled in the art will understand that, for Figure 2 , 4 Regarding the narrow-spectrum modulation region in the layouts shown in 6 and 7, adjacent narrow-spectrum modulation sub-units (e.g., RGB materials) can be fabricated in the same fabrication process as the aforementioned wide-spectrum modulation region, so that they also have overlapping regions with inclined surfaces (e.g., regular square truncated pyramids and inverted square truncated pyramids), or they can be fabricated separately. This application does not limit this.

[0222] Furthermore, although the truncated pyramidal broadband modulation subunit coexists with the base material layer in the above example, this application is not limited thereto; the truncated pyramidal broadband modulation subunit can be used independently (i.e., in...). Figure 1 The structure shown has a broadband modulation subunit (which is frustum-shaped) to achieve its effect. Specifically, the transmittance curve of each channel can be more precisely adjusted by changing the area and slope of the overlap region, thereby alleviating aliasing of the transmittance curve at certain wavelengths.

[0223] In a preferred example, the angle between the beveled overlap area and the surface of the spectral modulation layer is between 60 and 90 degrees.

[0224] Figure 17 The figure shows the transmittance curve of light incident on the broadband adjustment region of the spectral chip after being modulated by the spectral modulation layer in the case of a spectral chip structure without a base material layer, which satisfies the following constraint:

[0225] TC1: Valley transmittance wavelength is 400-550nm; T(430nm-495nm)≤10%; T(550nm-900nm)≥70%;

[0226] TC2: Valley transmittance wavelength is 400-575nm; T (420nm-550nm) ≤15%; T (585nm-900nm) ≥65%;

[0227] TC3: Valley transmittance wavelength is 400-580nm; T(455nm-550nm)≤15%; T(600nm-900nm)≥75%;

[0228] TC4: Valley transmittance wavelength is 500-680nm; T(400nm-500nm) ≥ 65%; T(580nm-645nm) ≤ 10%; T(700nm-900nm) ≥ 65%;

[0229] TC5: Valley transmittance wavelength is 500-675nm; T(400nm-500nm) ≥50%; T(550nm-650nm) ≤10%; T(700nm-900nm) ≥60%;

[0230] TC6: Valley transmittance wavelength is 550-700nm; T(400nm-535nm)≥65%; T(600nm-665nm)≤10%; T(700nm-900nm)≥75%;

[0231] TC7: Valley transmittance wavelength is 550-750nm; T(400nm-550nm) ≥65%; T(630nm-725nm) ≤10%; T(800nm-900nm) ≥75%;

[0232] TC8: Valley transmittance wavelength is 550-750nm; T(460nm-545nm)≥55%; T(625nm-740nm)≤10%; T(775nm-900nm)≥55%;

[0233] TC9: Valley transmittance wavelength is 600-700nm; T(465nm-540nm)≥70%; T(680nm-745nm)≤10%; T(785nm-900nm)≥70%.

[0234] Compare Figure 17 and Figure 13 It is evident that the combination of regular and inverted square pyramid shapes helps to separate partially overlapping transmittance curves, thereby improving the distinction between different spectra and enhancing the effect of spectral modulation.

[0235] Figure 18 The figure shows the transmittance curve of light incident on the broadband adjustment region of the spectral chip, modulated by the spectral modulation layer and the base material layer, in the case of a spectral chip structure with a base material layer, satisfying the following constraints:

[0236] TA1: Peak transmittance wavelength is 520-640nm; T(420nm-495nm)≤10%; T(580nm-625nm)≥55%; T(670nm-900nm)≥50%;

[0237] TA2: Peak transmittance wavelength is 570-645nm; T(425nm-550nm) ≤10%; T(600nm-630nm) ≥70%; T(670nm-900nm) ≥55%;

[0238] TA3: Peak transmittance wavelength is 575-645nm; T(460nm-555nm)≤15%; T(580nm-620nm)≥70%; T(660nm-900nm)≥55%;

[0239] TA4: Valley transmittance wavelength is 550-650nm; T (400nm-500nm) ≥ 45%; T (580nm-645nm) ≤ 10%; T (700nm-900nm) ≥ 50%;

[0240] TA5: Valley transmittance wavelength is 550-675nm; T(425nm-500nm) ≥35%; T(550nm-665nm) ≤10%; T(700nm-900nm) ≥45%;

[0241] TA6: Valley transmittance wavelength is 500-650nm; T(350nm-450nm)≤40%; T(485nm-615nm)≥50%; T(800nm-900nm)≥60%;

[0242] TA7: Valley transmittance wavelength is 575-750nm; T(400nm-570nm) ≥45%; T(630nm-730nm) ≤10%; T(775nm-900nm) ≥50%;

[0243] TA8: Valley transmittance wavelength is 600-750nm; T(460nm-565nm)≥40%; T(620nm-745nm)≤10%; T(800nm-900nm)≥45%;

[0244] TA9: Valley transmittance wavelength is 600-760nm; T(460nm-580nm)≥50%; T(635nm-645nm)≤15%; T(800nm-900nm)≥50%.

[0245] Compare Figure 18 and Figure 14It is evident that the combination of regular and inverted square truncated pyramid shapes not only helps to separate transmittance curves that are partially overlapped outside the preset wavelength range, but also helps to separate transmittance curves that are partially overlapped within the preset wavelength range, thereby improving the distinguishability between different spectra and enhancing the effect of spectral modulation.

[0246] Furthermore, by combining regular and inverted truncated square shapes, the transmittance of the broadband modulation region of the spectral chip within the preset wavelength range is also relatively improved.

[0247] In summary, this structure not only optimizes spectral discrimination but also improves the accuracy of spectral modulation, enabling the spectral chip to perform effective modulation and identification over a wider wavelength range. Through this structural optimization, the overall performance of the spectral chip is enhanced, allowing it to better handle complex spectral signal modulation requirements.

[0248] As mentioned earlier, it is often difficult to form a perfect cube or cuboid structure in the ideal broadband modulation subunit film layer in the process. There will always be some slope. The inventors of this application have taken advantage of this defect and amplified it by deliberately making it into the shape of a regular square truncated pyramid and an inverted square truncated pyramid to form an overlapping area. This is easier to achieve in the process preparation, thereby further improving the preparation accuracy and quality.

[0249] Furthermore, the above examples use 500-650nm as the key wavelength band; however, this application is not limited to this. For example, if considering applications in the ultraviolet spectral imaging field, such as surface defect detection, fluorescence imaging, forensic medicine and security (e.g., bloodstain and fingerprint recognition), and cultural relic identification and restoration, the key wavelength band is between 200-400nm. If considering applications in the infrared spectral imaging field, such as agriculture, food inspection, medical imaging, security monitoring, and material sorting, the key wavelength band is between 800-1700nm. Based on the teachings of this application, the transmittance curves of the corresponding basic material layers can be designed for the corresponding wavelength bands.

[0250] A second aspect of this application provides a spectral camera, comprising:

[0251] The spectral chip described in the first aspect;

[0252] The calculation unit is used to obtain the color temperature value of the ambient light of the environment where the target is located based on the first image data and to perform white balance processing on the second image data based on the color temperature value.

[0253] In one example, the computing unit is a computing chip.

[0254] Optionally, the spectroscopic camera may also include a microlens array formed on the spectral modulation layer.

[0255] A third aspect of this application provides a white balance processing method using the aforementioned spectral camera, such as... Figure 19 As shown, it includes:

[0256] S10. Take a picture of the target using a spectral camera;

[0257] S20. Obtain the first image data of the target through the broadband modulation region;

[0258] S30. Obtain the second image data of the target through the narrow-spectrum modulation region;

[0259] S40. Obtain the color temperature value of the ambient light of the environment where the target is located based on the first image data;

[0260] S50. Perform white balance processing on the second image data based on the color temperature value.

[0261] for Figure 2 The layout shown is illustrated; for details of the method, please refer to [reference needed]. Figure 3 ;for Figure 4 , Figure 6 The layout shown is illustrated; for details of the method, please refer to [reference needed]. Figure 5 The method of the third aspect of this application is also applicable. Figure 7 The layout.

[0262] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, additions, and sub-combinations therein.

[0263] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of this application.

[0264] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A spectroscopic chip, characterized by include: Silicon-based substrate, including the image sensing layer; A spectral modulation layer is disposed on the image sensing layer, wherein the spectral modulation layer comprises: Broadband modulation region, to acquire the first image data of the target; Narrow-spectrum modulation region, to acquire the second image data of the target. The first image data is used to perform white balance processing on the second image data.

2. The spectral chip according to claim 1, characterized in that, The broadband modulation region is located around the periphery of the spectral chip, surrounding the narrowband modulation region. The global color temperature value of the ambient light of the target's environment is obtained based on the first image data, and the global color temperature value is used to perform global white balance correction on the second image data.

3. The spectral chip according to claim 2, characterized in that, The broadband modulation region includes multiple broadband modulation units arranged in arrays along the row and column directions, respectively. Each broadband modulation unit is composed of broadband modulation sub-units with different transmittance curves. The narrowband modulation region includes multiple narrowband modulation units arranged in an array along the row and column directions, respectively. Each narrowband modulation unit includes a narrowband modulation subunit composed of at least two of the following: green filter material, red filter material, blue filter material, yellow filter material, and near-infrared filter material.

4. The spectral chip according to claim 3, characterized in that, The broadband modulation unit is a 3*3 broadband modulation subunit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; The narrowband modulation unit is a 2*2 narrowband modulation substructure made of green filter material, red filter material and blue filter material, wherein the two materials on one diagonal of the 2*2 structure are both green filter materials.

5. The spectral chip according to claim 1, characterized in that, The broadband modulation region is distributed across the spectral chip, and the remaining area is the narrowband modulation region. Each broadband modulation region includes a broadband modulation unit, and each broadband modulation unit consists of broadband modulation subunits with different transmittance curves; The narrowband modulation region includes multiple narrowband modulation units arranged in arrays along the row and column directions, respectively. Each narrowband modulation unit includes a narrowband modulation subunit composed of at least two of the following: green filter material, red filter material, blue filter material, yellow filter material, and near-infrared filter material. The local color temperature value of the ambient light of the target's environment is obtained based on the first image data of each broadband modulation zone, and the local color temperature value is used to perform local white balance correction on the second image data obtained by the narrowband modulation unit within a preset range of the broadband modulation zone.

6. The spectral chip according to claim 5, characterized in that, The broadband modulation unit is a 3*3 broadband modulation subunit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; The narrowband modulation unit is a 2*2 narrowband modulation subunit structure made of green and red filter materials, wherein the two materials on one diagonal of the 2*2 structure are green filter materials and the two materials on the other diagonal are red filter materials.

7. The spectral chip according to claim 5, characterized in that, The broadband modulation unit is a 3*3 broadband modulation subunit structure made of 9 different materials, denoted as C1, C2, C3, C4, C5, C6, C7, C8 and C9 respectively; The narrowband modulation unit includes a first narrowband modulation subunit and a second narrowband modulation subunit, wherein the first narrowband modulation subunit is a 2*2 narrowband modulation subunit structure made of red filter material, and the second narrowband subunit is a 2*2 structure made of green filter material.

8. The spectral chip according to claim 1, characterized in that, The spectral modulation layer includes periodic units arranged in an array along the row and column directions, respectively. The periodic units include at least two of green, red, and blue filter materials, as well as multiple broadband modulation materials with different transmittance curves. The green, red, and blue filter materials constitute the narrow-spectrum modulation region, and the broadband modulation materials constitute the broadband modulation region.

9. The spectral chip of claim 8, wherein, The periodic unit has a 4x4 structure, wherein, The first row, along the first direction, consists of red filter material, green filter material, first spectral modulation material C1, and green filter material in sequence; The second row, along the first direction, consists of green filter material, blue filter material, green filter material, and second spectral modulation material C2; The third row, along the first direction, consists of the third spectral modulation material C3, the green filter material, the fourth spectral modulation material C4, and the green filter material. The fourth row, along the first direction, consists of green filter material, fifth spectral modulation material C5, green filter material, and sixth spectral modulation material C6.

10. The spectroscopy chip according to any one of claims 3-9, wherein, The broadband modulation region material is a colloid composed of resin material, photoinitiator material, pigment, and solvent material.

11. A spectral camera, characterized by include: The spectral chip according to any one of claims 1-10; The calculation unit is used to obtain the color temperature value of the ambient light of the environment where the target is located based on the first image data and to perform white balance processing on the second image data based on the color temperature value.

Images