Spectral imaging chip and device, information processing method, and fingerprint living body recognition device and fingerprint module

Abstract

The present application provides a kind of spectral imaging chip and equipment and information processing method, and fingerprint living body recognition device and fingerprint module.Spectral imaging chip can obtain the spectral information of the object while not affecting the spatial resolution and imaging quality of the image formed, facilitate to grasp the more comprehensive information of the object to be imaged.Fingerprint living body recognition device and fingerprint module can realize fingerprint living body recognition through spectral imaging chip, which is conducive to improving the stability of device performance, while reducing the volume, weight and cost of spectral device, greatly improving the anti-counterfeiting ability of fingerprint recognition system.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and in particular to a spectral imaging chip, a spectral imaging device and its information processing method, as well as a fingerprint liveness detection device and fingerprint module using the spectral imaging chip and information processing method. Background Technology

[0002] Currently, imaging chips and imaging devices are widely used. Taking a camera as an example, it contains an image sensor (e.g., a CMOS image sensor or a CCD sensor) to acquire image information of the subject, such as the RGB color information of the subject.

[0003] However, regardless of whether a CMOS image sensor or a CCD sensor is used as the imaging chip, it can only acquire image information of the subject, but not its spectral information. In other words, existing imaging chips and devices cannot obtain the spectral information of objects, preventing the resulting images from being widely applicable to scenarios requiring spectral information as data support, such as intelligent AI recognition and qualitative and quantitative analysis of material components.

[0004] In addition, in modern information society, there are many scenarios in daily life that require users to verify their identity with fingerprints, which greatly improves the convenience of people's lives.

[0005] Existing technologies often verify a user's identity through fingerprint image information, but such technologies can be cracked by stickers, 3D printed models, etc., and their security cannot be guaranteed. Summary of the Invention

[0006] One advantage of this application is that it provides a spectral imaging chip and its fabrication and information processing methods, wherein the spectral imaging chip is able to obtain image information and / or spectral information of the subject through a special chip structure.

[0007] To achieve the above advantages, this application provides a spectral chip, which includes:

[0008] The image sensing layer includes multiple sensing units; and

[0009] A light modulation layer located on the light-sensing path of the image sensor, the light modulation layer including at least one modulation unit and at least one non-modulation unit spaced apart from the at least one modulation unit, wherein each of the modulation units corresponds to at least one of the sensing units along the light-sensing path, and each of the non-modulation units corresponds to at least one of the sensing units along the light-sensing path.

[0010] The modulation unit of the optical modulation layer is configured to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the spectral information of the imaging light; the non-modulation unit of the optical modulation layer is configured not to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light.

[0011] In the spectral imaging chip according to this application, the at least one modulation unit and the at least one non-modulation unit are located in the same plane.

[0012] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for a proportion of the area of ​​the effective region of the light modulation layer that is greater than 0 and less than or equal to 25%, wherein the effective region of the light modulation layer is the region defined by the modulation unit and the non-modulation unit of the light modulation layer.

[0013] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for a proportion of the area of ​​the effective region of the light modulation layer that is greater than or equal to 5 and less than or equal to 15%.

[0014] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than 25% and less than or equal to 45% of the area of ​​the effective region of the light modulation layer, wherein the effective region of the light modulation layer is the region defined by the modulation unit and the non-modulation unit of the light modulation layer.

[0015] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than 25% and less than or equal to 35% of the area of ​​the effective region of the light modulation layer.

[0016] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than 45% and less than or equal to 60% of the area of ​​the effective region of the light modulation layer, wherein the effective region of the light modulation layer is the region defined by the modulation unit and the non-modulation unit of the light modulation layer.

[0017] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than or equal to 50% and less than or equal to 55% of the area of ​​the effective region of the light modulation layer.

[0018] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than 60% of the area of ​​the effective region of the light modulation layer, wherein the effective region of the light modulation layer is the region defined by the modulation unit and the non-modulation unit of the light modulation layer.

[0019] In the spectral imaging chip according to this application, the area of ​​the modulation unit of the light modulation layer accounts for more than or equal to 80% and less than or equal to 95% of the area of ​​the effective region of the light modulation layer.

[0020] In the spectral imaging chip according to this application, the non-modulation unit of the light modulation layer is located in the edge region of the light modulation layer.

[0021] In the spectral imaging chip according to this application, the non-modulation units of the optical modulation layer are arranged symmetrically about the center of the optical modulation layer.

[0022] In the spectral imaging chip according to this application, the modulation unit of the light modulation layer is located in the middle region of the light modulation layer.

[0023] In the spectral imaging chip according to this application, the non-modulation unit of the light modulation layer is located in the edge region of the light modulation layer.

[0024] In the spectral imaging chip according to this application, the spectral imaging chip further includes a filter layer formed at the non-modulation unit of the optical modulation layer.

[0025] In the spectral imaging chip according to this application, the spectral imaging chip further includes a filter layer formed at the non-modulation unit of the optical modulation layer.

[0026] In the spectral imaging chip according to this application, the sparsity of the light modulation layer is greater than 0 and less than or equal to 0.5.

[0027] In the spectral imaging chip according to this application, the sparsity of the light modulation layer is greater than 0 and less than or equal to 0.15.

[0028] In the spectral imaging chip according to this application, the sparsity of the light modulation layer is greater than or equal to 0.5 and less than 1.

[0029] In the spectral imaging chip according to this application, the sparsity of the light modulation layer is greater than or equal to 0.85 and less than 1.

[0030] In the spectral imaging chip according to this application, the spectral imaging chip further includes: a signal processing circuit layer electrically connected to the image sensing layer.

[0031] In the spectral imaging chip according to this application, the spectral imaging chip further includes: a microlens layer of modulation units formed on the filter layer and / or the light modulation layer.

[0032] In the spectral imaging chip according to this application, the spectral imaging chip further includes: a light-transmitting medium layer integrally formed on the image sensing layer, wherein the upper surface of the light-transmitting medium layer is a flat surface.

[0033] In the spectral imaging chip according to this application, the light modulation layer is integrally bonded to the light-transmitting medium layer.

[0034] In the spectral imaging chip according to this application, the difference in refractive index between the light modulation layer and the light-transmitting mechanism layer is greater than or equal to 0.5.

[0035] In the spectral imaging chip according to this application, the light modulation layer is integrally bonded to the light-transmitting medium layer through a bonding layer, and the light-transmitting medium layer is bonded to the bonding layer.

[0036] In the spectral imaging chip according to this application, the light-transmitting medium layer and the bonding layer have the same material.

[0037] In the spectral imaging chip according to this application, each of the modulation units includes a plurality of modulation sub-units disposed along the photosensitive path.

[0038] In the spectral imaging chip according to this application, each of the modulation sub-units has a porous structure or a columnar structure.

[0039] In the spectral imaging chip according to this application, the cross-sectional shape of the aperture of different modulation subunits having aperture structures in each modulation unit is not exactly the same; and / or, the structural parameters of different modulation subunits having aperture structures in each modulation unit are not exactly the same.

[0040] According to another aspect of this application, an information processing method suitable for a spectral imaging chip is also provided, comprising:

[0041] Acquire the spectral information of the pixels corresponding to each modulation unit in the modulation layer of the spectral imaging chip, and the light intensity information of the pixels corresponding to each non-modulation unit in the modulation layer; and

[0042] Based on the spectral information of the pixel corresponding to each modulation unit, the spectral information of the subject is determined, and based on the light intensity information of the pixel corresponding to each non-modulation unit, the image information of the subject is determined.

[0043] In the information processing method suitable for a spectral imaging chip according to this application, the modulation unit of the light modulation layer is configured to modulate the imaging light entering the corresponding pixel, and the corresponding pixel is adapted to acquire the spectral information of the imaging light; wherein, the non-modulation unit of the light modulation layer is configured not to modulate the imaging light entering the corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light.

[0044] In the information processing method suitable for a spectral imaging chip according to this application, the modulation unit of the light modulation layer is located in the middle region of the light modulation layer.

[0045] In the information processing method suitable for a spectral imaging chip according to this application, the non-modulation unit of the optical modulation layer is located in the edge region of the optical modulation layer.

[0046] In the information processing method suitable for a spectral imaging chip according to this application, determining the image information of the subject based on the light intensity information of the pixel corresponding to each of the non-modulation units includes: determining the fitted light intensity information of the pixel corresponding to each of the modulation units based on the light intensity information of the pixel corresponding to at least one of the non-modulation units around each of the modulation units; and determining the image information of the subject based on the fitted light intensity information of the pixel corresponding to each of the modulation units and the light intensity information of the pixel corresponding to each of the non-modulation units.

[0047] In the information processing method suitable for a spectral imaging chip according to this application, determining the fitted light intensity information of the pixel corresponding to each modulation unit based on the light intensity information of the pixel corresponding to at least one non-modulation unit around each modulation unit includes: filtering the light intensity information of the pixel corresponding to at least one non-modulation unit around any modulation unit based on a smoothing filtering method to obtain the fitted light intensity information of the pixel corresponding to any modulation unit.

[0048] In the information processing method suitable for spectral imaging chips according to this application, determining the image information of the subject based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit includes: inputting an initial image obtained based on the light intensity information of the pixel corresponding to all non-modulation units into a fitting model to obtain the image information of the subject output by the fitting model; wherein, the fitting model is constructed based on an adversarial neural network, and the fitting model is trained based on the labels of empty sample images with blank pixels and complete sample images without blank pixels corresponding to the empty sample images.

[0049] In the information processing method suitable for spectral imaging chips according to this application, the training process of the fitting model includes: training the generator in the adversarial neural network based on the missing sample images and the complete sample image labels corresponding to the missing sample images, and performing competitive discrimination on the trained generator based on the discriminator in the adversarial neural network; and using the trained generator as the fitting model.

[0050] Another advantage of this application is that it provides a spectral imaging device, which includes: a spectral imaging chip and a signal processing circuit module, wherein the spectral imaging chip and the signal processing circuit module are two independent parts or the signal processing circuit module is integrated into the spectral imaging chip;

[0051] The spectral imaging chip includes: a light modulation layer and an image sensing layer stacked sequentially along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing units.

[0052] According to the spectral imaging device of the present invention, each modulation unit in the optical modulation layer includes multiple modulation sub-units, and each modulation sub-unit is a porous structure or a columnar structure.

[0053] According to the spectral imaging apparatus of the present invention, the cross-sectional shape of the aperture in different modulation sub-units of the aperture structure in each modulation unit is not exactly the same; and / or,

[0054] The structural parameters of different modulation subunits with the aperture structure in each modulation unit are not exactly the same.

[0055] According to the spectral imaging device of the present invention, the structural shapes and column heights of the different modulation subunits of the columnar structure in each modulation unit are the same, and the arrangement of all modulation subunits in each modulation unit has C4 symmetry.

[0056] According to the spectral imaging device of the present invention, the modulation subunit of the columnar structure is integrally formed or is composed of multiple layers of modulation columns stacked together.

[0057] Another advantage of this invention is that it provides a fingerprint liveness detection device for determining whether the object to be detected is the user's own live fingerprint.

[0058] To achieve the above advantages, this application provides a fingerprint liveness recognition device, comprising: a display screen, a light source module, a spectral imaging chip, a signal processing circuit module, and a recognition module. The display screen has a fingerprint detection area. The light source module illuminates the object to be detected within the fingerprint detection area, and the target light beam, reflected by the object, is incident on the spectral imaging chip. The spectral imaging chip and the signal processing circuit module are either two independent parts or the signal processing circuit module is integrated within the spectral imaging chip. The spectral imaging chip determines the spectral information of each pixel corresponding to a modulation unit in the light modulation layer of the spectral imaging chip after illumination by the target light beam, and the light intensity information of each pixel corresponding to a non-modulation unit in the light modulation layer. The signal processing circuit module determines the spectral information of the object to be detected based on the spectral information and the image information of the object to be detected based on the light intensity information. The recognition module identifies whether the object to be detected is a live fingerprint of a target user based on the spectral information and the image information of the object to be detected.

[0059] According to an embodiment of the fingerprint liveness recognition device of the present invention, the spectral imaging chip includes: a light modulation layer and an image sensing layer sequentially stacked along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has a plurality of sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing unit.

[0060] According to an embodiment of the fingerprint liveness recognition device of the present invention, the signal processing circuit module is specifically used to: determine the fitted light intensity information of the pixel corresponding to each modulation unit based on the light intensity information of the pixel corresponding to the multiple non-modulation units around each modulation unit after the target beam is irradiated;

[0061] Based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit, the image information of the object to be detected is determined.

[0062] According to an embodiment of the fingerprint liveness recognition device of the present invention, the signal processing circuit module is specifically used to: filter the light intensity information of at least one non-modulation unit corresponding to any modulation unit based on a smoothing filtering method to obtain the fitted light intensity information of the pixel corresponding to any modulation unit.

[0063] According to an embodiment of the fingerprint liveness recognition device of the present invention, the signal processing circuit module is further specifically used for:

[0064] The initial image obtained by the spectral imaging chip after all non-modulation units are illuminated by the target beam is input into the fitting model to obtain the image information of the object to be detected output by the fitting model;

[0065] The fitting model is constructed based on an adversarial neural network and is trained on empty sample images containing blank pixels and complete sample images corresponding to the empty sample images that do not contain blank pixels.

[0066] According to an embodiment of the fingerprint liveness recognition device of the present invention, each modulation unit in the optical modulation layer includes multiple modulation sub-units, and each modulation sub-unit is a hole-shaped structure or a columnar structure.

[0067] According to one embodiment of the fingerprint liveness recognition device of the present invention, the cross-sectional shape of the aperture cross-section of different modulation sub-units of the aperture structure in each modulation unit is not exactly the same; and / or,

[0068] The structural parameters of different modulation subunits with the aperture structure in each modulation unit are not exactly the same.

[0069] According to an embodiment of the fingerprint liveness recognition device of the present invention, the structural shapes and column heights of the different modulation sub-units of the columnar structure in each modulation unit are the same, and the arrangement of all modulation sub-units in each modulation unit has C4 symmetry.

[0070] According to an embodiment of the fingerprint liveness recognition device of the present invention, the modulation subunit of the columnar structure is integrally formed or is formed by stacking multiple modulation columns.

[0071] According to an embodiment of the fingerprint liveness recognition device of the present invention, it further includes: a lens group; the light source module is disposed between the display screen and the spectral imaging chip, and the lens group is disposed between the light source module and the spectral imaging chip; or, the light source module is disposed below the non-fingerprint detection area of ​​the display screen, and the lens group and the spectral imaging chip are sequentially disposed below the fingerprint detection area;

[0072] The lens group is used to collimate the target beam onto the spectral imaging chip so that the target beam is imaged on the spectral imaging chip.

[0073] The present invention further provides a fingerprint module, comprising: a spectral imaging chip and a circuit board, wherein the spectral imaging chip is electrically connected to the circuit board;

[0074] The spectral imaging chip comprises a light modulation layer, an image sensing layer, and a signal processing circuit module stacked sequentially along the thickness direction. The light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface. The image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction.

[0075] According to one embodiment of the fingerprint module, the further comprising: a reinforcing plate, the reinforcing plate being attached to the circuit board.

[0076] According to one embodiment of the fingerprint module, the packaged portion is formed on the upper surface of the circuit board.

[0077] According to one embodiment of the fingerprint module, the light-shielding part is disposed on the upper surface of the encapsulation part.

[0078] The further objectives and advantages of this application will become fully apparent from the following description and accompanying drawings. Attached Figure Description

[0079] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0080] Figure 1 The diagram illustrates a flowchart of the information processing method provided in this application;

[0081] Figure 2 The figure shows a schematic diagram of the spectral imaging device provided in this application;

[0082] Figure 3 The figure shows a schematic diagram of the specific structure of the spectral imaging chip provided in this application;

[0083] Figure 4 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0084] Figure 5 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0085] Figure 6 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0086] Figure 7The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0087] Figure 8 This is a schematic diagram of the optical modulation layer in the spectral imaging chip provided by the present invention;

[0088] Figure 9 This is a schematic diagram of the optical modulation layer in the spectral imaging chip provided by the present invention;

[0089] Figure 10 The figure shows a schematic diagram of the structural shape of the modulation column in the spectral imaging chip provided in this application;

[0090] Figure 11 The figure shows a schematic diagram of the longitudinal cross-sectional shape of the modulation column in the spectral imaging chip provided in this application;

[0091] Figure 12 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0092] Figure 13 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0093] Figure 14 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0094] Figure 15 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0095] Figure 16 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0096] Figure 17 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0097] Figure 18 The figure shows a schematic diagram of the optical modulation layer in the spectral imaging chip provided in this application;

[0098] Figure 19 The figure shows a schematic diagram of the structure of the CIS wafer in the spectral imaging chip provided in this application;

[0099] Figure 20 The figure shows a schematic diagram of the structure of the CIS wafer in the spectral imaging chip provided in this application;

[0100] Figure 21a The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0101] Figure 21b The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0102] Figure 22 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0103] Figure 23 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0104] Figure 24 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0105] Figure 25 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0106] Figure 26 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0107] Figure 27 The figure shows a schematic diagram of the structure of the spectral imaging chip provided in this application;

[0108] Figure 28 The illustration shows a schematic diagram of the sparsity of the optical modulation layer of the spectral imaging chip according to an embodiment of this application;

[0109] Figure 29 The illustration shows a schematic diagram of the fabrication process of the spectral imaging chip according to an embodiment of this application;

[0110] Figure 30 The illustration shows a schematic diagram of a modified embodiment of the fabrication process of the spectral imaging chip according to an embodiment of this application;

[0111] Figure 31a This is a schematic diagram of the fingerprint liveness recognition device provided by the present invention;

[0112] Figure 31b This is a top view of the fingerprint liveness recognition device provided by the present invention;

[0113] Figure 32 This is a schematic diagram of the fingerprint liveness recognition device provided by the present invention;

[0114] Figure 33 This is a schematic diagram of the fingerprint liveness recognition device provided by the present invention;

[0115] Figure 34 This is a schematic diagram of the structure of the optical fingerprint recognition component provided by the present invention;

[0116] Figure 35 This is a schematic diagram of the fingerprint module provided by the present invention;

[0117] Figure 36 This is a schematic diagram of the fingerprint module provided by the present invention. Detailed Implementation

[0118] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0119] Application Overview

[0120] As mentioned earlier, existing CMOS or CCD imaging chips and imaging devices cannot acquire the spectral information of objects, resulting in images that cannot be widely used in scenarios such as intelligent AI recognition and qualitative and quantitative analysis of material components that require the spectral information of objects as data support.

[0121] Those skilled in the art will know that light interacts with matter through processes such as absorption, scattering, fluorescence, and Raman spectroscopy, producing specific spectra, and each substance's spectrum is unique. In current methods, the spectral information of the subject requires specialized equipment (e.g., a spectrometer) to acquire. Therefore, when it is necessary to simultaneously obtain the spectral and image information of the subject, multiple camera modules and / or devices are often required to work together, and algorithms are used to integrate the acquired image and spectral information.

[0122] It is understandable that the need for multiple camera modules and / or devices increases costs. More importantly, this solution occupies a relatively large space, resulting in poor portability and integrability; that is, it is difficult to integrate it into terminal devices.

[0123] Based on this, the inventors of this application, through research on traditional photosensitive chips and spectral information measurement technologies, have proposed a spectral imaging chip that, through a special chip structure, can acquire image information and / or spectral information of the subject. Compared to traditional CMOS or CCD imaging chips, this spectral imaging chip can acquire both the spectral information and the image information of the subject, thereby effectively integrating these two types of information.

[0124] Furthermore, compared to traditional imaging chips, the spectral imaging chip according to this application can obtain the spectral information of the subject without affecting or minimally affecting the resolution of the subject's image information. With appropriate adjustments, the spectral imaging chip according to this application can obtain at least a portion of the subject's image information without affecting or minimally affecting the subject's spectral information.

[0125] It should be understood that since the spectral information of the subject can uniquely identify it, qualitative or quantitative analysis of the target can be performed based on this information. For example, the spectral imaging chip described in this application can be applied to fields such as fruit freshness analysis, air pollution level analysis, and AI scene recognition. In other words, compared to traditional CMOS or CCD imaging chips, the spectral imaging chip described in this application has a wider range of applications.

[0126] Based on this, this application provides a spectral imaging chip, comprising: an image sensing layer including a plurality of sensing units; and a light modulation layer located on the photosensitive path of the image sensor, the light modulation layer including at least one modulation unit and at least one non-modulation unit spaced apart from the at least one modulation unit, wherein each modulation unit corresponds to at least one sensing unit along the photosensitive path, and each non-modulation unit corresponds to at least one sensing unit along the photosensitive path; wherein the modulation unit of the light modulation layer is configured to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the spectral information of the imaging light; wherein the non-modulation unit of the light modulation layer is configured not to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light. The sensing unit can be one or more physical pixels of the imaging chip. Furthermore, taking one modulation unit corresponding to one physical pixel as an example, the modulation units can be arranged on the surface of the imaging chip at a certain period. For example, there are two non-modulation units between adjacent modulation units, which can also be understood as two physical pixels corresponding to non-modulation units between adjacent modulation units.

[0127] Based on this, this application also provides an information processing method suitable for a spectral imaging chip, which includes: acquiring spectral information of pixels corresponding to each modulation unit in the modulation layer of the spectral imaging chip and light intensity information of pixels corresponding to each non-modulation unit in the modulation layer; and determining spectral information of the subject based on the spectral information of the pixels corresponding to each modulation unit, and determining image information of the subject based on the light intensity information of the pixels corresponding to each non-modulation unit.

[0128] The modulation section mainly includes modulation units to modulate the incident light. The modulation section can be understood as having a broadband transmission spectrum, modulating the incident light, and receiving it from the sensing unit. That is, the modulation unit has a corresponding transmission spectrum curve to modulate the incident light. The non-modulation section, in different embodiments, mainly performs adjustment functions such as filtering, deflection, convergence, refraction, diffraction, diffusion, and / or collimation on the incident light. It can be implemented as a structure with specific adjustment functions, such as a filter, concave lens, convex lens, or optical diffraction. For example, the non-modulation unit is a filter unit that performs the filtering function. The filter unit can be an R, G, B, W, Y filter unit, for example, the filter unit can form an RGGB, RYYB, RGBW Bayer filter, or a single filter unit or multiple filter units combined to form an irregular Bayer filter. The non-modulation unit can also be without any structure, and the incident light can be directly received by the sensing unit across the entire wavelength range (without considering filters).

[0129] Based on this, this application further provides a spectral imaging device, which includes: a spectral imaging chip and a signal processing circuit module, wherein the spectral imaging chip and the signal processing circuit module are two independent parts or the signal processing circuit module is integrated within the spectral imaging chip; the spectral imaging chip includes: a light modulation layer and an image sensing layer sequentially stacked along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing units.

[0130] Based on this, this application also provides a fingerprint liveness detection device, comprising: a display screen, a light source module, a spectral imaging chip, a signal processing circuit module, and a recognition module. The display screen has a fingerprint detection area. The light source module illuminates the object to be detected within the fingerprint detection area, and the target light beam, reflected by the object, is incident on the spectral imaging chip. The spectral imaging chip and the signal processing circuit module are two independent parts, or the signal processing circuit module is integrated within the spectral imaging chip. The spectral imaging chip determines the spectral information of each pixel corresponding to a modulation unit in the light modulation layer of the spectral imaging chip after illumination by the target light beam, and the light intensity information of each pixel corresponding to a non-modulation unit in the light modulation layer. The signal processing circuit module determines the spectral information of the object to be detected based on the spectral information, and determines the image information of the object to be detected based on the light intensity information. The recognition module identifies whether the object to be detected is a live fingerprint of a target user based on the spectral information and the image information of the object to be detected.

[0131] Based on this, this application further provides a fingerprint module, characterized in that it includes: a spectral imaging chip and a circuit board, wherein the spectral imaging chip is electrically connected to the circuit board; wherein the spectral imaging chip has a light modulation layer, an image sensing layer and a signal processing circuit module sequentially stacked along the thickness direction; wherein the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction.

[0132] After introducing the basic principles of this application, various non-limiting embodiments of the functions of this application will be described in detail below with reference to the accompanying drawings.

[0133] Exemplary spectral imaging chip and information processing method suitable for said spectral imaging chip

[0134] Cameras have extremely wide applications across various industries, but existing camera image sensors can only acquire image information of the subject, such as the RGB color information, but cannot acquire the spectral information of the subject (or rather, lose most of the spectral information). Therefore, this application provides an information processing method suitable for spectral imaging chips, and utilizes a spectral imaging chip applied to this information processing method in a camera, so that the camera can simultaneously acquire the spectral information of the object during imaging, which can provide assistance in intelligent AI recognition, qualitative and quantitative analysis of material composition, etc.

[0135] Figure 1 This application provides an information processing method applicable to spectral imaging chips. For example... Figure 1 As shown, the information processing method applicable to spectral imaging chips includes:

[0136] S1, acquire the spectral information of each pixel corresponding to each modulation unit in the modulation layer of the spectral imaging chip after the target beam from the object to be imaged is irradiated, and the light intensity information of each pixel corresponding to each non-modulation unit in the modulation layer.

[0137] S2, based on the spectral information of the pixel corresponding to each modulation unit after the target beam is irradiated, determine the spectral information of the object to be imaged, and based on the light intensity information of the pixel corresponding to each non-modulation unit after the target beam is irradiated, determine the image information of the object to be imaged.

[0138] Specifically, the information processing method provided in this application embodiment can be applied to a spectral imaging chip. Its execution entity can be a signal processing circuit layer contained in the spectral imaging chip. This signal processing circuit layer enables imaging of the object to be imaged and acquires the image information and spectral information of the object. The object to be imaged can be a person and related person recognition features, a scene, etc.

[0139] It is worth noting that the above method is not absolute. In certain application scenarios, the above method can be understood as the spectral imaging chip only needing to acquire the image information or spectral information of the object to be imaged. In this case, step S1 can be understood as only needing to acquire the spectrum information or light intensity information; while correspondingly, step S2 only needs to determine the image information of the corresponding object to be imaged based on the information acquired in step S1. For example, when using this imaging device to acquire a spectral image, only spectral information needs to be acquired in step S1, and the spectral image needs to be output in step S2.

[0140] The application scenarios of this method can be as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of a spectral imaging device, which provides a spectral imaging device including: a spectral imaging chip and a signal processing circuit module. The spectral imaging chip and the signal processing circuit module are two independent parts, or the signal processing circuit module is integrated within the spectral imaging chip. When the signal processing circuit module is integrated within the spectral imaging chip as a signal processing circuit layer, the spectral imaging chip is a spectral imaging device. Light emitted from the light source module 300 illuminates the object to be imaged 200. The spectral imaging chip 100 receives the target beam reflected from the object to be imaged 200. After processing the target beam, the spectral imaging chip 100 obtains the image information and spectral information of the object to be imaged. The light source module 300 can be a broadband light source from visible light to near-infrared.

[0141] First, proceed with step S1. The specific structure of the spectral imaging chip can be as follows: Figure 3As shown, the spectral imaging chip 100 may include: a light modulation layer 110, an image sensing layer 120, and a signal processing circuit layer 130a, which are sequentially stacked along the thickness direction. The light modulation layer 110 has at least one modulation unit 1101 and at least one non-modulation unit 1102 distributed along its surface; that is, the light modulation layer 110 includes at least one modulation unit 1101 and at least one non-modulation unit 1102 distributed along its defined surface. The image sensing layer 120 has a plurality of sensing units 1201 distributed along its surface, and each modulation unit 1101 and each non-modulation unit 1102 corresponds to at least one sensing unit 1201 along the thickness direction. Each modulation unit 1101 and each non-modulation unit 1102, together with its corresponding sensing unit 120, constitutes a pixel of the spectral imaging chip 100. The signal processing circuit layer 130a is electrically connected to the sensing unit 1201. The signal processing circuit layer 130a is used to perform information processing methods to determine the image information and spectral information of the object to be imaged.

[0142] Accordingly, in step S1, the spectral information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit represent the spectral information of the pixel to which the modulation unit belongs and the light intensity information of the pixel to which each non-modulation unit belongs. That is, the correspondence here does not represent the positional relationship between the non-modulation unit or the modulation unit and the pixel, but rather the subordinate relationship between the modulation unit / non-modulation unit and the pixel.

[0143] The optical modulation layer 110 has a thickness of 60nm to 1200nm and can be directly fabricated on the image sensing layer 120. Specifically, one or more layers of material can be grown directly on the image sensing layer 120 and then the modulation units can be fabricated by etching. Alternatively, the modulation units can be directly etched on the image sensing layer 120 to obtain the optical modulation layer 110. The image sensing layer 120 can be a CIS wafer, and each sensing unit 1201 in the image sensing layer 120 corresponds to a pixel unit in the CIS wafer, used to detect the light beam passing through the optical modulation layer 110. For example, but not limited to, the micro-nano structure optical modulation layer 110 can be monolithically integrated on the CIS wafer at the wafer level, and the fabrication of the spectral imaging chip can be completed in one fabrication using CMOS technology.

[0144] Each modulation unit 1101 can be a micro / nano structure unit used to modulate the target beam. Each non-modulation unit 1102 does not have modulation capability and cannot modulate the target beam; instead, it allows the target beam to pass through directly. Each modulation unit 1101 on the optical modulation layer 110 can be directly fabricated on the photosensitive area surface of the CIS wafer. Areas on the CIS wafer without modulation units remain traditional RGB or black-and-white pixels, corresponding to non-modulation units on the optical modulation layer. Since the optical modulation layer of the spectral imaging chip includes both modulation and non-modulation units, it can acquire both the spectral information of the pixel corresponding to each modulation unit (i.e., the spectral information obtained after modulation by each modulation unit and detected by the corresponding sensing unit) and the light intensity information of the pixel corresponding to each non-modulation unit (i.e., the light intensity information directly obtained by the corresponding sensing unit after modulation by each non-modulation unit). Spectral information refers to the light intensity information corresponding to different wavelengths of light at the pixel corresponding to each modulation unit. Different modulation units may have the same or different modulation effects on different wavelengths of light, which can be set as needed; this embodiment does not specifically limit this.

[0145] Then, step S2 is executed. Based on the spectral information of the pixels corresponding to each modulation unit after the target beam illumination, the spectral information of the object to be imaged can be determined. Specifically, this can be achieved by encoding the spectral information obtained from each modulation unit onto the corresponding sensing unit of the CIS wafer, and then reconstructing it using a unit array response processing method. Spectral reconstruction algorithms may include, but are not limited to, least squares, non-negative least squares, simulated annealing, Tikhonov regularization, truncated singular value decomposition, and sparse optimization methods.

[0146] Since the spectral information of the pixels corresponding to each modulation unit may not be sufficient to determine the complete image information of the object to be imaged, it is necessary to determine the image information of the object to be imaged based on the light intensity information of the pixels corresponding to each non-modulation unit after the target beam illuminates it. Specifically, one approach is to ignore the light intensity information of the pixels corresponding to each modulation unit and only use the light intensity information of the pixels corresponding to all non-modulation units to determine the image information of the object to be imaged; another approach is to first determine the light intensity information of the pixels corresponding to each modulation unit using the light intensity information of the pixels corresponding to each non-modulation unit, and then combine this information with the light intensity information of the pixels corresponding to all non-modulation units to jointly determine the image information of the object to be imaged. This embodiment does not impose specific limitations on this approach.

[0147] The information processing method provided in this application embodiment can be applied to spectral imaging chips. By acquiring the spectral information of the pixels corresponding to each modulation unit in the modulation layer of the spectral imaging chip after the target beam from the object to be imaged is irradiated, the spectral information of the object to be imaged is determined. Based on the light intensity information of the pixels corresponding to each non-modulation unit in the modulation layer after the target beam is irradiated, the image information of the object to be imaged is determined. Compared with traditional image sensors, spectral information can be obtained without affecting the spatial resolution and imaging quality of the image, making it easier to obtain more comprehensive information about the object to be imaged. Moreover, since the spectral information of the object to be imaged can be used to uniquely identify the object, qualitative or quantitative analysis of the object to be imaged can be achieved through the spectral information of the object. This allows spectral imaging chips to be applied in fields such as fruit freshness, air pollution levels, AI scene recognition, and liveness detection, increasing the application scenarios of spectral imaging chips and providing a theoretical basis for their widespread application. In addition, in this application embodiment, the light modulation layer and image sensing layer are monolithically integrated without discrete components, which is beneficial to improving the stability of the device and reducing its size and cost.

[0148] Based on the above embodiments, in the information processing method provided in this application embodiment, step S1, the process of determining the image information of the object to be imaged based on the light intensity information of the pixel corresponding to each non-modulation unit after the target beam is irradiated, includes: first, determining the fitted light intensity information of the pixel corresponding to each modulation unit based on the light intensity information of the pixel corresponding to at least one non-modulation unit around each modulation unit after the target beam is irradiated; then, determining the image information of the object to be imaged based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit.

[0149] Specifically, in this embodiment, when determining the image information of the object to be imaged, the fitted light intensity information of the pixel corresponding to each modulation unit can first be determined based on the light intensity information of the pixel corresponding to at least one non-modulation unit around each modulation unit after the target beam is irradiated. "Around each modulation unit" refers to the eight positions closest to the modulation unit: left, upper left, top, upper right, right, lower right, bottom, and lower left. The number of non-modulation units around each modulation unit is related to the structure of the light modulation layer. The light modulation layer is generally a rectangular structure, with three units around each unit at the vertex position, five units around each unit at other edge positions (excluding the vertex position), and eight units around each unit at other edge positions (excluding the vertex position). If the structure of the light modulation layer is as follows... Figure 4 As shown, each modulation unit can be surrounded by a maximum of 5 non-modulation units and a minimum of 3 non-modulation units; if the structure of the optical modulation layer is as shown... Figure 5 As shown, each modulation unit can be surrounded by a maximum of 6 non-modulation units and a minimum of 0 non-modulation units; if the structure of the optical modulation layer is as shown... Figure 6 As shown, each modulation unit can be surrounded by a maximum of 4 non-modulation units and a minimum of 0 non-modulation units; if the structure of the optical modulation layer is as follows... Figure 7 As shown, each modulation unit is surrounded by 8 non-modulation units. This application uses a rectangular structure as an example for ease of understanding; the specific optical modulation layer structure can be designed according to requirements.

[0150] by Figure 7 Taking the structure of the light modulation layer shown as an example, the light intensity information of the pixels corresponding to at least one non-modulation unit around each modulation unit can be fitted to determine the fitted light intensity information of the pixels corresponding to each modulation unit. The fitting method can be an arithmetic mean, a weighted average, or selecting the median of the light intensity information, etc., and this application embodiment does not specifically limit this method.

[0151] Then, based on the fitted light intensity information of the pixels corresponding to each modulation unit and the light intensity information of the pixels corresponding to each non-modulation unit, the image information of the object to be imaged can be determined. This image information is complete image information, which contains the light intensity information of each pixel.

[0152] In this embodiment, by using the light intensity information of the pixels corresponding to at least one non-modulation unit around each modulation unit after the target beam is irradiated, the fitted light intensity information of the pixels corresponding to each modulation unit is determined. This ensures that the image information of the object to be imaged, determined based on the fitted light intensity information of the pixels corresponding to each modulation unit and the light intensity information of the pixels corresponding to each non-modulation unit, is complete image information, thus guaranteeing the integrity of the image and not affecting the overall imaging.

[0153] Based on the above embodiments, the information processing method provided in this application embodiment, which determines the fitted light intensity information of the pixel corresponding to each modulation unit based on the light intensity information of the pixels corresponding to multiple non-modulation units around each modulation unit after the target beam is irradiated, specifically includes:

[0154] Based on the smoothing filtering method, the light intensity information of the pixels corresponding to multiple non-modulation units around any modulation unit is filtered to obtain the fitted light intensity information of the pixels corresponding to the modulation unit.

[0155] Specifically, in this embodiment, determining the fitted light intensity information of the pixel corresponding to each modulation unit can be achieved through a smoothing filtering method. This smoothing filtering method may include median filtering, smoothing filtering, Gaussian filtering, etc.

[0156] For median filtering, if the filter window size is 3*3 pixels, then the light intensity information of the pixel corresponding to any modulation unit A is:

[0157] f(x,y)=median[f(x-1,y-1),f(x,y-1),f(x+1,y-1),

[0158] f(x-1,y),f(x+1,y),f(x-1,y+1),f(x,y+1),f(x+1,y+1)]

[0159] Where, median represents the median operation, f(x,y) is the light intensity information of the pixel corresponding to modulation unit A, (x,y) is the coordinate value of the pixel corresponding to modulation unit A; f(x-1,y-1) is the light intensity information of the pixel corresponding to the non-modulation unit to the lower left of modulation unit A, and this value is 0 if the lower left of modulation unit A is not a non-modulation unit; f(x,y-1) is the light intensity information of the pixel corresponding to the non-modulation unit below modulation unit A, and this value is 0 if the lower part of modulation unit A is not a non-modulation unit; f(x+1,y-1) is the pixel corresponding to the non-modulation unit to the lower right of modulation unit A, and this value is 0 if the lower right of modulation unit A is not a non-modulation unit; f(x-1,y) is the pixel corresponding to the non-modulation unit to the left of modulation unit A, and this value is 0 if the left side of modulation unit A is not a non-modulation unit. f(x+1,y) represents the pixel corresponding to the non-modulation unit to the right of modulation unit A. If there is no non-modulation unit to the right of modulation unit A, this value is 0. f(x-1,y+1) represents the pixel corresponding to the non-modulation unit to the upper left of modulation unit A. If there is no non-modulation unit to the upper left of modulation unit A, this value is 0. f(x,y+1) represents the pixel corresponding to the non-modulation unit above modulation unit A. If there is no non-modulation unit above modulation unit A, this value is 0. f(x+1,y+1) represents the pixel corresponding to the non-modulation unit to the upper right of modulation unit A. If there is no non-modulation unit to the upper right of modulation unit A, this value is 0.

[0160] For mean filtering, if the filter window size is 3*3 pixels, then the light intensity information of the pixel corresponding to any modulation unit A is:

[0161] f(x,y)=[f(x-1,y-1)+f(x,y-1)+f(x+1,y-1)+f(x-1,y)

[0162] +f(x+1,y)+f(x-1,y+1)

[0163] +f(x,y+1)+f(x+1,y+1)] / 8

[0164] For Gaussian filtering, if the size of the defiltering window is 3*3 pixels, then the light intensity information of the pixel corresponding to any modulation unit A is:

[0165] f(x,y)=0.111*[f(x-1,y-1)+f(x+1,y-1)+f(x-1,y+1)+f(x

[0166] +1,y+1)]+0.139*[f(x,y-1)+f(x-1,y)+f(x+1,y)

[0167] +f(x,y+1)]

[0168] Here, 0.11 and 0.139 are the weights of the light intensity information of the corresponding pixels.

[0169] Based on the above embodiments, in the information processing method provided in this application embodiment, the step of determining the image information of the object to be imaged based on the light intensity information of the pixel corresponding to each non-modulation unit after the target beam is irradiated includes: inputting the initial image obtained based on the light intensity information of the pixel corresponding to all non-modulation units after the target beam is irradiated into the fitting model to obtain the image information of the object to be imaged output by the fitting model; wherein, the fitting model is constructed based on an adversarial neural network, and the fitting model is trained based on the labels of empty sample images with blank pixels and complete sample images without blank pixels corresponding to the empty sample images.

[0170] Specifically, in this embodiment, determining the image information of the object to be imaged can be achieved through machine learning methods such as adversarial neural networks. In this embodiment, a fitting model is constructed using an adversarial neural network, and the model is trained using empty sample images containing blank pixels and complete sample image labels corresponding to the empty sample images that do not contain blank pixels. Finally, the initial image obtained after the target beam illuminates the object, based on the light intensity information of the pixels corresponding to all non-modulation units, is input into the fitting model to obtain the image information of the object to be imaged, output by the fitting model. Here, the complete sample image label refers to the actual complete sample image corresponding to the empty sample image.

[0171] In this embodiment, a fitting model based on an adversarial neural network is introduced, which can determine the image information of the object to be imaged more quickly and accurately.

[0172] Based on the above embodiments, the information processing method provided in this application embodiment, wherein the fitting model is trained based on the labels of empty sample images containing blank pixels and complete sample images corresponding to the empty sample images that do not contain blank pixels, specifically includes:

[0173] The generator in the adversarial neural network is trained based on the missing sample images and the corresponding complete sample image labels, and the trained generator is competitively discriminated based on the discriminator in the adversarial neural network.

[0174] The generator obtained from the training is used as the fitted model.

[0175] Specifically, the adversarial neural network consists of a generator and a discriminator. The generator takes a missing sample image as input and generates a complete sample image corresponding to the missing sample image as output. The discriminator takes multiple complete sample images as input, including the complete sample image labels and the complete sample images generated by the generator. During training, the generator and discriminator compete with each other. The generator aims to output complete sample images with high scores from the discriminator, while the discriminator aims to maximize the label scores of the complete sample images and minimize the scores of the complete sample images output by the generator. After training, the generator is obtained and used as the fitting model.

[0176] like Figure 3 As shown, based on the above embodiments, this application provides a spectral imaging chip, including: a light modulation layer 110, an image sensing layer 120, and a signal processing circuit layer 130a sequentially stacked along the thickness direction; wherein,

[0177] The light modulation layer 110 has at least one modulation unit 1101 and at least one non-modulation unit 1102 distributed along its surface. The image sensing layer 120 has multiple sensing units 1201 distributed along its surface, and each modulation unit 1101 and each non-modulation unit 1102 corresponds to at least one sensing unit 1201 along the thickness direction. Each modulation unit 1101 and each non-modulation unit 1102, together with the corresponding sensing unit 1201, constitutes a pixel of the spectral imaging chip 100. The signal processing circuit layer 130a is electrically connected to the sensing unit 1201 and is used by the information processing methods provided in the above embodiments to determine the image information and spectral information of the object to be imaged. The thickness of the light modulation layer 110 is 60 nm to 1200 nm, and the light modulation layer 110 can be directly fabricated on the image sensing layer 120. Specifically, one or more layers of material can be grown directly on the image sensing layer 120 and then the modulation units can be fabricated by etching, or the modulation units can be directly etched on the image sensing layer 120 to obtain the optical modulation layer 110. The image sensing layer 120 can be a CIS wafer, and each sensing unit in the image sensing layer 120 corresponds to a pixel in the CIS wafer, used to detect the light beam passing through the optical modulation layer. The micro-nano structure optical modulation layer can be monolithically integrated on the CIS wafer at the wafer level, and the fabrication of the spectral imaging chip can be completed in one fabrication using CMOS technology.

[0178] The spectral imaging chip provided in this embodiment can simultaneously determine the image information and spectral information of the object to be imaged, enabling its application in fields such as fruit freshness, air pollution levels, and AI scene recognition. This expands the application scenarios of spectral imaging chips and provides a theoretical basis for their widespread use. Furthermore, this embodiment integrates the light modulation layer and image sensing layer on a single chip, eliminating discrete components and improving device stability while reducing size and cost.

[0179] As a preferred embodiment, the optical modulation layer can be silicon nitride with a thickness of 200nm-500nm. The optical modulation layer can have 1000-250000 units distributed on it, with each unit having a size of 100μm. 2 ~40000μm 2 Of these, modulation units account for 10% of the total number of units, while the remaining 90% are non-modulation units.

[0180] As a preferred embodiment, the optical modulation layer can be silicon with a thickness of 100-400 nm. The optical modulation layer can have 1000-250000 units distributed on it, with each unit having a size of 100 μm. 2 ~40000μm 2 Of these, modulation units account for 15% of the total number of units, while the remaining 85% are non-modulation units.

[0181] Based on the above embodiments, in the spectral imaging chip provided in this application embodiment, the modulation unit in the light modulation layer can specifically be a micro / nano structure unit, obtained through etching. The structure of the light modulation layer is as follows: Figures 4-7 As shown. Figure 4 In the optical modulation layer, multiple modulation units are distributed along its edge. Each modulation unit corresponds to one or more sensing units. Modulation units may or may not occupy the edge position, can be continuous or discontinuous, and can be located at any edge position. The remaining positions in the optical modulation layer are non-modulation units, i.e., without any modulation units, and are blank units. The target beam can be directly transmitted to the RGB or black-and-white pixels of the CIS wafer below the optical modulation layer. Each unit (including modulation and non-modulation units) in the optical modulation layer has a corresponding sensing unit below it. Each modulation unit in the optical modulation layer has a different modulation effect on different wavelengths of light. The modulation methods of the input spectrum can be the same or different among the modulation units. Different modulation methods can include, but are not limited to, scattering, absorption, transmission, reflection, interference, excitation, resonance enhancement, etc. The final effect of the modulation effect is that different wavelengths of light have different transmission spectra after passing through the modulation unit. After the light is modulated by the modulation unit, the light intensity information is detected by the corresponding sensing unit below the modulation unit. Each unit and the sensing unit below it constitute a pixel. The intensity distribution of each wavelength at a pixel can be obtained through algorithms.

[0182] Figure 5 In the optical modulation layer, multiple different modulation units are distributed at the edge and center positions. Each modulation unit can correspond to one or more sensing units. The modulation units can be located at any edge or center position, and can be continuously or discontinuously distributed. The positions can be arbitrarily selected. Each modulation unit can be an array composed of multiple identical modulation sub-units, or an array composed of multiple different modulation sub-units.

[0183] Figure 6 In the optical modulation layer, every four modulation units form a group of modulation units, and every four non-modulation units form a group of non-modulation units. Each group of modulation units and each group of non-modulation units are distributed alternately.

[0184] Figure 7 In the optical modulation layer, the eight positions surrounding each modulation unit are non-modulation units, and there are no other modulation units.

[0185] Figure 8 and Figure 9As shown, at least one set of modulation units 111 are distributed along the surface of the optical modulation layer 110. Each set of modulation units 111 includes multiple modulation units distributed along the surface of each set of modulation units 111, and each modulation unit has C4 symmetry. C4 symmetry means that after the modulation unit is rotated 90°, 180°, or 270° along the axis of symmetry, the structure in the modulation unit coincides with the original structure. The optional structures of the modulation units include circles, crosses, regular polygons, squares, etc. The eight positions surrounding each set of modulation units on the optical modulation layer are non-modulation units, and there are no other modulation units there.

[0186] In some embodiments, such as Figure 8 , Figure 9 As shown, at least a portion of the modulation units in each group of modulation units 111 have multiple modulation holes arranged along the thickness direction. These multiple modulation holes are distributed along the surface of each modulation unit to form a modulation hole array. A modulation hole array indicates that the multiple modulation holes on the modulation unit are arranged in rows and columns, for example... Figure 8 The modulation aperture arrays shown in the first modulation unit 112, second modulation unit 113, third modulation unit 114, and fourth modulation unit 115 are all arranged in 4 rows * 4 columns. Of course, Figure 8 The images shown are for illustrative purposes only and do not represent that the modulation aperture array in the embodiments of the present invention is limited to 4 rows * 4 columns.

[0187] like Figure 8 and Figure 9 As shown, each modulation unit 111 contains multiple modulation units with modulation apertures, and the shapes of the modulation apertures in each modulation unit are different. Each modulation unit modulates light of different wavelengths differently, and the modulation methods for the input spectrum differ between units. These modulation methods may include, but are not limited to, scattering, absorption, transmission, reflection, interference, excitation, and resonance enhancement. The final effect of the modulation is that different wavelengths of light have different transmission spectra after passing through the modulation aperture array of the modulation unit. After the light is modulated by the modulation aperture array, the light intensity is detected by the sensing unit below the modulation aperture array. Each modulation unit and the sensing unit below it constitute a pixel. The intensity distribution of each wavelength at a pixel can be obtained through an algorithm. An image under a certain modulation method can be obtained through the same modulation aperture array pixels of different modulation units. Multiple pixels constitute an image containing multiple spectral information.

[0188] In any two sets of modulation units 111, the modulation aperture arrays of the modulation subunits located at the same position are identical. For example... Figure 8In the illustrated embodiment, each modulation unit 111 includes a first modulation unit 112, a second modulation unit 113, a third modulation unit 114, and a fourth modulation unit 115. The modulation aperture arrays in the first modulation unit 112, second modulation unit 113, third modulation unit 114, and fourth modulation unit 115 are all different. The modulation aperture arrays of the first modulation unit 112 in each group of modulation units 111 are all the same, the modulation aperture arrays of the second modulation units 113 in each group of modulation units 111 are all the same, the modulation aperture arrays of the third modulation units 114 in each group of modulation units 111 are all the same, and the modulation aperture arrays of the fourth modulation units 115 in each group of modulation units 111 are all the same.

[0189] In this embodiment, at least a portion of the modulation units in each group of modulation units 111 have a plurality of modulation holes arranged along the thickness direction, indicating at least two implementation methods.

[0190] Firstly, such as Figure 8 As shown, the optical modulation layer 110 is provided with one or more sets of repeating modulation units 111, for example... Figure 8 In the illustrated embodiment, six sets of repeating modulation units 111 are shown, each set of modulation units 111 being divided into multiple modulation units. Figure 8 In the illustrated embodiment, each modulation unit 111 includes a first modulation unit 112, a second modulation unit 113, a third modulation unit 114, and a fourth modulation unit 115. Each of the first modulation unit 112, the second modulation unit 113, the third modulation unit 114, and the fourth modulation unit 115 is provided with a modulation aperture array. The modulation unit has C4 symmetry, and correspondingly, the modulation aperture array also has C4 symmetry. C4 symmetry means that after the modulation unit is rotated 90°, 180°, or 270° along the axis of symmetry, the structure in the modulation unit coincides with the original structure.

[0191] Secondly, such as Figure 9 As shown, the optical modulation layer 110 is provided with one or more sets of repeating modulation units 111, for example... Figure 9 In the illustrated embodiment, six sets of repeating modulation units 111 are shown, each set of modulation units 111 being divided into multiple modulation units. Figure 9In the illustrated embodiment, each modulation unit 111 includes a first modulation unit 112, a second modulation unit 113, a third modulation unit 114, and a fourth modulation unit 115. Each of the first, second, and third modulation units 112 and 113 contains a modulation aperture array. However, the fourth modulation unit 115 is entirely transparent, allowing direct light transmission, which can be used to calibrate the intensity of the direct light transmission within that unit. The remaining modulation aperture arrays correspond to specific wavelengths of light and provide narrowband filtering. Modulation aperture arrays at the same positions in different modulation unit groups 111 are identical, and each modulation aperture array has a corresponding sensing unit below it. After the light passes through the narrowband-filtered modulation aperture array, the sensing unit below the array detects the light intensity and performs differential analysis with the direct incident light to obtain the intensity of each wavelength after narrowband filtering. Each group of modulation units 111 and the sensing unit below the modulation unit 111 constitute a pixel. An image at a certain wavelength can be obtained through the pixels of the same modulation aperture array of different groups of modulation units 111. Multiple pixels constitute an image containing multiple spectral information.

[0192] For both of the above schemes, the modulation effect can be altered by changing the shape of the modulation subunit or the modulation aperture array. The geometry of the modulation aperture array can include, but is not limited to, circles, crosses, regular polygons, rectangles, and any combination thereof. The modulation effect can also be altered by changing the parameters of the modulation aperture array. Changes in structural parameters can include, but are not limited to, the modulation aperture array period, radius, side length, duty cycle, thickness, and any combination thereof. One or a combination of these two schemes can be selected. The modulation aperture array on the optical modulation layer 110 can be a hole penetrating the flat plate or a blind aperture with a certain depth.

[0193] Based on the above embodiments, the spectral imaging chip provided in this application embodiment includes a plurality of modulation sub-units arranged along the thickness direction in each modulation unit of the optical modulation layer, and each modulation sub-unit is a hole-shaped structure or a columnar structure.

[0194] Specifically, in this embodiment, each modulation unit in the optical modulation layer includes multiple modulation sub-units disposed along the thickness direction. The modulation sub-units can be either aperture-like or columnar structures. A modulation sub-unit with an aperture-like structure can be called a modulation aperture, and a modulation sub-unit with a columnar structure can be called a modulation column. It should be noted that the same modulation unit can only contain either a modulation aperture or a modulation column; it cannot contain both modulation apertures and modulation columns simultaneously. Therefore, a modulation unit containing a modulation aperture can be called an aperture modulation unit, and a modulation unit containing a modulation column can be called a column modulation unit.

[0195] Based on the above embodiments, the spectral imaging chip provided in this application embodiment has a non-uniform aperture cross-sectional shape in each modulation unit that is a different modulation sub-unit of the aperture structure; and / or,

[0196] The structural parameters of different modulation subunits with the aperture structure in each modulation unit are not exactly the same.

[0197] Specifically, in the embodiments of this application, the cross-sectional shapes of different modulation holes in each modulation unit can be the same, completely different, or partially the same and partially different. That is, all modulation holes within the same modulation unit have the same or different cross-sectional shapes. The structural parameters of the modulation holes in each modulation unit can also be the same, completely different, or partially the same and partially different. Whether the cross-sectional shapes of different modulation holes in each modulation unit are the same does not affect whether the structural parameters are the same. The cross-sectional shapes of the holes include circles, ellipses, crosses, regular polygons, stars, or rectangles. The structural parameters may include, but are not limited to, the period, radius, side length, duty cycle, thickness, major axis length, minor axis length, rotation angle, or number of angles of the modulation holes in each modulation unit. The modulation holes can be arranged row by row or column by column according to a preset periodic order, or arranged in an array according to a gradual change in the size of the structural parameters.

[0198] Since the different cross-sectional shapes and / or structural parameters of the modulation aperture in the modulation unit will affect its modulation effect, the modulation effect can be changed by changing the shape of the modulation aperture in the modulation unit. The structural parameters can be changed by any combination of the above-mentioned structural parameters.

[0199] The period of the modulation aperture in each pillar modulation unit can be between 50nm and 800nm, and the duty cycle can be between 5% and 95%. The period of the modulation aperture can also be between 80nm and 600nm, and the duty cycle can be between 10% and 90%.

[0200] Different modulation units exert different modulation effects on the spectrum. Spectral modulation effects can include, but are not limited to, scattering, absorption, transmission, reflection, interference, surface plasmon resonance, and other effects. The modulation effect can be altered by changing the structural parameters (including but not limited to one or any combination of parameters such as period, radius, side length, duty cycle, and thickness) and arrangement of the modulation holes within the column modulation unit. Increasing the number of modulation columns can improve the sensitivity to differences between different spectra.

[0201] Based on the above embodiments, the spectral imaging chip provided in this application embodiment has the same structural shape and column height for different modulation sub-units of the columnar structure in each modulation unit, and all modulation sub-units are arranged in each modulation unit with C4 symmetry.

[0202] Specifically, each column modulation unit contains several modulation columns. All modulation columns within the same column modulation unit have the same structural shape, and the modulation columns are arranged row by row or column by column according to a preset periodic sequence, exhibiting C4 symmetry. All modulation columns within the same column modulation unit have the same height, while the heights of modulation columns in different column modulation units may be the same or different. The structural parameters of the modulation columns may include height, longitudinal section structural parameters, and cross-sectional structural parameters. The structural shape of the modulation columns can be as follows: Figure 10 As shown, the shapes include, but are not limited to, cylinders, cubes, frustums, cones, bell shapes, etc. The longitudinal cross-sectional shape of the modulation column can be as follows: Figure 11 As shown, shapes include, but are not limited to, rectangles, trapezoids, triangles, bell shapes, etc. The cross-sectional shape of the modulation column includes rectangles, circles, etc.

[0203] The height of the modulation column can be between 100 nm and 400 nm. For a cylindrical modulation column, the diameter can be between 10 nm and 300 nm. For a cubic modulation column, the cross-section can be square or rectangular, with side lengths between 10 nm and 400 nm. For a frustum-shaped modulation column, the diameters of the two circular cross-sections can be between 10 nm and 400 nm. For a conical modulation column, the diameter of the base circle can be between 10 nm and 400 nm. For a bell-shaped modulation column, the diameter of the base circle can be between 10 nm and 400 nm.

[0204] Based on the above embodiments, the spectral imaging chip provided in this application embodiment has a columnar modulation subunit that is integrally formed and / or stacked.

[0205] Specifically, such as Figure 12 As shown, the spectral imaging chip includes: a light modulation layer 110, an image sensing layer 120, and a signal processing circuit layer 130a. The modulation unit of the light modulation layer 110 includes a modulation pillar, which is integrally formed. Figure 13 As shown, the modulation column can also be composed of multiple layers of sub-modulation columns stacked together. The structural shape of each layer of sub-modulation columns can be the same or different. Each layer of sub-modulation columns can be a cube, a cylinder, etc. The material of each layer of sub-modulation columns can be the same or different, and can be any of metal or dielectric. Figure 13 In the middle, the modulation column is composed of multiple layers of cuboids stacked together.

[0206] A single modulation unit may contain only a one-piece modulation column, or only a stacked modulation column, or it may contain both one-piece and stacked modulation columns.

[0207] Based on the above embodiments, the modulation unit includes, but is not limited to, one-dimensional photonic crystals, two-dimensional photonic crystals, surface plasmon resonances, metamaterials, metasurfaces, etc. Specific materials may include silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, metals, III-V group materials, etc., wherein silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, etc.

[0208] Based on the above embodiments, in the vertical direction, the optical modulation layer may include at least one sub-modulation layer disposed along the thickness direction. The material of each sub-modulation layer may be the same or different to increase the modulation capability of the optical modulation layer on the target beam in the spectrum, thereby making the sampling capability of the target beam stronger and improving the spectral recovery accuracy. The optical modulation layer may have the following four cases in the vertical direction.

[0209] 1) such as Figure 14 As shown, the polarization-independent light modulation layer is a single material layer, including a first sub-modulation layer 117, and the thickness of the light modulation layer is 60nm to 1200nm.

[0210] 2) such as Figure 15 and Figure 16 As shown, the polarization-independent light modulation layer 110 may include multiple sub-modulation layers, each made of a different material. The thickness of each sub-modulation layer ranges from 60 nm to 1200 nm. The materials of each sub-modulation layer may include silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, metals, group III-V materials, etc., where silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, and silicon carbide. For example... Figure 15 In the illustrated embodiment, the optical modulation layer includes a first sub-modulation layer 117 and a second sub-modulation layer 118; for example... Figure 16 In the embodiment shown, the optical modulation layer includes a first sub-modulation layer 117, a second sub-modulation layer 118, and a third sub-modulation layer 119.

[0211] 3) such as Figure 17 As shown, the polarization-independent light modulation layer 110 may include multiple sub-modulation layers, each made of a different material. The thickness of each sub-modulation layer is 60 nm to 1200 nm. One or more sub-modulation layers may not be penetrated by the modulation aperture 116. The materials of each sub-modulation layer may include silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, metals, III-V group materials, etc., wherein silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, etc.

[0212] 4) such as Figure 18 As shown, the polarization-independent light modulation layer 110 is fabricated by directly etching the structure on the photodetector layer 122 of the back-illuminated CIS wafer, with an etching depth of 60 nm to 1200 nm. The modulation aperture array of the light modulation layer 110 can be one of the two schemes mentioned above or any combination thereof.

[0213] Based on the above embodiments, in the vertical structure, the optical modulation layer may not be penetrated by modulation pillars or modulation apertures. The modulation pillars may have a certain thickness, specifically 60nm to 1200nm, and the thickness of the entire optical modulation layer may be 120nm to 2000nm. The thickness of the modulation aperture may be 160nm to 1000nm, and the thickness of the entire optical modulation layer may be 220nm to 1500nm.

[0214] Based on the above embodiments, in the vertical structure, the optical modulation layer can be composed of two different materials: a silicon layer and a gold layer. The thickness of the silicon layer can be 60nm to 1200nm, and the thickness of the gold layer can be 60nm to 1200nm.

[0215] In summary, the spectral imaging chip provided in this application has the following advantages: 1) This spectral imaging chip can acquire both image and spectral information, providing complete image information as well as spectral information at different points within the field of view. 2) The spectral chip can be fabricated in a single CMOS fabrication process, which helps reduce device failure rate, improve device yield, and reduce costs. 3) Monolithically integrating the optical modulation layer and image sensing layer, without discrete components, helps improve device stability and greatly promotes the miniaturization and weight reduction of image sensors. 4) Monolithic integration at the wafer level minimizes the distance between the sensor and the optical modulation layer, which helps reduce the size of the unit, achieve higher resolution, and reduce packaging costs.

[0216] Based on the above embodiments, the image sensing layer in this application embodiment is specifically a CIS wafer, which can be either front-illuminated or back-illuminated. Figure 19 As shown, this is a front-illuminated CIS wafer. The front-illuminated CIS wafer includes a photodetector layer 122 and a metal line layer 121 connected along the thickness direction of the image sensing layer. The photodetector layer 122 is located below the metal line layer 121. The CIS wafer does not integrate microlenses or filters; the light modulation layer is directly integrated onto the metal line layer 121. The metal line layer 121 is used for preliminary signal processing of the spectral signals received by the wafer, converting the optical signal data into electrical signals first. This accelerates the processing efficiency of the signal processing circuit layer, making signal conversion and signal processing more stable and accurate.

[0217] like Figure 20As shown, this is a back-illuminated CIS wafer. The back-illuminated CIS wafer includes a photodetector layer 122 and a metal line layer 121 connected along the thickness direction of the image sensing layer. The photodetector layer 122 is above the metal line layer 121. The CIS wafer does not integrate microlenses or filters; the light modulation layer is directly integrated onto the photodetector layer 122. Since the target beam directly illuminates the photodetector layer 122 after passing through the light modulation layer, the adverse effects of the metal line layer on the target beam can be effectively eliminated, improving the quantum efficiency of the spectral imaging chip.

[0218] Based on the above embodiments, such as Figure 21a Figure 21b As shown, the spectral imaging chip also includes a light-transmitting medium layer 160, which is located between the light modulation layer 110 and the image sensing layer 120. Figure 21a The modulation unit of the intermediate light modulation layer 110 includes a modulation aperture 116. Figure 21b The modulation unit of the intermediate light modulation layer 110 includes modulation pillars 1103. The light-transmitting dielectric layer 160 has a thickness of 50 nm to 1 μm and can be made of silicon dioxide. For a direct deposition growth process, the light-transmitting dielectric layer 160 can be covered on the image sensing layer 120 by chemical vapor deposition, sputtering, spin coating, etc., and then the light modulation layer can be deposited and etched on top of it. For a transfer process, the light modulation layer can be processed on the silicon dioxide first, and then both parts can be transferred together onto the image sensing layer 120.

[0219] It is worth mentioning that in some application scenarios, the purpose of using the spectral imaging chip and its corresponding camera module or device is to obtain an image of the subject with relatively high imaging quality. For example, the spectral imaging chip can be used to obtain an image that more closely approximates the actual color and texture of the subject. Accordingly, in these application scenarios, the focus of the information processing method is to acquire the image information of the subject and then use the acquired spectral information to adjust the obtained image.

[0220] In these application scenarios, preferably, the image information of the subject obtained by the spectral imaging chip is relatively complete and accurate, so the image generated based on the image information of the subject is also relatively complete and accurate. In contrast, the spectral information of the subject obtained by the spectral imaging chip can be understood as auxiliary information used to assist image adjustment.

[0221] Accordingly, in these application scenarios, preferably, the total area of ​​the modulation unit 1101 of the spectral imaging chip accounts for 0% to 25% of the effective area of ​​the spectral imaging chip. Here, the effective area of ​​the spectral imaging chip refers to the area in the spectral imaging chip used to acquire optical frequency information and light intensity information. That is, preferably, in these application scenarios, the total area of ​​the modulation unit 1101 in the spectral imaging chip accounts for no more than 25% of the effective area of ​​the spectral imaging chip. In this way, the modulation unit 1101, which occupies a relatively small area proportion, can acquire spectral information about the subject for image adjustment, while the non-modulation unit 1102, which occupies a relatively large area proportion, can ensure that enough light intensity information is acquired to reconstruct a high-quality image. Furthermore, the image is adjusted using the acquired spectral information to obtain a relatively higher imaging quality, making the imaging effect of the image more realistic.

[0222] It is worth mentioning that in these application scenarios, the area ratio occupied by the modulation unit 1101 can be set and adjusted according to the actual situation, for example, 5%, 10%, 15%, 20%, etc. The inventors of this application found through testing that when the area ratio is 5% to 15%, the imaging quality of the final image obtained approaches or even surpasses the imaging quality of images acquired by existing CMOS and CCD imaging chips under algorithm matching.

[0223] In addition, in some other application scenarios, it is desirable for the spectral imaging chip to obtain a relatively large amount of spectral information, so as to obtain a more accurate description of the features or information of the subject using the relatively large amount of spectral information. At the same time, it is also desirable that the image information obtained by the spectral imaging chip is not affected too much.

[0224] For example, when a terminal device equipped with the aforementioned spectral imaging chip is used for biometric detection (e.g., iris recognition, fingerprint recognition, etc.), existing biometric recognition schemes generally require imaging the iris or fingerprint to be identified to obtain the texture information of the captured iris or fingerprint, and then matching the obtained texture information with a preset standard to achieve the recognition function. However, in practical applications, some criminals use prosthetics to replicate biometric features to bypass the system. Therefore, in these application scenarios, a liveness detection function is desired. Liveness detection can be achieved based on the spectral information of the object being photographed.

[0225] Accordingly, in these application scenarios, preferably, the total area of ​​the modulation unit 1101 of the spectral imaging chip accounts for 20% to 45% of the effective area of ​​the spectral imaging chip. Here, the effective area of ​​the spectral imaging chip refers to the area in the spectral imaging chip used to acquire spectral information and light intensity information. That is, preferably, in these application scenarios, the total area of ​​the modulation unit 1101 in the spectral imaging chip accounts for 20% to 45% of the effective area of ​​the spectral imaging chip. In this way, the non-modulation unit 1102, which has a relatively large area ratio, can also ensure that enough light intensity information is acquired to reconstruct a high-quality image. The modulation unit 1101 with this area ratio can acquire the spectral information of the subject for image adjustment, and then determine whether the subject is alive based on the spectral information.

[0226] It is worth mentioning that in these application scenarios, the area ratio occupied by the modulation unit 1101 can be set and adjusted according to the actual situation, for example, 25%, 30%, 35%, etc. The inventors of this application found through testing that when the area ratio is 25% to 35%, the spectral imaging chip can obtain high-quality biometric images, and the spectral information obtained is sufficient to accurately determine whether the subject is a living organism.

[0227] In addition, in other application scenarios, the need to obtain image information and spectral information of the subject is similar. For example, in some application scenarios, it is first necessary to identify or perceive the scene through images, and then use spectral information to perform spectral analysis on the scene.

[0228] Accordingly, in these application scenarios, preferably, the total area of ​​the modulation unit 1101 of the spectral imaging chip accounts for 45% to 65% of the effective area of ​​the spectral imaging chip. Here, the effective area of ​​the spectral imaging chip refers to the area in the spectral imaging chip used to acquire spectral information and light intensity information. That is, preferably, in these application scenarios, the total area of ​​the modulation unit 1101 in the spectral imaging chip accounts for 45% to 65% of the effective area of ​​the spectral imaging chip. In this way, the non-modulation unit 1102, whose proportion is roughly equivalent to that of the modulation unit 1101, can also ensure that enough light intensity information is acquired to reconstruct a high-quality image. The modulation unit 1101 can acquire the spectral information of the subject for image adjustment.

[0229] It is worth mentioning that in these application scenarios, the area ratio occupied by the modulation unit 1101 can be set and adjusted according to the actual situation, for example, 55%, 60%, etc. The inventors of this application found through testing that when the area ratio is 50% to 55%, the spectral imaging chip can obtain high-quality biometric images, and the obtained spectral information is sufficient to determine whether the subject is alive. That is, more preferably, in these application examples, the area ratio is 50% to 55%.

[0230] Furthermore, in other application scenarios, the core requirement for the spectral imaging chip is to obtain the necessary information through precise spectral analysis of the subject, and then perform quantitative or qualitative analysis based on the obtained information. For example, the spectral imaging chip can be used to test proteins in milk, detect the components and acidity of fruits, and monitor water quality. In these application scenarios, higher precision spectral information is required, while the demand for image information of the subject is relatively low or essentially unnecessary. That is, in these application scenarios, the image information of the target is only used as auxiliary information (e.g., for monitoring some emergencies) or even as useless information.

[0231] Accordingly, in these application scenarios, preferably, the overall area of ​​the modulation unit 1101 of the spectral imaging chip accounts for more than 60% of the effective area of ​​the spectral imaging chip, more preferably, between 80% and 95%. Here, the effective area of ​​the spectral imaging chip refers to the area in the spectral imaging chip used to acquire spectral information and light intensity information.

[0232] It is worth mentioning that, in these application scenarios, the area ratio occupied by the modulation unit 1101 can be set and adjusted according to the actual situation, for example, 85% or 90% can be selected. Of course, in specific examples of these application scenarios, this area ratio can be greater than 95%, which is not limited by this application.

[0233] Furthermore, in some application scenarios, the spectral imaging chip according to this application is suitable for combination with optical elements to form an optical module, wherein the optical elements are located on the photosensitive path of the spectral imaging chip and are used to modulate the light entering the spectral imaging chip. In specific examples, the optical elements include, but are not limited to, lenses (or lens groups), diffraction elements, and diffusion elements (e.g., frosted glass, etc.).

[0234] Those skilled in the art will know that for camera modules using traditional CMOS or CCD imaging chips as photosensitive chips, when the field of view of the optical lens is too large (e.g., exceeding 75°), the resulting image will be distorted in its four corner regions and / or edge regions, affecting the image imaging effect. However, in this embodiment, the spectral information obtained by the spectral imaging chip is not used for imaging; that is, image distortion does not affect the acquisition and analysis of spectral information.

[0235] Preferably, in these application scenarios, the modulation unit 1101 of the spectral imaging chip is arranged in the corner region and / or edge region of the spectral imaging chip (that is, the modulation unit 1101 is arranged in the corner region and / or edge region of the light modulation layer 110), while the non-modulation unit 1102 of the light modulation layer 110 is arranged in the middle region of the spectral imaging chip. In this way, the spectral information of the subject can be obtained without affecting the imaging.

[0236] It is worth noting that in these application scenarios, the corner and edge regions of the spectral imaging chip include the edges and corners of the effective area of ​​the spectral imaging chip, as well as the regions adjacent to the corners and edges. For example, in a specific example, the region whose distance from the central axis of the spectral imaging chip is less than or equal to 70% of the total distance of the effective area in that direction can be positioned as the central region, while the other regions of the effective area are defined as edge and corner regions.

[0237] Furthermore, in some applications, it is desirable for the spectral imaging chip to obtain more accurate spectral information about the subject. It should be understood that, regardless of whether optical elements are configured, the information acquired in the intermediate region of the spectral imaging chip has higher accuracy.

[0238] Therefore, in these application scenarios, preferably, the modulation unit 1101 of the spectral imaging chip is arranged in the middle region of the effective area of ​​the spectral imaging chip, where the effective area of ​​the spectral imaging chip represents the region in the spectral imaging chip used to acquire spectral information and light intensity information. That is, the quality of the spectral information of the subject acquired by the spectral imaging chip is optimized by adjusting its position. In these application scenarios, the non-modulation unit 1102 can be arranged around the modulation unit 1101.

[0239] It is worth mentioning that, in the spectral imaging chip according to the embodiments of this application, the density of the modulation units 1101 also affects its imaging quality and spectral analysis. Therefore, the inventors of this application have also attempted to find a suitable density to improve the performance of the spectral imaging chip.

[0240] Specifically, in a specific example of this application, sparsity is defined to the following degree: First, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit 1101 is represented by the geometric center of the modulation unit 1101; then, the points corresponding to the modulation unit 1101 are encoded from 1 to n, and denoted by d. ij This represents the distance between the geometric center of the i-th modulation unit 1101 and the geometric center of the j-th modulation unit 1101 (e.g., ...). Figure 28 As shown), correspondingly, in this specific example, the sparsity A can be calculated using the following formula:

[0241]

[0242] It is worth mentioning here that, in the above definition process, if two adjacent modulation units 1101 have the same structure, the two modulation units 1101 can be regarded as one modulation unit 1101 and the integrated modulation unit 1101 can be identified by the geometric center of the two modulation units 1101, so as to facilitate calculation.

[0243] In this embodiment, the sparsity value is greater than 0 and less than or equal to 1. Preferably, the sparsity is less than or equal to 0.5. Here, when the sparsity value is less than or equal to 0.5, the arrangement density of the modulation units 1101 of the spectral imaging chip is relatively sparse. Therefore, the modulation units 1101 have less impact on the imaging of the spectral imaging chip, which is beneficial for image restoration. In particular, when the sparsity value is less than or equal to 0.15, the spectral imaging chip can be defined as a sparse spectral imaging chip, wherein the arrangement density of the modulation units 1101 of the sparse spectral imaging chip is relatively sparse, while the arrangement density of the corresponding non-modulation units 1102 is relatively high. Through this configuration, the spectral imaging chip can obtain more and richer light intensity information, which is beneficial for improving the image imaging quality. Furthermore, the spectral information of the subject acquired by the sparse spectral imaging chip can be integrated with the image to improve the realism of the image.

[0244] In some examples of this application, the sparsity can also be set between 0.15 and 0.35. It is understood that if the sparsity of the spectral imaging chip is too low, the spectral information may be incomplete or have low accuracy. Therefore, the sparsity value can be appropriately increased to improve the density of the spectral information and thus improve the accuracy of spectral information measurement.

[0245] In other examples of this application, the sparsity value is set to a range of 0.5 to 1 (excluding 0.5). It should be understood that when the sparsity value of the spectral imaging chip is high, the arrangement density of the modulation units 1101 of the spectral imaging chip is large. Therefore, the spectral imaging chip can be more widely used in applications of spectral acquisition and analysis. Correspondingly, the image obtained by the spectral imaging chip can serve as auxiliary information to improve the accuracy of spectral information. In these examples, preferably, the sparsity value of the spectral imaging chip is greater than or equal to 0.85 to ensure sufficient spectral information is available. Of course, in some examples of this application, the sparsity value is set to a range of 0.65 to 0.85. In this case, the spectral imaging chip can acquire relatively sufficient spectral information and obtain an image more suitable for auxiliary analysis to assist and / or complement complete spectral analysis.

[0246] In some embodiments, such as Figures 22-27 As shown, the spectral imaging chip 100 further includes at least one of a lens 140 and a filter 150, wherein at least one of the lens 140 and the filter 150 is connected to the side of the light modulation layer 110 opposite to or close to the image sensing layer 120.

[0247] like Figure 22 As shown, the spectral imaging chip 100 integrates a lens 140, and the lens 140 is located on the side of the light modulation layer 110 close to the image sensing layer 120, that is, the lens 140 is located between the light modulation layer 110 and the image sensing layer 120.

[0248] like Figure 23 As shown, the spectral imaging chip 100 integrates a lens 140, and the lens 140 is located on the side of the light modulation layer 110 opposite to the image sensing layer 120.

[0249] like Figure 24 As shown, the spectral imaging chip 100 integrates a filter 150, and the filter 150 is located on the side of the light modulation layer 110 close to the image sensing layer 120, that is, the filter 150 is located between the light modulation layer 110 and the image sensing layer 120.

[0250] like Figure 25 As shown, the spectral imaging chip 100 integrates a filter 150, and the filter 150 is located on the side of the light modulation layer 110 opposite to the image sensing layer 120.

[0251] like Figure 26As shown, the spectral imaging chip 100 integrates a lens 140 and a filter 150, with the lens 140 and filter 150 located on the side of the light modulation layer 110 away from the image sensing layer 120, and the filter 150 located between the lens 140 and the light modulation layer 110.

[0252] like Figure 27 As shown, the spectral imaging chip 100 integrates a lens 140 and a filter 150, and the lens 140 and the filter 150 are located on the side of the light modulation layer 110 close to the image sensing layer 120, that is, the lens 140 and the filter 150 are located between the light modulation layer 110 and the image sensing layer 120, and the filter 150 is located between the lens 140 and the image sensing layer 120.

[0253] like Figure 1 As shown, a spectral recognition device is proposed based on various embodiments of this application. The device includes a light source module 300 and a spectral imaging chip 100. The light source module 300 emits a spectrum toward the object to be imaged 200, so that the spectrum, after reflection from the object 200, forms a target beam that enters the spectral imaging chip 100. The spectral imaging chip 100 and the light source module 300 are simultaneously disposed on the same side of the object 200. The visible-near-infrared spectrum generated by the light source module 300 can illuminate the object 200, and under reflection, the entire spectrum forms a target beam that enters the spectral imaging chip 100. This structural arrangement expands the detection space and improves the ease of use for object imaging and recognition.

[0254] It is understood that the spectral imaging chip 100 and spectral recognition device described in this application can perform imaging recognition on any object 200 to be imaged. It is only necessary to adjust the volume of each modulation unit 111 of the spectral imaging chip 100 and the corresponding wavelength of the incident light according to the imaging requirements.

[0255] Exemplary method for fabricating a spectral imaging chip

[0256] Figure 29 The illustration shows a schematic diagram of another specific example of the fabrication method of the spectral imaging chip according to an embodiment of this application. Here, the spectral imaging chip involved in this application is applied to a computational spectrometer, wherein the most significant difference between a computational spectrometer and a conventional spectrometer lies in the difference in filtering. In a conventional spectrometer, the filter used for wavelength selection is a bandpass filter. The higher the spectral resolution, the narrower and more numerous the passband filters must be, which increases the size and complexity of the entire system. At the same time, when the spectral response curve narrows, the light flux decreases, resulting in a decrease in the signal-to-noise ratio.

[0257] In computational spectrometers, each filter is a broadband filter, making the data detected by the system appear completely different from the original spectrum. However, by applying computational reconstruction algorithms, the original spectrum can be recovered computationally. Since broadband filters allow more light to pass through than narrowband filters, computational spectrometers can detect spectra in darker scenes. Furthermore, based on compressed sensing theory, the spectral curves of the filters can be appropriately designed to recover sparse spectra with high probability, and the number of filters is much smaller than the desired number of spectral channels (recovering a higher-dimensional vector from a lower-dimensional vector), which is undoubtedly very beneficial for miniaturization. On the other hand, by using a larger number of filters, regularization algorithms (obtaining a denoised lower-dimensional vector from a higher-dimensional vector) can be used to reduce noise, which increases the signal-to-noise ratio and makes the entire system more robust.

[0258] In contrast, traditional spectrometers require the design of filters tailored to specific wavelengths (equivalent to the optical modulation structure of a spectral chip) to allow light of that particular wavelength to pass through. These filters are typically designed to enhance the transmission of incident light at specific wavelengths, while blocking incident light of other wavelengths. The resonance conditions can be controlled by altering the period and diameter of structures such as nanodisks, thus changing the center wavelength of the incident light that can be enhanced, thereby achieving the filtering characteristics. In other words, traditional spectrometers require careful control of the size and positional accuracy of the optical modulation structure while simultaneously finding ways to improve the transmittance at specific wavelengths. Computational spectrometers, however, need to receive light across a wider wavelength range (e.g., 350nm to 900nm), therefore, the design must focus more on the refractive index.

[0259] Specifically, in this embodiment, the spectral imaging chip is fabricated using the aforementioned method, such that a light modulation layer with a regular crystal orientation structure is formed on its surface. This light modulation layer has a relatively large refractive index, enabling the collection and utilization of light across a relatively wide wavelength range. Furthermore, in this embodiment, the light modulation layer has at least one modulation unit and at least one non-modulation unit along its surface. The modulation unit is configured to collect optical frequency information to obtain the spectral information of the subject, and the non-modulation unit is configured to collect light intensity information to obtain the image information of the subject.

[0260] like Figure 29 As shown, the fabrication process of the spectral imaging chip according to the embodiments of this application first includes providing a spectral chip semi-finished product 400, wherein the spectral chip semi-finished product 400 includes an image sensing layer 410 and a signal processing circuit layer 420 connected to the image sensing layer 410.

[0261] In this specific example, the spectral chip semi-finished product 400 can be provided by a manufacturer or obtained by processing an existing photosensitive chip. Those skilled in the art will know that existing photosensitive chips, such as CCD photosensitive chips and CMOS photosensitive chips, include a microlens layer, a color filter layer (here, if it is a monochrome chip, the color filter layer is not included), an image sensing layer 410, and a signal processing circuit layer 420. Accordingly, the spectral chip semi-finished product 400 can be obtained by removing the microlens layer and color filter layer of the existing photosensitive chip (if it is a monochrome chip, only the microlens layer needs to be removed).

[0262] like Figure 29 As shown, in this specific example, the fabrication process of the spectral imaging chip further includes pre-treating the surface of the spectral chip semi-finished product 400 to form a flat bonding surface on the surface of the spectral chip semi-finished product 400 for bonding the transfer member 500 having the target transfer layer 510.

[0263] Specifically, in this specific example, the process of pre-treating the surface of the spectral chip semi-finished product 400 includes: forming a light-transmitting medium layer 430 on the surface of the spectral chip semi-finished product 400, wherein the light-transmitting medium layer 430 is made of a light-transmitting material and has a relatively high light transmittance so that it does not affect the light entering the spectral chip semi-finished product 400.

[0264] It is worth mentioning that, in specific implementation, although the light-transmitting dielectric layer 430 needs to have a relatively high refractive index, the refractive index of the light-transmitting dielectric layer 430 should not be too high. The reason is that it is necessary to ensure the difference in refractive index between the light-transmitting dielectric layer 430 and the semiconductor structure layer located thereon.

[0265] In this specific example, the material used to make the light-transmitting dielectric layer 430 is preferably a silicide, such as silicon dioxide or silicon nitride. Those skilled in the art will know that silicon dioxide has a refractive index of approximately 1.45, while silicon nitride has a refractive index between 1.9 and 2.3.

[0266] In specific implementations, the light-transmitting dielectric layer 430 can be formed on the surface of the spectral chip semi-finished product 400 using a process such as non-metallic vapor deposition. Of course, in other embodiments of this specific implementation, the light-transmitting dielectric layer 430 can also be formed using other processes. This application does not limit the formation process. Specifically, in this specific example, the thickness of the light-transmitting dielectric layer 430 is not limited by this application; its specific value can be adjusted according to the specific needs of the application scenario. Generally, its thickness is less than or equal to 300 nm, and in some special scenarios, it is even less than 100 nm.

[0267] like Figure 29 As shown, preferably, the upper surface of the light-transmitting medium layer 430 is a flat surface, or in other words, the portion of the upper surface of the light-transmitting medium layer 430 used to bond the transfer member 500 has a relatively high degree of flatness, so as to facilitate the transfer of the transfer member 500 onto the spectral chip semi-finished product 400.

[0268] It is worth noting that in some cases of this specific example, the surface of the spectral chip semi-finished product 400 may be uneven, and the upper surface of the light-transmitting dielectric layer 430 formed on the surface of the spectral chip semi-finished product 400 by the deposition process may also be uneven. Therefore, in this specific implementation, the pretreatment process further includes: polishing the surface of the spectral chip semi-finished product 400, and / or polishing the upper surface of the light-transmitting dielectric layer 430. Here, the polishing process can be chemical mechanical polishing, or other processes that can increase surface smoothness; this application does not limit this.

[0269] It is worth mentioning that, in this specific example, if the surface flatness of the spectral chip semi-finished product 400 meets the preset requirements, the light-transmitting medium layer 430 may not be provided on the surface of the spectral chip semi-finished product 400, that is, the spectral chip semi-finished product 400 does not need to be pre-treated.

[0270] Furthermore, such as Figure 29 As shown, the fabrication process of the spectral imaging chip further includes providing a transfer element 500. Specifically, in this particular example, the transfer element 500 is selected as an SOI device (Silicon on Insulator), which, from bottom to top, comprises: a silicon substrate layer 511, a silicide layer 512, and a silicon crystal layer 513. The silicon crystal layer 513 is the target transfer layer 510 of the transfer element 500; that is, in this particular example, the target transfer layer 510 of the transfer element 500 is located on the topmost layer of the transfer element 500. Those skilled in the art should know that SOI devices are existing components. Using an off-the-shelf component as the transfer element 500 reduces costs, and the existing device technology is mature and has stable and predictable performance. Of course, the transfer element 500 in this embodiment can also be other types of devices, such as a silicon rod, silicon wafer, or a combination of silicon crystal and silicide.

[0271] Furthermore, those skilled in the art should understand that in the SOI device, the atomic arrangement in the silicon substrate layer 511, the silicide layer 512, and the silicon crystal layer 513 is regular, that is, all three have good crystal orientation structures. Preferably, in this specific example, the surface of the silicon crystal layer 513 is a flat surface.

[0272] like Figure 29 As shown, the fabrication process of the spectral imaging chip further includes: coupling the transfer member 500 to the spectral chip semi-finished product 400 by bonding the upper surface of the transfer member 500 to the upper surface of the light-transmitting dielectric layer 430 of the spectral chip semi-finished product 400. That is, the SOI device is transferred to the spectral chip semi-finished product 400 by bonding the surface of the silicon crystal layer 513 of the SOI device to the upper surface of the light-transmitting dielectric layer 430 of the spectral chip semi-finished product 400.

[0273] To ensure the bonding strength between the transfer member 500 and the spectral chip semi-finished product 400, preferably, in this embodiment, the upper surface of the transfer member 500 has a good bonding reaction with the upper surface of the light-transmitting medium layer 430, so that the two surfaces can generate a good bonding reaction and a greater bonding force during bonding. For example, in this specific example, the upper surface of the transfer member 500 is configured to have the same material as the light-transmitting medium layer 430, so that the two can generate a good bonding reaction and a greater bonding force during bonding.

[0274] Taking silicon dioxide as an example of the light-transmitting dielectric layer 430, it should be understood that in this specific example, the upper surface of the transfer member 500 is formed by the surface of the silicon crystal layer 513. Therefore, in this specific embodiment, before bonding the upper surface of the transfer member 500 to the upper surface of the light-transmitting dielectric layer 430, the process further includes: treating the upper surface of the transfer member 500 so that the upper surface of the transfer member 500 is made of silicon dioxide material.

[0275] In a specific implementation, oxygen ions can be injected into the surface of the silicon crystal layer 513 to partially form a silicon dioxide layer on the surface of the silicon crystal layer 513, so that the upper surface of the transfer member 500 is formed of silicon dioxide. It should be understood that the silicon crystal layer 513 has a regular crystal orientation structure; therefore, the silicon dioxide layer also has a regular crystal orientation structure, which is beneficial to improving its bonding effect with the light-transmitting dielectric layer 430.

[0276] Of course, in other embodiments of this application, a bonding layer 520 may also be stacked on the surface of the silicon crystal layer 513, wherein the bonding layer 520 is made of silicon dioxide material, for example, the bonding layer 520 is formed on the surface of the silicon crystal layer 513 by a non-metallic vapor deposition process, so as to improve the bonding strength between the transfer member 500 and the spectral chip semi-finished product 400 through the bonding layer 520.

[0277] It is worth mentioning that, in this specific example, the process of processing the upper surface of the transfer member 500 can also be completed during the step of providing the transfer member 500, and this is not limited to this application.

[0278] like Figure 29 As shown, the fabrication process of the spectral imaging chip further includes: exposing the target transfer layer 510 of the transfer member 500, that is, exposing the silicon crystal layer 513 of the transfer member 500. In this specific example, one or a combination of processes such as mechanical polishing, chemical mechanical polishing, and etching can be used to remove the silicon substrate layer 511 and the silicide layer 512, so that the silicon crystal layer 513 of the transfer member 500 is exposed.

[0279] It is worth mentioning that mechanical grinding is efficient but has poor precision, while chemical mechanical polishing and etching processes are inefficient but have high precision. Therefore, in this specific example, it is preferable to first use mechanical grinding and polishing to perform a first-stage treatment on the silicon substrate layer 511 and the silicide layer 512, and then use chemical mechanical polishing or etching processes to perform a second-stage treatment on the silicon substrate layer 511 and the silicide layer 512, so as to balance efficiency and precision.

[0280] Specifically, in this embodiment, the refractive index of the silicon crystal layer 513 is about 3.42, and the difference in refractive index between the silicon crystal layer 513 and the light-transmitting medium layer 430 is greater than or equal to 0.5, preferably greater than or equal to 0.7.

[0281] Specifically, in this particular example, the spectral chip has certain requirements regarding the thickness of the silicon crystal layer 513. The thickness of the silicon crystal layer 513 ranges from 5 nm to 1000 nm, preferably from 50 nm to 750 nm. This thickness facilitates the processing of the silicon substrate layer 511, thereby optimizing and ensuring the imaging effect of the spectral chip. More preferably, the thickness of the silicon crystal layer 513 is between 150 nm and 250 nm.

[0282] Accordingly, in this specific example, in order to meet the thickness requirements, the process of removing the silicon substrate layer 511 and the silicide layer 512 further includes removing a portion of the silicon crystal layer 513 so that the thickness of the silicon crystal layer 513 meets the preset requirements.

[0283] like Figure 29 As shown, the fabrication process of the spectral imaging chip further includes: forming a light modulation layer 501 on the exposed silicon crystal layer 513, so that when external imaging light enters the interior of the spectral imaging chip through the light modulation layer 501, the light modulation layer 501 can modulate the imaging light to extract and utilize the optical frequency information and light intensity information in the imaging light.

[0284] Specifically, in this embodiment, the process of forming the light modulation layer 501 on the exposed silicon crystal layer 513 includes: first, applying photoresist to the silicon crystal layer 513; then, exposing the photoresist with a mask having a specific pattern; next, removing a predetermined portion of the silicon crystal layer 513 and the photoresist based on the specific pattern of the mask by an etching process, wherein the unetched portion of the silicon crystal layer 513 forms the light modulation layer 501.

[0285] like Figure 29 As shown, in this embodiment, the optical modulation layer 501 includes modulation units 5010 formed in its central region and non-modulation units 5011 formed around the modulation units 5010. That is, in this embodiment, the modulation units 5010 of the optical modulation layer 501 are located in the central region of the optical modulation layer 501, and the non-modulation units 5011 are located around the optical modulation units 5010. It should be understood that in other examples of this application, the modulation units 5010 and non-modulation units 5011 of the optical modulation layer 501 can be arranged in other ways, and this is not limited to this application.

[0286] Specifically, each modulation unit 5010 can be a micro / nano structure unit used to modulate the light beam entering the spectral imaging chip. Each non-modulation unit 5011 does not have modulation capability and cannot modulate the target light beam; instead, it directly allows the imaging light to pass through. Furthermore, in this embodiment, each modulation unit 5010 corresponds to at least one sensing unit 4101 of the image sensing layer 410. The imaging light, after being modulated by the modulation unit 5010, reaches the corresponding sensing unit 4101, which is used to acquire the optical frequency information in the imaging light. Similarly, each non-modulation unit 5011 also corresponds to at least one sensing unit 4101 of the image sensor 410. The imaging light, after passing through the non-modulation unit 5011, reaches the corresponding sensing unit 4101, which is used to acquire the light intensity information in the imaging light. It is worth noting that different modulation units 5010 may have the same or different modulation effects on different wavelengths of light, and can be set as needed. This embodiment does not specifically limit this.

[0287] Specifically, in this particular example, the refractive index of the light modulation layer 501 is between 1 and 5, and the difference between the refractive index of the light modulation layer 501 and the refractive index of the light-transmitting medium layer 430 is greater than or equal to 0.5, preferably greater than or equal to 0.7. In this way, light of a relatively large range of wavelengths can pass through the light modulation layer 501 and then through the light-transmitting medium layer 430 and the image sensing layer 410 of the spectral chip.

[0288] It is worth mentioning that, in this specific example, the atoms within the silicon crystal layer 513 of the transfer member 500 have a regular crystal orientation distribution, and the internal structure of the silicon crystal layer 513 remains unchanged when it is transferred to the surface of the spectral chip semi-finished product 400 by the fabrication method described above. Therefore, the spectral chip prepared according to the fabrication method disclosed in this specific example has a light modulation layer 410 with a preferred crystal orientation arrangement formed on its surface.

[0289] like Figure 29 As shown in this embodiment, the fabrication method of the spectral imaging chip further includes: forming a filter layer 600 at the non-modulation unit 5011 of the light modulation layer 501, so that when the imaging light passes through the non-modulation unit 5011, the filter layer 600 can perform color filtering processing on the sensing unit 4101 entering the image sensing layer 410 to obtain light intensity information of a specific wavelength band in the imaging light. Here, the filter layer 600 can be implemented as a Bayer filter layer.

[0290] In particular, in some specific examples of this application, the thickness of the filter layer 600 is similar to or equal to the thickness of the optical modulation unit 5010, so that the optical modulation layer 501 and the filter layer 600 form a relatively flat surface.

[0291] In some embodiments, the spectral imaging chip does not need to form a filter layer 600. In this case, the non-modulation unit 5011 may be left unprocessed and implemented as a silicon crystal layer 513, through which light can pass directly.

[0292] like Figure 29 As shown in the embodiment of this application, the method for fabricating the spectral imaging chip further includes: forming a microlens layer 700 on the upper surface of the light modulation layer 501 and / or the filter layer, wherein the microlens layer 700 includes a plurality of microlens units for converging the imaging light entering the light modulation layer 501 and the filter layer 600.

[0293] It is worth mentioning that, in the embodiments of this application, the method for fabricating the spectral imaging chip may also omit the formation of the microlens layer 700 on the upper surface of the light modulation layer 501 and / or the filter layer. That is, in some examples of this application, the spectral imaging chip does not include the microlens layer 700, which is not limited to this application.

[0294] In summary, the spectral imaging chip and its fabrication method based on this specific example are explained, in which a silicon crystal layer 513 with a better crystal orientation is migrated to the surface of the spectral chip semi-finished product 400 by a specific fabrication method, so that the surface of the finally fabricated spectral imaging chip has a light modulation layer 501 with a better crystal orientation.

[0295] Figure 30 The illustration shows a schematic diagram of a modified embodiment of the fabrication method of the spectral imaging chip according to an embodiment of this application. For example... Figure 30 As shown, in this modified embodiment, before the transfer member 500 is transferred to the spectral chip semi-finished product 400 via a bonding process, the silicon crystal layer 513 of the transfer member 500 is pretreated to form the light modulation layer 501, wherein the thickness of the light modulation layer 501 is 200-1000 nm, preferably 350-600 nm. Accordingly, when the silicon crystal layer 513 is subsequently exposed, the light modulation layer 501 is also exposed simultaneously.

[0296] That is, compared to Figure 29 In the illustrated fabrication scheme, in this modified embodiment, the light modulation layer 501 is prefabricated on the transfer member 500, or in other words, the process of forming the light modulation layer 501 is advanced.

[0297] Exemplary fingerprint liveness detection device and fingerprint module

[0298] Figure 31a This is a schematic diagram of the structure of a fingerprint liveness recognition device provided in an embodiment of the present invention. Figure 31b This is a top view of a fingerprint liveness detection device provided in an embodiment of the present invention. Figure 31a and Figure 31b As shown, the fingerprint liveness recognition device includes: a display screen 800, a light source module 300, a spectral imaging chip 100, a signal processing circuit module, and a recognition module 810. Figure 31a and Figure 31b The signal processing circuit module is not shown separately because it can be integrated into the spectral imaging chip 100 or as a separate component from the spectral imaging chip 100 in the fingerprint liveness recognition device, such as a computer. This embodiment of the invention does not impose specific limitations on this. The display screen 800 may include a cover glass, a touch module, and a display module. A fingerprint detection area 800a is provided on the display screen 800, within which the object to be detected 200a can be placed. The object to be detected 200a can be the user's own live fingerprint, another person's live fingerprint, or a sticker or 3D printed model carrying fingerprint information. This embodiment of the invention does not impose specific limitations on this.

[0299] The light source module 300 is used to illuminate the object 200a to be detected within the fingerprint detection area. The light source module 300 can be a separate component independent of the display screen, such as an LED light source or a laser light source of a specific wavelength, such as a 940nm laser. Alternatively, the light source module can be integrated into the display screen 800, becoming part of the self-emissive display module within the display screen 800; however, this is not specifically limited in this embodiment of the invention. A lens may be provided on the upper surface of the light source module 300 to focus the light beam emitted by the light source module.

[0300] After being reflected by the object to be detected 200a, a target beam is obtained, which is then incident on the spectral imaging chip 100. After illumination by the target beam, each pixel corresponding to a modulation unit in the light modulation layer of the spectral imaging chip 100 possesses spectral information, and each pixel corresponding to a non-modulation unit in the light modulation layer possesses light intensity information. Spectral information refers to the light intensity information corresponding to different wavelengths of light at each pixel corresponding to a modulation unit. Different modulation units may have the same or different modulation effects on different wavelengths of light, and this can be set as needed; however, this is not specifically limited in this embodiment. Therefore, the spectral imaging chip 100 can determine the spectral information and light intensity information. Then, the signal processing circuit module determines the spectral information and image information of the object to be detected 200a based on the spectral information and light intensity information, respectively. When the signal processing circuit module is integrated within the spectral imaging chip, it can be considered that the spectral imaging chip has the function of determining the spectral information and image information of the object to be detected 200a.

[0301] In a fingerprint liveness detection device, the vertical distance from the light source module to the display screen can be between 0mm and 30mm, and correspondingly, the vertical distance from the light source module to the spectral imaging chip can also be between 0mm and 30mm. Alternatively, the vertical distance from the light source module to the display screen can be between 0.5mm and 20mm, and correspondingly, the vertical distance from the light source module to the spectral imaging chip can also be between 0.5mm and 20mm. It is worth noting that, for example, in some embodiments, the spectral imaging chip is attached to the display screen using adhesive. Even if there is adhesive between the spectral imaging chip and the display screen, it can be understood that the distance is 0mm, meaning that errors caused by assembly, fixing, etc., do not affect the starting point of this invention.

[0302] The identification module 810 is electrically connected to the signal processing circuit module. The identification module 810 can acquire the spectral and image information of the object to be detected 200a determined by the signal processing circuit module, and identify whether the object to be detected 200a is a live fingerprint of the target user based on the spectral and image information. The identification module 810 can pre-store the fingerprint image information and fingerprint spectral information of the target user. First, it compares the image information of the object to be detected 200a determined by the spectral imaging chip 100 with the pre-stored fingerprint image information of the target user in the identification module 810. If they are the same or the error is within a preset range, it can be determined that the object belongs to the target user. Then, it compares the spectral information of the object to be detected 200a determined by the spectral imaging chip 100 with the pre-stored fingerprint spectral information in the identification module 810. If they are the same or the error is within a preset range, it can be determined that the object to be detected is a live fingerprint of the target user.

[0303] The fingerprint liveness recognition device provided in this embodiment of the invention includes a display screen, a light source module, a spectral imaging chip, a signal processing circuit module, and a recognition module. The display screen has a fingerprint detection area. The light source module illuminates the object to be detected within the fingerprint detection area. The target light beam, reflected by the object, is incident on the spectral imaging chip. The spectral imaging chip determines the spectral information and light intensity information, and the signal processing circuit module determines the spectral information and image information of the object to be detected. Finally, the recognition module identifies whether the object to be detected is a live fingerprint of the target user based on the spectral information and image information of the object. Compared with existing fingerprint recognition devices, the fingerprint liveness recognition device provided in this embodiment of the invention can achieve fingerprint liveness recognition, which is beneficial to improving the stability of device performance. It also reduces the size, weight, and cost of the spectral device, greatly improving the anti-counterfeiting capability of the fingerprint recognition system. Moreover, the spectral imaging chip used in this embodiment of the invention, compared with traditional image sensors, can obtain spectral information without affecting the spatial resolution and imaging quality of the formed image, facilitating a more comprehensive understanding of the object to be detected. In addition, since the spectral information of the object to be detected can be used to uniquely identify the object, it is possible to determine whether the object to be detected is the live fingerprint of the target user by using the spectral information of the object to be detected, which can improve the accuracy of detection and make fingerprint identity verification through fingerprint liveness recognition device more secure and reliable.

[0304] Based on the above embodiments, the fingerprint liveness recognition device provided in this embodiment of the invention includes a spectral imaging chip comprising: a light modulation layer and an image sensing layer sequentially stacked along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing units. Specifically, it is applied in the fingerprint liveness recognition device of this embodiment of the invention. Figure 3 The structure of a spectral imaging chip. For example... Figure 3As shown, the spectral imaging chip 100 may include: a light modulation layer 110, an image sensing layer 120, and a signal processing circuit module 130, that is, in this embodiment of the invention, the signal processing circuit module 130 is integrated within the spectral imaging chip 100. The light modulation layer 110, the image sensing layer 120, and the signal processing circuit module 130 are sequentially stacked along the thickness direction. The light modulation layer 110 has at least one modulation unit 1101 and at least one non-modulation unit 1102 distributed along its surface. The image sensing layer 120 has a plurality of sensing units 1201 distributed along its surface, and each modulation unit 1101 and each non-modulation unit 1102 corresponds to at least one sensing unit 1201 along the thickness direction. At least one modulation unit 1101 and the plurality of non-modulation units 1102 surrounding it, together with the corresponding sensing unit 120, constitute a pixel of the spectral imaging chip 100. The signal processing circuit module 130 is electrically connected to the sensing unit 1201 on the image sensing layer 120. The signal processing circuit module 130 is used to determine the image information and spectral information of the object to be detected.

[0305] The specific implementation is the same as that of the spectral imaging chip, and will not be repeated here.

[0306] Each modulation unit 1101 can be a micro / nano structure unit used to modulate the target beam, while each non-modulation unit 1102 has no modulation capability and cannot modulate the target beam. Each non-modulation unit 1102 can be a blank unit or a solid unit that does not have modulation capability but allows the target beam to pass through directly and is made of the same material as the modulation unit. Each modulation unit 1101 on the optical modulation layer 110 can be directly fabricated on the photosensitive area surface of the CIS wafer, while the areas on the CIS wafer without modulation units are still traditional RGB or black and white pixels. These areas correspond to the non-modulation units on the optical modulation layer. Since the optical modulation layer of the spectral imaging chip includes both modulation units and non-modulation units, it can acquire both the spectral information of the pixel corresponding to each modulation unit (i.e., the spectral information modulated by each modulation unit and detected by the corresponding sensing unit) and the light intensity information of the pixel corresponding to each non-modulation unit (i.e., the light intensity information detected by the corresponding sensing unit after passing through each non-modulation unit).

[0307] Based on the above embodiments, the fingerprint liveness recognition device provided in this embodiment of the invention, wherein the signal processing circuit module is specifically used for:

[0308] Based on the light intensity information of the pixels corresponding to multiple non-modulation units around each modulation unit after the target beam is irradiated, the fitted light intensity information of the pixels corresponding to each modulation unit is determined.

[0309] Based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit, the image information of the object to be detected is determined.

[0310] Specifically, in this embodiment of the invention, when determining the image information of the object to be detected, the signal processing circuit module can first determine the fitted light intensity information of the pixel corresponding to each modulation unit based on the light intensity information of the pixel corresponding to at least one non-modulation unit around each modulation unit after the target beam is irradiated. "Around each modulation unit" refers to a position within a preset range of distance from the modulation unit. The preset range can be set as needed, and this embodiment of the invention does not specifically limit it. For example, the area around each modulation unit can be one of the eight positions closest to the modulation unit: left, upper left, upper, upper right, right, lower right, lower, and lower left. It can also be the second closest position to the modulation unit. The number of non-modulation units around each modulation unit is related to the structure of the optical modulation layer. The optical modulation layer is generally a rectangular structure. Taking the position closest to the modulation unit as an example, each unit at the vertex position has 3 units around it, each unit at other edge positions (excluding the vertex position) has 5 units around it, and each unit at other edge positions (excluding the vertex position) has 8 units around it. If the structure of the optical modulation layer is as follows... Figure 4 As shown, each modulation unit can be surrounded by a maximum of 5 non-modulation units and a minimum of 3 non-modulation units; if the structure of the optical modulation layer is as shown... Figure 5 As shown, each modulation unit can be surrounded by a maximum of 6 non-modulation units and a minimum of 0 non-modulation units; if the structure of the optical modulation layer is as shown... Figure 6 As shown, each modulation unit can be surrounded by a maximum of 4 non-modulation units and a minimum of 0 non-modulation units; if the structure of the optical modulation layer is as follows... Figure 7 As shown, each modulation unit is surrounded by 8 non-modulation units.

[0311] Based on the above embodiments, the fingerprint liveness recognition device provided in this embodiment of the invention further includes: a lens group;

[0312] The light source module is disposed between the display screen and the spectral imaging chip, and the lens group is disposed between the light source module and the spectral imaging chip; or...

[0313] The light source module is located below the non-fingerprint detection area of ​​the display screen, and the lens group and the spectral imaging chip are sequentially located below the fingerprint detection area;

[0314] The lens group is used to collimate the target beam onto the spectral imaging chip so that the target beam is imaged on the spectral imaging chip.

[0315] Specifically, such as Figure 32 As shown, the lens group 900 is disposed between the display screen 800 and the spectral imaging chip 100, and the lens group 900 is disposed between the light source module 300 and the spectral imaging chip 100.

[0316] like Figure 33 As shown, the light source module 300 is located below the non-fingerprint detection area of ​​the display screen 800, and the lens group 900 and the spectral imaging chip 100 are sequentially located below the fingerprint detection area 800a of the display screen 800.

[0317] The lens assembly may integrate a filter to filter out light within a specific wavelength range of interest, such as light from 400nm to 598nm, and to remove other interfering light. The vertical thickness of the lens module can be 0.5mm-20mm. As a preferred embodiment, the vertical thickness of the lens module can be 5mm-10mm.

[0318] The lens group can smoothly guide the target beam onto the spectral imaging chip, ensuring that the target beam is imaged on the spectral imaging chip.

[0319] Furthermore, such as Figure 34 As shown, the present invention provides an optical fingerprint recognition component, which includes a display screen 800 and a fingerprint module 1000, wherein the fingerprint module 1000 is disposed at the lower end of the display screen 800, and the display screen 800 can be an LCD screen, an OLED screen, etc. Further, the optical fingerprint recognition component also includes a light source module 300, which is disposed in the fingerprint liveness detection device and corresponds to the fingerprint module 1000. During recognition, the light source module 300 emits light to the object to be detected 200a, and after reflection by the object to be detected 200a, a target beam is obtained, which is then received by the fingerprint module 1000.

[0320] In some embodiments, the fingerprint module does not necessarily have to be used in conjunction with the display screen. That is, in this embodiment, the terminal product does not need to be equipped with the display screen, and the fingerprint module can still achieve fingerprint recognition.

[0321] It is worth mentioning that, in some embodiments, the light source module 300 is integrated into the display screen 800. For example, when the display screen 800 is implemented as an OLED screen, the light beam projected by the display screen itself can be used to assist in identifying the object to be detected 200a.

[0322] Furthermore, such as Figure 35As shown, the fingerprint module 1000 includes a spectral imaging chip 100 and a circuit board 1001. The spectral imaging chip 100 is electrically connected to the circuit board 1001. Preferably, the spectral imaging chip 100 is attached to the circuit board 1001. The circuit board 1001 can be implemented as a rigid board (PCB), a flexible printed circuit (FPC), a rigid-flex PCB (F-PCB), a ceramic substrate, etc., and is mainly used for conducting and / or supporting the spectral imaging chip 100.

[0323] Furthermore, for example, the spectral imaging chip 100 is attached to the circuit board 1001. The spectral imaging chip 100 can be connected to the circuit board 1001 by wire bonding or by direct back-mounted pad connection. In some embodiments, the fingerprint module further includes a reinforcing plate 1002, which is attached to the circuit board 1001 to enhance the reliability of the fingerprint module 1000. In some embodiments, the fingerprint module 1000 further includes an encapsulation portion 1003, which is formed on the upper surface of the circuit board 1001. The encapsulation portion 1003 can be integrally molded onto the circuit board using a molding process or applied to the circuit board 1001 as an adhesive. It is worth mentioning that the encapsulation portion 1003 can also integrally wrap the conductive leads. The encapsulation portion 1003 helps improve the reliability of the fingerprint module. Furthermore, the encapsulation portion 1003 has a flat surface, allowing the fingerprint module to better adhere to the display screen 800. Due to the presence of the reinforcing plate 1002, the circuit board can be implemented as a flexible board, thereby reducing the thickness of the fingerprint module. In some embodiments, the fingerprint module 1000 may further include a light-shielding portion 1004, which is disposed on the upper surface of the encapsulation portion 1003 to prevent stray light from entering and affecting imaging. The light-shielding portion can be implemented as foam.

[0324] Another embodiment of the present invention, such as Figure 36 As shown, the fingerprint module 1000 includes a spectral imaging chip 100, a circuit board 1001, a bracket 1005, and a light source module 300. The spectral imaging chip 100 and the bracket 1005 are fixed to the circuit board 1001, and the light source module 300 is fixed to the bracket 1005.

[0325] Furthermore, in this embodiment of the invention, the fingerprint module may also include a lens group for optical focusing, and the lens group is disposed on the light-sensing path of the spectral imaging chip 100. The lens assembly may be a vertical collimating lens, a thin-film lens, a microlens array, a wide-angle lens, etc.

[0326] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0327] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0328] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spectral imaging chip, characterized in that, include: The image sensing layer includes multiple sensing units; as well as A light modulation layer located on the photosensitive path of the image sensing layer, the light modulation layer including at least one modulation unit and at least one non-modulation unit spaced apart from the at least one modulation unit, wherein each of the modulation units corresponds to at least one sensing unit along the photosensitive path, and each of the non-modulation units corresponds to at least one sensing unit along the photosensitive path. The modulation unit of the optical modulation layer is configured to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the spectral information of the imaging light; the non-modulation unit of the optical modulation layer is configured not to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light. The density of the modulation units affects the imaging quality and spectral analysis of the spectral imaging chip. The density of the modulation units is defined by sparsity A, and the formula for calculating sparsity A is as follows: A= Specifically, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit is represented by the geometric center of the modulation unit. The point corresponding to the modulation unit is encoded from 1 to n, d ij This represents the distance between the geometric center of the i-th modulation unit and the geometric center of the j-th modulation unit; The sparsity A is selected within a range greater than 0 and less than or equal to 1 to correspond to the performance requirements of the spectral imaging chip.

2. The spectral imaging chip according to claim 1, wherein, The at least one modulation unit and the at least one non-modulation unit are located in the same plane.

3. The spectral imaging chip according to claim 2, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for a proportion of the area of ​​the effective region of the optical modulation layer that is greater than 0 and less than or equal to 25%, wherein the effective region of the optical modulation layer is the region defined by the modulation unit and the non-modulation unit of the optical modulation layer.

4. The spectral imaging chip according to claim 3, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than or equal to 5% and less than or equal to 15% of the area of ​​the effective region of the optical modulation layer.

5. The spectral imaging chip according to claim 2, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than 25% and less than or equal to 45% of the area of ​​the effective region of the optical modulation layer, wherein the effective region of the optical modulation layer is the region defined by the modulation unit and the non-modulation unit of the optical modulation layer.

6. The spectral imaging chip according to claim 5, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than 25% and less than or equal to 35% of the area of ​​the effective region of the optical modulation layer.

7. The spectral imaging chip according to claim 2, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than 45% and less than or equal to 60% of the area of ​​the effective region of the optical modulation layer, wherein the effective region of the optical modulation layer is the region defined by the modulation unit and the non-modulation unit of the optical modulation layer.

8. The spectral imaging chip according to claim 7, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than or equal to 50% and less than or equal to 55% of the area of ​​the effective region of the optical modulation layer.

9. The spectral imaging chip according to claim 2, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than 60% of the area of ​​the effective region of the optical modulation layer, wherein the effective region of the optical modulation layer is the region defined by the modulation unit and the non-modulation unit of the optical modulation layer.

10. The spectral imaging chip according to claim 9, wherein, The area of ​​the modulation unit of the optical modulation layer accounts for more than or equal to 80% and less than or equal to 95% of the area of ​​the effective region of the optical modulation layer.

11. The spectral imaging chip according to claim 2, wherein, The non-modulation units of the optical modulation layer are located in the edge region or the middle region of the optical modulation layer.

12. The spectral imaging chip according to claim 1, further comprising a filter layer formed at the non-modulation unit of the optical modulation layer.

13. The spectral imaging chip according to claim 2, wherein, The sparsity A is greater than 0 and less than or equal to 0.

5.

14. The spectral imaging chip according to claim 13, wherein, The sparsity A is greater than 0 and less than or equal to 0.

15.

15. The spectral imaging chip according to claim 2, wherein, The sparsity A is greater than or equal to 0.5 and less than 1.

16. The spectral imaging chip according to claim 15, wherein, The sparsity A is greater than or equal to 0.85 and less than 1.

17. The spectral imaging chip according to claim 1, further comprising: The signal processing circuit layer is electrically connected to the image sensing layer.

18. The spectral imaging chip according to claim 12, further comprising: A microlens layer formed in the modulation unit of the filter layer and / or the light modulation layer.

19. The spectral imaging chip according to claim 1, further comprising: A light-transmitting medium layer integrally formed on the image sensing layer, wherein the upper surface of the light-transmitting medium layer is a flat surface.

20. The spectral imaging chip according to claim 19, wherein, The light modulation layer is integrally bonded to the light-transmitting medium layer.

21. The spectral imaging chip according to claim 20, wherein, The difference in refractive index between the light modulation layer and the light-transmitting medium layer is greater than or equal to 0.

5.

22. The spectral imaging chip according to claim 20, wherein, The light modulation layer is integrally bonded to the light-transmitting medium layer via a bonding layer, and the light-transmitting medium layer is bonded to the bonding layer.

23. The spectral imaging chip according to claim 22, wherein, The light-transmitting medium layer and the bonding layer are made of the same material.

24. The spectral imaging chip according to claim 2, wherein, Each of the modulation units includes a plurality of modulation sub-units arranged along the photosensitive path.

25. The spectral imaging chip according to claim 24, wherein, Each of the modulation subunits has a porous structure or a columnar structure.

26. The spectral imaging chip according to claim 25, wherein, The cross-sectional shape of the aperture in different modulation subunits with aperture structures in each modulation unit is not exactly the same; and / or, the structural parameters of different modulation subunits with aperture structures in each modulation unit are not exactly the same.

27. An information processing method suitable for spectral imaging chips, characterized in that, include: Acquire the spectral information of the pixel corresponding to each modulation unit in the optical modulation layer of the spectral imaging chip, and the light intensity information of the pixel corresponding to each non-modulation unit in the modulation layer; as well as Based on the spectral information of the pixel corresponding to each modulation unit, the spectral information of the subject is determined, and based on the light intensity information of the pixel corresponding to each non-modulation unit, the image information of the subject is determined. The density of the modulation units affects the imaging quality and spectral analysis of the spectral imaging chip. The density of the modulation units is defined by sparsity A, and the formula for calculating sparsity A is as follows: A= Specifically, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit is represented by the geometric center of the modulation unit. The point corresponding to the modulation unit is encoded from 1 to n, d ij This represents the distance between the geometric center of the i-th modulation unit and the geometric center of the j-th modulation unit; The sparsity A is selected within a range greater than 0 and less than or equal to 1 to correspond to the performance requirements of the spectral imaging chip.

28. The information processing method suitable for a spectral imaging chip according to claim 27, wherein, The modulation unit of the light modulation layer is configured to modulate the imaging light entering its corresponding pixel, and the corresponding pixel is adapted to acquire the spectral information of the imaging light; wherein, the non-modulation unit of the light modulation layer is configured not to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light.

29. The information processing method suitable for a spectral imaging chip according to claim 28, wherein, The modulation unit of the optical modulation layer is located in the middle region of the optical modulation layer or the edge region of the optical modulation layer.

30. The information processing method suitable for a spectral imaging chip according to claim 28, wherein, Based on the light intensity information of the pixel corresponding to each of the unmodulated units, the image information of the photographed object is determined, including: Based on the light intensity information of the pixel corresponding to at least one of the non-modulation units around each modulation unit, the fitted light intensity information of the pixel corresponding to each modulation unit is determined. The image information of the subject is determined based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit.

31. The information processing method suitable for a spectral imaging chip according to claim 30, wherein, Based on the light intensity information of pixels corresponding to at least one of the non-modulation units surrounding each modulation unit, the fitted light intensity information of pixels corresponding to each modulation unit is determined, including: Based on the smoothing filtering method, the light intensity information of the pixel corresponding to at least one non-modulation unit around any of the modulation units is filtered to obtain the fitted light intensity information of the pixel corresponding to any of the modulation units.

32. The information processing method suitable for a spectral imaging chip according to claim 31, wherein, Based on the fitted light intensity information of the pixels corresponding to each modulation unit and the light intensity information of the pixels corresponding to each non-modulation unit, the image information of the subject is determined, including: An initial image obtained based on the light intensity information of the pixels corresponding to all the non-modulation units is input into the fitting model to obtain the image information of the photographed object output by the fitting model; The fitting model is constructed based on an adversarial neural network and is trained on labels of empty sample images containing blank pixels and complete sample images corresponding to the empty sample images that do not contain blank pixels.

33. The information processing method suitable for a spectral imaging chip according to claim 32, wherein, The training process of the fitted model includes: The generator in the adversarial neural network is trained based on the missing sample images and the corresponding complete sample image labels, and the trained generator is competitively discriminated against based on the discriminator in the adversarial neural network; and The generator obtained from the training is used as the fitted model.

34. A spectral imaging device, characterized in that, include: The spectral imaging chip and the signal processing circuit module are either two independent parts or the signal processing circuit module is integrated into the spectral imaging chip. The spectral imaging chip includes: a light modulation layer and an image sensing layer sequentially stacked along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing units. The density of the modulation units affects the imaging quality and spectral analysis of the spectral imaging chip. The density of the modulation units is defined by sparsity A, and the formula for calculating sparsity A is as follows: A= Specifically, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit is represented by the geometric center of the modulation unit. The point corresponding to the modulation unit is encoded from 1 to n, d ij This represents the distance between the geometric center of the i-th modulation unit and the geometric center of the j-th modulation unit; The sparsity A is selected within a range greater than 0 and less than or equal to 1 to correspond to the performance requirements of the spectral imaging chip.

35. The spectral imaging device according to claim 34, characterized in that, Each modulation unit in the optical modulation layer includes multiple modulation sub-units, and each modulation sub-unit is a porous structure or a columnar structure.

36. The spectral imaging device according to claim 35, characterized in that, The cross-sectional shape of the apertures in different modulation sub-units of the aperture structure within each modulation unit is not exactly the same; and / or, The structural parameters of different modulation subunits with the aperture structure in each modulation unit are not exactly the same.

37. The spectral imaging device according to claim 35, characterized in that, The different modulation subunits of the columnar structure in each modulation unit have the same structural shape and column height, and all modulation subunits are arranged in a C4 symmetry in each modulation unit.

38. The spectral imaging device according to claim 35, characterized in that, The modulation subunit of the columnar structure is integrally formed or is composed of multiple layers of modulation columns stacked together.

39. A fingerprint liveness detection device, characterized in that, include: The system comprises a display screen, a light source module, a spectral imaging chip, a signal processing circuit module, and a recognition module. The display screen has a fingerprint detection area. The light source module illuminates the object to be detected within the fingerprint detection area, and the target light beam, after being reflected by the object, is incident on the spectral imaging chip. The spectral imaging chip and the signal processing circuit module are two independent parts, or the signal processing circuit module is integrated within the spectral imaging chip. The spectral imaging chip is used to determine the spectral information of each pixel corresponding to each modulation unit in the optical modulation layer of the spectral imaging chip after the target beam illuminates it, and the light intensity information of each pixel corresponding to each non-modulation unit in the optical modulation layer; the signal processing circuit module is used to determine the spectral information of the object to be detected based on the spectral information, and to determine the image information of the object to be detected based on the light intensity information; the recognition module is used to identify whether the object to be detected is a live fingerprint of the target user based on the spectral information and the image information of the object to be detected. The density of the modulation units affects the imaging quality and spectral analysis of the spectral imaging chip. The density of the modulation units is defined by sparsity A, and the formula for calculating sparsity A is as follows: A= Specifically, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit is represented by the geometric center of the modulation unit. The point corresponding to the modulation unit is encoded from 1 to n, d ij This represents the distance between the geometric center of the i-th modulation unit and the geometric center of the j-th modulation unit; The sparsity A is selected within a range greater than 0 and less than or equal to 1 to correspond to the performance requirements of the spectral imaging chip.

40. The fingerprint liveness detection device according to claim 39, characterized in that, The spectral imaging chip includes: a light modulation layer and an image sensing layer stacked sequentially along the thickness direction; the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction; the signal processing circuit module is electrically connected to the sensing unit.

41. The fingerprint liveness recognition device according to claim 39, characterized in that, The signal processing circuit module is specifically used for: Based on the light intensity information of the pixels corresponding to multiple non-modulation units around each modulation unit after the target beam is irradiated, the fitted light intensity information of the pixels corresponding to each modulation unit is determined. Based on the fitted light intensity information of the pixel corresponding to each modulation unit and the light intensity information of the pixel corresponding to each non-modulation unit, the image information of the object to be detected is determined.

42. The fingerprint liveness recognition device according to claim 41, characterized in that, The signal processing circuit module is specifically used for: Based on the smoothing filtering method, the light intensity information of the pixel corresponding to at least one non-modulation unit around any modulation unit is filtered to obtain the fitted light intensity information of the pixel corresponding to the modulation unit.

43. The fingerprint liveness recognition device according to claim 39, characterized in that, The signal processing circuit module is also specifically used for: The initial image obtained by the spectral imaging chip after all non-modulation units are illuminated by the target beam is input into the fitting model to obtain the image information of the object to be detected output by the fitting model; The fitting model is constructed based on an adversarial neural network and is trained on empty sample images containing blank pixels and complete sample images corresponding to the empty sample images that do not contain blank pixels.

44. The fingerprint liveness detection device according to claim 40, characterized in that, Each modulation unit in the optical modulation layer includes multiple modulation sub-units, and each modulation sub-unit is a porous structure or a columnar structure.

45. The fingerprint liveness detection device according to claim 44, characterized in that, The cross-sectional shape of the apertures in different modulation sub-units of the aperture structure within each modulation unit is not exactly the same; and / or, The structural parameters of different modulation subunits with the aperture structure in each modulation unit are not exactly the same.

46. ​​The fingerprint liveness detection device according to claim 44, characterized in that, The different modulation subunits of the columnar structure in each modulation unit have the same structural shape and column height, and all modulation subunits are arranged in a C4 symmetry in each modulation unit.

47. The fingerprint liveness detection device according to claim 44, characterized in that, The modulation subunit of the columnar structure is integrally formed or is composed of multiple layers of modulation columns stacked together.

48. The fingerprint liveness detection device according to any one of claims 39-47, characterized in that, Also includes: Lens group; The light source module is disposed between the display screen and the spectral imaging chip, and the lens group is disposed between the light source module and the spectral imaging chip; or, The light source module is located below the non-fingerprint detection area of ​​the display screen, and the lens group and the spectral imaging chip are sequentially located below the fingerprint detection area; The lens group is used to collimate the target beam onto the spectral imaging chip so that the target beam is imaged on the spectral imaging chip.

49. A fingerprint module, characterized in that, include: A spectral imaging chip and a circuit board, wherein the spectral imaging chip is electrically connected to the circuit board; The spectral imaging chip comprises a light modulation layer, an image sensing layer, and a signal processing circuit module stacked sequentially along the thickness direction; wherein the light modulation layer has at least one modulation unit and at least one non-modulation unit distributed along its surface; the image sensing layer has multiple sensing units distributed along its surface, and each modulation unit and each non-modulation unit corresponds to at least one sensing unit along the thickness direction. The density of the modulation units affects the imaging quality and spectral analysis of the spectral imaging chip. The density of the modulation units is defined by sparsity A, and the formula for calculating sparsity A is as follows: A= Specifically, the side length of the spectral imaging chip is normalized, and the corresponding modulation unit is represented by the geometric center of the modulation unit. The point corresponding to the modulation unit is encoded from 1 to n, d ij This represents the distance between the geometric center of the i-th modulation unit and the geometric center of the j-th modulation unit; The sparsity A is selected within a range greater than 0 and less than or equal to 1 to correspond to the performance requirements of the spectral imaging chip.

50. The fingerprint module according to claim 49, characterized in that, Also includes: A reinforcing plate is attached to the circuit board.

51. The fingerprint module according to claim 49, characterized in that, Also includes: The encapsulation portion is formed on the upper surface of the circuit board.

52. The fingerprint module according to claim 51, characterized in that, Also includes: A light-shielding portion is disposed on the upper surface of the encapsulation portion.

Images