Spectral analysis devices and terminal equipment and imaging methods

Abstract

This invention provides a spectral analysis device, a terminal device with the spectral analysis device, and a method for operating the device. The spectral analysis device includes a spectral chip, an optical system, and at least one data processing unit. The optical system is located in the optical path of the spectral chip, and the spectral chip is electrically connected to the at least one data processing unit. The spectral chip has multiple transmission spectrum matrices, and the optical system has a variable focal length. By adjusting the focal length of the optical system, the optical system modulates the incident light incident on the spectral chip, and the data processing unit calculates the spectral information corresponding to the incident light based on a specific transmission spectrum matrix corresponding to the spectral chip.

Description

Technical Field

[0001] This invention relates to the field of spectral imaging technology, and more particularly to a spectral analysis device, a terminal device, and an imaging method. Background Technology

[0002] With the development of spectral technology, spectral analysis has been widely applied in daily life and industry; for example, it is used for non-invasive examinations in medical and cosmetic fields, food testing of fruits and vegetables, and monitoring of water quality. Its working principle is that light interacts with matter through processes such as absorption, scattering, fluorescence, and Raman spectroscopy, producing specific spectra. Each substance's spectrum is unique. Spectroscopic analysis devices can directly detect the spectral information of substances, obtaining the presence and composition of the target material, making them one of the important testing instruments in materials characterization, chemical analysis, and other fields. Therefore, spectral information can be considered the "fingerprint" of everything.

[0003] However, specific spectra often require matching spectral analysis devices for detection and identification to achieve higher efficiency and accuracy. This leads to the need for spectral analysis devices with varying performance for different scenarios and analytes. Existing spectral analysis devices require a specific distance from the analyte to achieve good spectral detection results, but in practical use, they struggle to adapt to different types of objects, resulting in insufficient detection and identification performance. Furthermore, images captured by spectral imaging devices based on spectral reconstruction algorithms suffer from insufficient spatial resolution, leading to unclear images.

[0004] Therefore, there is an urgent need to develop a spectral analysis device that can be applied to the detection of different scenarios and objects simultaneously. Summary of the Invention

[0005] A key advantage of this invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral analysis device obtains spectral information based on spectral calculations, which helps to improve the resolution of the spectral information.

[0006] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral analysis device includes a filter structure, and the spatial resolution of the spectral analysis device is improved by moving the filter structure.

[0007] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral analysis device includes a spectral chip, and the spatial resolution of the spectral analysis device is improved by moving the spectral chip of the spectral analysis device.

[0008] Another advantage of the present invention is that it provides a spectral analysis device and terminal equipment as well as an imaging method, wherein the spectral analysis device includes an optical system having a variable focal length, and the spatial resolution of the spectral analysis device is improved by adjusting the focal length of the optical system.

[0009] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral chip of the spectral analysis device is drivably moved at multiple shooting positions and captures spectral information at each shooting position. By synthesizing the spectral information corresponding to each position, it is beneficial to improve the spatial resolution of the two-dimensional image.

[0010] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the filter structure of the spectral chip can be driven and moved relative to the image sensor, such that each physical pixel of the image sensor can correspond to a different structural unit of the filter structure, thereby facilitating the spectral analysis device to capture different spectral images.

[0011] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral chip can be moved as a whole so that the spectral analysis device can capture different spectral images, which is beneficial to improving spatial resolution.

[0012] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral chip is provided with different transmission spectrum matrices, and the transmission spectrum matrix of the spectral chip corresponds to the focal length of the optical system. When the focal length of the optical system changes, imaging is performed with a specific transmission spectrum matrix, thereby improving the spectral resolution of the spectral analysis device.

[0013] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the spectral analysis device is adjusted according to the characteristics of the object to be measured, so that the principal angle and / or the receiving cone angle of the incident light containing the information of the object to be measured to the structural pixels of the spectral chip changes, and the transmission spectrum matrix of the spectral chip changes, making it more suitable for the characteristics of the object to be measured, thereby improving the recognition and detection accuracy.

[0014] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method. The spectral analysis device includes a spectral chip and an optical system disposed in the optical path of the spectral chip. The optical system is focusable. By adjusting the focal length of the optical system, the principal angle and / or the cone angle of the incident light reaching the surface of the filter structure changes, thereby changing the transmission spectrum matrix corresponding to the filter structure. Based on the characteristics of the object under test, a suitable transmission spectrum matrix is ​​obtained by selecting the corresponding focal length for identification and detection, thereby improving the accuracy of identification and detection.

[0015] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the optical system is implemented as a zoom lens group, and the zoom of the optical system is achieved by moving the lenses of the zoom lens group, thereby selecting the corresponding focal length for identification and detection according to the characteristics of the object to be measured, so as to improve the identification and detection accuracy.

[0016] Another advantage of the present invention is that it provides a spectral analysis device and terminal equipment as well as an imaging method, wherein the optical system includes a liquid lens, the focal length of the optical system is adjusted by the liquid lens, and the height of the spectral analysis device can be further reduced.

[0017] Another advantage of the present invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the optical system is implemented as a periscope lens, which can effectively reduce the height of the spectral analysis device in the optical axis direction.

[0018] Another advantage of this invention is that it provides a spectral analysis device, a terminal device, and an imaging method, wherein the change in the focal length of the optical system causes a change in the principal angle and / or the cone angle of the incident light reaching the surface of the filter structure. Due to the change in the principal angle and / or the cone angle of the incident light, the transmission spectrum curve corresponding to the structural unit of the spectral chip changes. Therefore, the spectral analysis device of this invention can select the corresponding focal length (or the corresponding transmission spectrum curve) according to the characteristics of the object to be measured for identification and detection, thereby improving the accuracy of identification and detection.

[0019] According to one aspect of the present invention, a spectral analysis apparatus of the present invention, capable of achieving the aforementioned and other objects and advantages, comprises:

[0020] A spectral chip;

[0021] An optical system, wherein the optical system is located in the optical path of the spectral chip, and incident light reaches the spectral chip via the optical system; and

[0022] A driving mechanism is provided, wherein the driving mechanism is driven by the spectral chip, the spectral chip is driven by the driving mechanism and moves between multiple shooting positions, wherein the spectral data captured by the spectral chip at each position is integrated into a spectral image.

[0023] According to at least one embodiment of the present invention, the spectral chip includes an image sensor and at least one filter structure disposed on the photosensitive side of the image sensor, the moving direction of the spectral chip is parallel to the plane where the two-dimensional physical pixels of the image sensor are located, and the spectral data obtained at each of the shooting positions are synthesized by the data processing unit to form the spectral image.

[0024] According to at least one embodiment of the present invention, the movement of the spectral chip is in units of the size of the physical pixels of the image sensor in the X / Y axis direction.

[0025] According to at least one embodiment of the present invention, the filter structure is connected to the driving mechanism, the filter structure includes a plurality of structural units, wherein the structural unit of the filter structure corresponds to at least one physical pixel of the image sensor, and the driving mechanism drives the filter structure to move relative to the image sensor.

[0026] According to at least one embodiment of the present invention, the moving direction of the filter structure is parallel to the physical pixel surface of the image sensor.

[0027] According to at least one embodiment of the present invention, the single movement distance of the filter structure is the periodic distance of the structural unit in the X / Y axis direction; or the filter structure moves in units of the length dimension of the physical pixel in the X / Y axis direction; or the filter structure moves in units of an integer multiple of the periodic distance of the structural unit.

[0028] According to at least one embodiment of the present invention, there is a gap between the filter structure and the image sensor, the gap being less than or equal to 20 μm and greater than or equal to 0.5 μm.

[0029] According to at least one embodiment of the present invention, the spectral chip includes an image sensor and at least one filter structure disposed on the photosensitive side of the image sensor. The image sensor further includes a plurality of structural units, and the structural units of the image sensor correspond to at least one physical pixel of the image sensor. The driving mechanism is tractable to the filter structure of the spectral chip. Through the relative movement of the filter structure and the image sensor, the physical pixels of the image sensor correspond to different structural units and generate corresponding spectral signals. The data processing unit obtains the spectral image based on the spectral signals.

[0030] According to at least one embodiment of the present invention, one structural unit of the filter structure and a plurality of corresponding physical pixels constitute a structural pixel, wherein the moving distance of the filter structure is in units of the period length of the structural unit corresponding to the structural pixel.

[0031] According to at least one embodiment of the present invention, the filter structure is driven to move in units of the side length of the physical pixel of the image sensor in the X / Y direction.

[0032] According to another aspect of the present invention, the present invention further provides a spectral analysis apparatus, comprising:

[0033] A spectral chip; the spectral chip has multiple transmission spectrum matrices; and

[0034] An optical system, wherein the optical system is located in the optical path of the spectral chip;

[0035] The optical system has a variable focal length, and the variable focal length of the optical system corresponds to the plurality of transmission spectrum matrices of the spectral chip. By adjusting the focal length of the optical system, a specific transmission spectrum matrix is ​​configured for the spectral chip. The spectral chip obtains at least one spectral data based on each focal length position of the optical system with a specific transmission spectrum matrix, and integrates the spectral data to obtain a spectral image.

[0036] According to at least one embodiment of the present invention, the optical system includes at least one lens assembly and at least one moving mechanism, wherein the at least one lens assembly is transversely connected to the at least one moving mechanism, and the at least one lens assembly is driven to move by the at least one moving mechanism to adjust the focal length of the optical system.

[0037] According to at least one embodiment of the present invention, the optical system further includes at least one deflector, wherein the deflector is disposed in the optical axis direction of the at least one lens assembly, and the deflector deflects the transmission direction of light incident on or exiting the at least one lens assembly.

[0038] According to at least one embodiment of the present invention, the lens assembly further includes a first lens group, a second lens group, and a third lens group, wherein the first lens group, the second lens group, and the third lens group are arranged along the same optical axis direction, the second lens group is located between the first lens group and the third lens group, and wherein the second lens group is connected to the moving mechanism and is driven to move by the moving mechanism.

[0039] According to at least one embodiment of the present invention, the second lens group further includes at least one zoom lens and at least one compensation lens, the at least one zoom lens and the at least one compensation lens being tractably connected to the moving mechanism, and zooming is achieved by moving the zoom lens and the compensation lens.

[0040] According to at least one embodiment of the present invention, the turning member further includes a first turning member and a second turning member, wherein the first turning member is located at the front end of the first lens group, and the second turning member is located between the second lens group and the third lens group.

[0041] According to at least one embodiment of the present invention, the optical system includes at least one liquid lens assembly and at least one lens assembly, the liquid lens assembly and the lens assembly being arranged one after the other along the same optical axis, and the liquid lens assembly being capable of changing its curvature.

[0042] According to at least one embodiment of the present invention, the liquid lens assembly may include at least one deformable lens body, a flexible transparent cover component, and an actuator, wherein the flexible transparent cover component is attached to the surface of the at least one deformable lens body, and the actuator is located on the upper surface of the flexible transparent cover component, thereby driving the flexible transparent cover component to move to change the shape of the deformable lens body.

[0043] According to at least one embodiment of the present invention, the device further includes a circuit board and at least one heat sink, wherein the spectral chip is electrically connected to the circuit board, and the heat sink may be attached to the circuit board or to the spectral chip.

[0044] According to at least one embodiment of the present invention, a bracket is further included, the bracket being disposed on the circuit board, the optical system being disposed on the bracket, the bracket having a light-transmitting hole corresponding to the photosensitive area of ​​the spectral chip.

[0045] According to at least one embodiment of the present invention, the spectral chip records the principal angle corresponding to each of the transmission spectrum matrices and / or the zoom position of the optical system corresponding to each of the transmission spectrum matrices.

[0046] According to at least one embodiment of the present invention, the first lens group includes a first lens and a second lens, the second lens group includes the third lens and the fourth lens, and the third lens group includes the fifth lens and the sixth lens. Along the optical axis of the optical system from the object side to the image side, the first lens, the second lens, the third lens, the fourth lens, the fifth lens, and the sixth lens are arranged sequentially, and the optical system satisfies the following relationships: -3 < f2 / f1 < 0; 0 < f3 / f1 < 4; 0 < f4 / f1 < 4; -7 < f5 / f1 < -2; -3 < f6 / f1 < 0. f1 is the focal length of the first lens, f2 is the focal length of the second lens, f3 is the focal length of the third lens, f4 is the focal length of the fourth lens, f5 is the focal length of the fifth lens, and f6 is the focal length of the sixth lens.

[0047] According to at least one embodiment of the present invention, the spectral chip further includes an image sensor and at least one filter structure disposed on the photosensitive side of the image sensor, wherein the filter structure is located above the image sensor and the filter structure is a broadband filter structure in the frequency domain or wavelength domain.

[0048] According to at least one embodiment of the present invention, the filter structure of the spectral chip is selected from a combination of metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS, FP etalon, cavity layer, waveguide layer and diffraction element.

[0049] According to at least one embodiment of the present invention, the data processing unit is selected from a combination of processing units consisting of MCU, CPU, GPU, FPGA, NPU and ASIC.

[0050] According to another aspect of the present invention, the present invention further provides a terminal device, comprising:

[0051] A terminal device host; and

[0052] As described in any of the above-described spectral analysis devices, wherein the spectral analysis device is electrically connected to the terminal device host, and the terminal device host sends control commands to the spectral analysis device to adjust the focal length of the spectral analysis device.

[0053] According to at least one embodiment of the present invention, a selection module is further included, wherein the selection module selects the test object and generates the control command.

[0054] According to at least one embodiment of the present invention, a judgment module is further included, wherein the judgment module identifies and judges the spectral characteristics of the object to be tested, and further generates the control command based on the spectral characteristics of the object to be tested.

[0055] According to at least one embodiment of the present invention, an imaging module is further included, wherein the imaging module is electrically connected to the host of the terminal device, thereby acquiring image information of the object under test to analyze the spectral characteristics of the object under test.

[0056] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, comprising:

[0057] (301) Drive a spectral chip to move and capture spectral information at corresponding positions; and

[0058] (302) The spectral information captured at each location is synthesized into a spectral image.

[0059] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, comprising:

[0060] (201) Driving a filter structure of a spectral chip to move, such that the filter structure moves at least one position relative to the image sensor, and capturing spectral information at corresponding positions; and

[0061] (202) The spectral information captured at each location is synthesized into a spectral image.

[0062] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, comprising:

[0063] (401) Drive a filter structure of a spectral chip to move, such that each physical pixel of an image sensor of the spectral chip corresponds to multiple different structural units of the green light structure, and each physical pixel of the spectral chip obtains a set of signals based on the corresponding structural units; and

[0064] (402) Based on the obtained signal, the spectral curves corresponding to each pixel are obtained, and the spectral curves are arrayed to obtain a spectral image.

[0065] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, comprising:

[0066] (501) The incident light is modulated by adjusting the focal length of an optical system, and multiple spectral data of the incident light are obtained based on the transmission spectrum matrices of the spectral chip corresponding to different focal lengths of the optical system; and

[0067] (502) Integrate the spectral data and recover the spectral curve from the integrated spectral data.

[0068] According to at least one embodiment of the present invention, the imaging method further includes the following steps: matching a corresponding transmission spectrum matrix to the spectral chip based on the focal length of the optical system, and calculating the spectral information of the incident light based on the transmission spectrum matrix.

[0069] The further objects and advantages of the invention will become fully apparent from the following description and accompanying drawings.

[0070] These and other objects, features and advantages of the present invention will become fully apparent from the following detailed description and accompanying drawings. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the frame of a spectral analysis device according to a preferred embodiment of the present invention.

[0072] Figure 2A and Figure 2B This is a schematic diagram of an optional embodiment of a spectral chip of the spectral analysis device according to the preferred embodiment of the present invention.

[0073] Figure 3A and Figure 3B This is a schematic diagram of another optional embodiment of a spectral chip of the spectral analysis device according to the preferred embodiment of the present invention described above.

[0074] Figure 4A and Figure 4B This is a schematic diagram of another optional embodiment of a spectral chip of the spectral analysis device according to the preferred embodiment of the present invention described above.

[0075] Figure 5A and Figure 5B This is a schematic diagram of the effect of the transmission spectrum curve of the spectral analysis device according to the above-described preferred embodiment of the present invention.

[0076] Figure 6 This is a schematic diagram of another optional embodiment of the spectral chip of the spectral analysis device according to the preferred embodiment of the present invention described above.

[0077] Figure 7 This is a schematic diagram of another optional embodiment of a spectral chip of the spectral analysis device according to the preferred embodiment of the present invention described above.

[0078] Figure 8 This is a schematic diagram of the pixel structure of the spectral chip in the spectral analysis device according to the preferred embodiment of the present invention.

[0079] Figure 9 This is a schematic diagram of the system framework of a spectral analysis device according to another preferred embodiment of the present invention.

[0080] Figure 10A and Figure 10B This is a schematic diagram of spectral imaging of the spectral analysis device according to any of the preferred embodiments of the present invention.

[0081] Figure 11 This is a schematic diagram of the spectral analysis device according to any of the preferred embodiments of the present invention, wherein the resolution is improved by moving its filter structure.

[0082] Figure 12 This is a schematic diagram of the spectral analysis device according to any of the preferred embodiments of the present invention, wherein the resolution is improved by moving its spectral chip.

[0083] Figure 13 This is a schematic diagram of the spectral analysis device according to any of the preferred embodiments of the present invention, wherein the resolution is improved by moving its filter structure.

[0084] Figure 14 This is a schematic diagram of the structure of an optical system of the spectral analysis apparatus according to any of the preferred embodiments of the present invention, wherein the optical system is a vertical lens.

[0085] Figure 15A and Figure 15B This is a schematic diagram of the operation of the optical system of the spectral analysis apparatus according to any of the preferred embodiments of the present invention.

[0086] Figure 16 This is a schematic diagram of another alternative embodiment of an optical system of the spectral analysis apparatus according to any of the preferred embodiments described above, wherein the optical system is a liquid lens.

[0087] Figure 17 This is a schematic diagram of the operation of the optical system of the spectral analysis apparatus according to any of the preferred embodiments of the present invention.

[0088] Figure 18A and Figure 18B This is a schematic diagram of another alternative embodiment of an optical system of the spectral analysis apparatus according to any of the preferred embodiments described above, wherein the optical system is a periscope lens.

[0089] Figure 19 This is a schematic diagram of the operation of the optical system of the spectral analysis apparatus according to any of the preferred embodiments of the present invention.

[0090] Figure 20A and Figure 20BThis is a schematic diagram of a spectral analysis device according to another preferred embodiment of the present invention. Detailed Implementation

[0091] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0092] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0093] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0094] Referring to the accompanying drawings of this invention Figure 1As shown, a spectral analysis device according to a preferred embodiment of the present invention will be described below. The spectral analysis device of this embodiment is a computational spectral analysis device, which approximates or even reconstructs the spectrum of incident light through calculation. The spectral analysis device includes a spectral chip 10, an optical system 20 located in the photosensitive path of the spectral chip 10, and at least one data processing unit 30 electrically connected to the spectral chip 10. In this preferred embodiment of the present invention, the optical system 20 of the spectral analysis device is optional and can be implemented as a lens assembly, a homogenizing assembly, or other optical systems. The spectral chip 10 further includes an image sensor 11 and at least one filter structure 12 disposed on the photosensitive side of the image sensor 11, wherein the filter structure 12 is located above the image sensor 11, and the filter structure 12 is a broadband filter structure in the frequency domain or wavelength domain. The optical system 20 is located at the front end of the spectral chip 10 in the photosensitive direction. Light emitted or reflected by the object under test, carrying information about the object, is guided to the spectral chip 10 via the optical system 20. The spectral chip 10 converts the incident light signal of the object under test into an electrical signal suitable for processing by the data processing unit 30, and transmits it to the data processing unit 30. The signal processing unit 30 is equipped with an algorithm processing system that can process the differential response based on an algorithm to reconstruct the original spectrum.

[0095] It is worth noting that the transmittance of the filter structure 12 is not entirely the same for light of different wavelengths. The filter structure 12 can be implemented as a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in the embodiments of this application, the filter structure 12 can be the light modulation layer in Chinese Patent CN201921223201.2.

[0096] The image sensor 11 of the spectral chip 10 can be a CMOS image sensor (CIS), CCD, array photodetector, etc. In this preferred embodiment of the invention, the optional data processing unit 30 in the spectral analysis device can be a processing unit such as an MCU, CPU, GPU, FPGA, NPU, ASIC, etc., which can export the data generated by the image sensor 11 to an external source for processing. It is worth noting that the data processing unit 30 can be integrated into the spectral chip 10; alternatively, it can be a separate processing unit, such as a computer, microcontroller, or cloud computing platform.

[0097] It is worth mentioning that after the image sensor 11 of the spectral analysis device measures the light intensity information, it transmits it to the data processing unit 30 for processing, such as spectral reconstruction and spectral imaging. The process is described in detail below:

[0098] The intensity signals of the incident light of the test object at different wavelengths λ are denoted as x(λ), and the transmission spectrum curve of the filter structure 12 is denoted as T(λ). The filter structure 12 has m groups of structural units 121, and the transmission spectrum of each group of structural units 121 is different. The image sensor 11 has multiple physical pixels, wherein the physical pixels of the image sensor 11 correspond to the structural units 121 of the filter structure 12. The structural units 121 of the filter structure 12 can be denoted as Ti(λ) (i = 1, 2, 3, ..., m). Each group of structural units 121 of the filter structure 12 corresponds to at least one physical pixel of the image sensor 11, that is, there is a corresponding physical pixel below each group of structural units 121 of the filter structure 12, and the image sensor 11 detects the light intensity bi modulated by the filter structure 12.

[0099] In this preferred embodiment of the present invention, one physical pixel of the image sensor 11 corresponds to a group of structural units 121, but this is not limited to this. In other embodiments of the present invention, multiple physical pixels form a group corresponding to a group of structural units 121, wherein each of the structural units 121 of the filter structure 12 and at least one group of physical pixels of the image sensor 11 constitutes a structural pixel 102. Therefore, in the computational spectral analysis apparatus according to the embodiments of the present application, at least two structural pixels 102 constitute a spectral pixel 1020. It is understood that in this preferred embodiment of the present invention, multiple groups of structural units 121 of the filter structure 12 and the corresponding image sensor 11 constitute the spectral pixel 1020.

[0100] It is worth mentioning that, in this preferred embodiment of the present invention, the number of effective transmission spectra (the transmission spectrum used for spectral recovery is called the effective transmission spectrum) Ti(λ) of the filter structure 12 may not be the same as the number of structural units 121. The transmission spectrum of the filter structure 12 is set, tested, or calculated according to certain rules based on the identification or recovery requirements (for example, the transmission spectrum obtained by testing each structural unit 121 is the effective transmission spectrum). Therefore, the number of effective transmission spectra of the filter structure 12 may be less than the number of structural units 121, or even more than the number of structural units 121 (considering that the structural units 121 can be reconstructed according to requirements). Therefore, it is understood that, in this preferred embodiment of the present invention, a certain transmission spectrum curve is not necessarily determined by a group of structural units 121, but may be determined by multiple structural units 121.

[0101] The relationship between the spectral distribution of the incident light on the object under test and the measurement value of the image sensor 11 can be expressed by the following formula:

[0102] b i =∫x(λ)*T i (λ)*R(λ)dλ

[0103] After discretization, we get

[0104] b i =Σ(x(λ)*T i (λ)*R(λ))

[0105] Where R(λ) is the response of the image sensor, denoted as:

[0106] A i (λ)=T i (λ)*R(λ)

[0107] The above equation can then be extended into matrix form:

[0108]

[0109] Among them, b i (i = 1, 2, 3, ..., m) represents the response of the image sensor 11 after the light to be measured passes through the filter structure 12, corresponding to the light intensity measurement values ​​of the image sensor 11 corresponding to each of the m structural units. When one physical pixel corresponds to one structural unit 121, it can be understood that the light intensity measurement values ​​corresponding to the m physical pixels can form a vector of length m. A is the system's response to light of different wavelengths, determined by the transmittance of the filter structure 12 and the quantum efficiency of the image sensor 11, and can be called the transmission spectrum matrix. A is a matrix, where each row vector corresponds to a set of structural units 121 responding to incident light of different wavelengths. As an example, in this invention, the incident light is sampled discretely and uniformly, with a total of n sampling points. The number of columns in A is the same as the number of sampling points of the incident light, where x(λ) is the light intensity of the incident light at different wavelengths λ, which is the incident light spectrum to be measured.

[0110] In other optional embodiments of the present invention, the filter structure 12 can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc., which directly form the filter structure or material (nanowires, quantum dots, etc.) in the photosensitive area of ​​the image sensor 11. In other words, the filter structure 12 is integrally formed on the photosensitive side surface of the image sensor 11. With the filter structure formed on the upper surface of the image sensor 11, the transmission spectrum curve and the response of the image sensor are integrated, that is, it can be understood that the response of the image sensor and the transmission spectrum curve are the same curve. In this case, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following formula:

[0111] bi=Σ(x(λ)*Ri(λ))

[0112] In this embodiment, the transmission spectrum Ai(λ) = Ri(λ)

[0113] It is understood that in other alternative embodiments of the present invention, at least one other filter structure 12b for modulating incident light is provided on the image sensor having a filter structure 12a, thus forming a spectral chip 10 with a dual filter structure. It can be understood that the image sensor 11 in the first embodiment, which could be a CMOS image sensor (CIS), CCD, array photodetector, etc., can be replaced with an image sensor integrating a filter structure in the second embodiment.

[0114] The relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor 11 can be expressed by the following formula:

[0115] bi=∫x(λ)*Ti(λ)*Ri(λ)dλ

[0116] After discretization, we get

[0117] bi=Σ(x(λ)*Ti(λ)*Ri(λ))

[0118] In this embodiment, Ai(λ) = Ti(λ) * Ri(λ)

[0119] It should be noted that the spectral chip 10 of the spectral analysis device is sensitive to the principal angle and the receiving cone angle of the incident light signal. Changes in the principal angle and / or the receiving cone angle of the incident light signal of the analyte will cause changes in the transmission spectrum matrix of the spectral chip, thereby affecting the accuracy of spectral recovery.

[0120] The principal angle at any specific location of the spectral chip 10 represents the angle between the principal ray guided to the spectral chip 10 and the normal. The principal ray represents the line connecting the point where the light signal is emitted from the target and the point where it arrives at the corresponding structural pixel 102 of the spectral chip 10. The normal represents a line perpendicular to the photosensitive surface of the spectral chip 10. Those skilled in the art will understand that the principal angles of different structural pixels 102 can vary considerably, but the light incident on the same structural pixel 102 needs to maintain a small angular difference. It should also be noted that the spectral chip 10 is quite sensitive to the light cone angle at which the incident light signal arrives at various locations on the spectral chip 10. In practical applications, a large change in the light cone angle of the incident light signal will significantly affect the transmission spectrum matrix corresponding to the spectral chip.

[0121] Specifically, when the incident light signal of the analyte reaches a certain structural pixel 102 of the spectral chip, the incident angle of the light signal to the structural pixel 102 of the spectral chip 10 (for the structural pixel 102, this incident angle can also be defined as the light-receiving cone angle of the structural unit 121) is a factor. If the incident angle changes, the parameter value at the corresponding position in the transmission spectrum matrix A will also change accordingly, thus affecting the accuracy of spectral recovery. Furthermore, when the light-receiving cone angle of the incident light signal is large, it is equivalent to the superposition of transmission spectra of collimated light incident at multiple angles. At this time, the randomness and complexity of the spectrum transmitted by the filter structure 12 decrease, and the correlation between different light modulation units increases, resulting in a decrease in the spectral recovery effect; conversely, the smaller the light-receiving cone angle, the better the spectral recovery effect.

[0122] In other words, due to the angular sensitivity of the filter structure 12, the transmission spectrum matrix A is affected by the principal angle and / or the receiving cone angle of the incident light signal during the calculation and reconstruction process. In actual use environments, the spatial distribution of the incident light from the analyte and the angular distribution of the light rays are uncertain. Therefore, the principal angle and the receiving cone angle of different structural units 121 incident on the spectral chip 10 are also uncertain, resulting in a large error in the spectral measurement. In short, different types of analytes and different incident light from the analytes may lead to different principal angles or receiving cone angles of the light signal, which may affect the accuracy of the calculation and reconstruction by the spectral analysis device.

[0123] It is worth noting that different objects to be tested have different properties and exhibit different characteristics. Therefore, a corresponding transmission spectrum matrix A is needed to modulate the incident light containing information about the object to be tested, which can improve the accuracy of object identification and detection. Therefore, in this preferred embodiment of the present invention, the spectral analysis device causes a change in the transmission spectrum matrix A based on the change in the principal angle and / or the receiving cone angle. Furthermore, the spectral information of the object to be tested under different transmission spectrum matrices can also be obtained and then used to reconstruct the spectral curve, thereby improving the spectral resolution.

[0124] For ease of explanation, the linear correlation between each row of the transmission spectrum matrix A is defined as the correlation coefficient. For example, the commonly used Pearson correlation coefficient is used. A good match refers to a low correlation coefficient between each row of the transmission spectrum matrix A in the band corresponding to the spectral characteristics of the object being identified or detected. In this invention, a low Pearson correlation coefficient means a correlation coefficient less than or equal to 0.9, preferably less than or equal to 0.7, and even more preferably less than or equal to 0.4.

[0125] The optical system 20 can be adjusted according to the characteristics of the object under test, so that the principal angle and / or the receiving cone angle of the incident light containing the information of the object under test reaching the structural pixel 102 of the spectral chip 10 changes, and the transmission spectrum matrix A of the spectral chip 10 changes, becoming more suitable for the incident light of the object under test, thereby improving the recognition and detection accuracy. Preferably, in this preferred embodiment of the present invention, the incident angle of the incident light of the optical system 20 is adjusted by adjusting the focal length of the optical system 20, thereby adjusting the principal angle and / or the receiving cone angle of the incident light of the object under test reaching the structural pixel 102 of the spectral chip 10, and thus changing the transmission spectrum matrix A, so as to be suitable for the spectral analysis device to reconstruct the relevant spectrum of the object under test or to detect and identify the object under test.

[0126] It should be noted that, generally speaking, the larger the zoom ratio of the optical system 20, the larger the range of the principal angle variation. Preferably, the zoom ratio of the optical system 20 is greater than or equal to 2, for example, 3 or 4 times. More preferably, in this preferred embodiment of the present invention, the optical system 20 has a zoom ratio greater than or equal to 5 times. Since the spectral analysis device of the present invention requires specific angles for the principal angle and the receiving light cone angle, it is necessary to consider the values ​​of the principal angle and the receiving light cone angle at the focal length corresponding to the zoom ratio, that is, to ensure that the values ​​of the principal angle and the receiving light cone angle make the corresponding transmission spectrum matrix A more suitable for the needs of analyte identification, detection, or corresponding spectral recovery.

[0127] The structure of the spectral chip 10 of the spectral analysis device of the above preferred embodiments of the present invention is further illustrated below through Embodiments 1 to 3.

[0128] Example 1

[0129] like Figure 2A and Figure 2B The diagram illustrates the structure of an optional embodiment of a spectral chip 10 in the zoom spectral analysis device of the preferred embodiment of the present invention. The spectral chip 10 includes a filter structure 12 and an image sensor 11. The filter structure 12 is disposed along the photosensitive path of the image sensor 11. The image sensor 11 may be, but is not limited to, a CMOS image sensor (CIS), a CCD, an array photodetector, etc. The filter structure 12 includes at least one light modulation layer 120. The light modulation layer 120 has at least one structural unit 121, which corresponds to at least one physical pixel of the image sensor 11. The structural unit 121 modulates the incident light, which is then received by the corresponding physical pixel.

[0130] In this preferred embodiment of the present invention, the structural unit 121 of the filter structure 12 and at least one physical pixel of the image sensor 11 corresponding to the structural unit 121 constitute a structural pixel 102. Preferably, the structural unit 121 further has at least one modulation aperture 1210, wherein the modulation aperture 1210 of the structural unit 121 is positively opposite to the physical pixel of the image sensor 11. It is worth mentioning that, in this preferred embodiment of the present invention, the structural unit 121 of any structural pixel 102 may have the same or different types of modulation apertures 1210, that is, the structural unit 121 may have multiple modulation apertures, and at least two modulation apertures 1210 have different structures and parameters. Preferably, a structural pixel 102 is composed of only one type of modulation aperture 1210 with the same structure and size. The material of the light modulation layer 120 may be silicon, germanium, germanium-silicon materials, silicon compounds, germanium compounds, metals and III-V group materials, tantalum oxide, and / or titanium dioxide, etc., wherein silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, and silicon carbide. It is worth mentioning that the material of the light modulation layer 120 can be, but is not limited to, low-refractive-index materials such as silicon dioxide and polymers.

[0131] It is worth mentioning that the modulation aperture 1210 of each of the structural units 121 all possess C4 symmetry, meaning that after rotating the modulation aperture 1210 along the axis of symmetry by 90°, 180°, or 270°, the structure of the modulation aperture 1210 coincides with the original structure. Correspondingly, the structure of the modulation aperture 1210 of the structural unit 121 includes circles, crosses, regular polygons, squares, ellipses, etc. This enables the spectral chip 10 to achieve polarization independence, allowing it to measure the spectral information of incident light without being affected by the polarization characteristics of the incident light.

[0132] The light modulation layer 120 can be formed on the upper surface of the image sensor 11 through processes such as bonding, coupling, bonding, and deposition. As an example, a corresponding light modulation layer material is deposited on the upper surface of the image sensor 11, and then etched to form corresponding modulation holes, thereby fabricating the filter structure 12 on the surface of the image sensor 11. Optionally, a dielectric layer material can be deposited on the upper surface of the image sensor 11 first, then the upper surface of the dielectric layer can be planarized to obtain a flat upper surface dielectric layer, then a light modulation layer material can be deposited on the upper surface of the dielectric layer, then a photoresist layer can be coated, and exposure etching can be performed to form the structural units corresponding to the light modulation layer. The photoresist layer can then be removed to obtain the desired spectral chip.

[0133] Those skilled in the art will understand that the spectral chip 10 can also be fabricated to obtain a light modulation layer first, and then the light modulation layer can be combined with the image sensor by coupling and bonding. It should be noted that the upper surface of the image sensor needs to be kept flat in this process. Therefore, preferably, a dielectric layer with a flat surface needs to be formed on the upper surface of the image sensor first.

[0134] Example 2

[0135] like Figure 3A and Figure 3B The diagram illustrates the structure of an optional embodiment of a spectral chip 10A in the zoom spectral analysis device of the preferred embodiment of the present invention. In this embodiment, the spectral chip 10A has a regional chip structure. Specifically, the spectral chip 10A includes a filter structure 12A and an image sensor 11A, with the filter structure 12A disposed along the photosensitive path of the image sensor 11A. The filter structure 12A includes a light modulation layer 120A, wherein the light modulation layer 120A further includes a plurality of modulation regions 122A and at least one non-modulation region 123A for spacing adjacent modulation regions 122A, wherein the modulation regions 122A modulate the incident light, and the modulated incident light is received by the image sensor 11A, and the corresponding spectrum can be recovered by calculation.

[0136] The light modulation layer 120A of the filter structure 12A further includes a plurality of structural units 121A, wherein the structural units 121A of the light modulation layer 120A are located in the modulation region 122A of the light modulation layer 120A, and the structural units 121A of the light modulation layer 120A have corresponding transmission spectrum curves; while the non-modulation region 123A may not have any structure, that is, the incident light is received by the physical pixels of the image sensor in the corresponding region without any processing. Optionally, the non-modulation region 123A also has adjustment functions such as filtering, deflection, convergence, refraction, diffraction, diffusion and / or collimation of the incident light, and it can be implemented as a structure with specific adjustment functions such as a filter, concave lens, convex lens, optical diffraction.

[0137] Preferably, in this preferred embodiment of the present invention, the modulation region 122A of the light modulation layer 120A is implemented as the structural unit 121A composed of micro-nano structures such as modulation apertures, while the non-modulation region 123A is composed of common imaging pixels such as RGB pixels or black and white pixels.

[0138] It is worth mentioning that, in this preferred embodiment of the present invention, the spectral information of the object under test is determined by the spectral information of the pixel corresponding to each structural unit 121A of the modulation region 122A of the light modulation layer 120A of the spectral chip 10A, based on the spectral information of the target beam from the object under test illuminating the pixel corresponding to each non-modulation region 123A in the light modulation layer 120A; and the image information of the object to be imaged is determined based on the light intensity information of the pixel corresponding to each non-modulation region 123A in the light modulation layer 120A illuminating the target beam. Therefore, compared with the image sensor of the prior art, the spectral chip 10A of the spectral analysis device of the present invention can obtain spectral information without affecting the spatial resolution and imaging quality of the image, making it easier to obtain more comprehensive information about the object to be imaged. Since the spectral information of the object under test can be used to uniquely identify the object to be imaged, qualitative or quantitative analysis of the object to be imaged can be achieved through the spectral information of the object to be imaged. This allows the spectral chip to be applied to fields such as fruit freshness, air pollution levels, AI scene recognition, and liveness detection, increasing the application scenarios of the spectral imaging chip and providing a theoretical basis for the widespread application of the spectral imaging chip.

[0139] Example 3

[0140] like Figures 4A to 6The diagram illustrates the structure of an optional embodiment of a spectral chip 10B in the zoom spectral analysis device of the preferred embodiment of the present invention. In this embodiment, the spectral chip 10B has a multilayer structure. In practical industry, due to limitations in processing technology, fabricating structural units with complex structures and forming structural units with high processing precision present a technical contradiction. Specifically, when the structural unit used to modulate incident light is a modulation aperture (i.e., when the structural unit is a modulation aperture, such as a through-hole or blind aperture), ideally, the more complex the modulation aperture, the better the modulation effect on the incident light. However, in practical industry, it is difficult to obtain complex modulation apertures using existing production processes. In particular, the deeper the modulation aperture, the more difficult it is to guarantee its precision. For example, in etching, the precision is higher at shallower depths, but as the aperture is processed deeper, the etching solution concentration, etching time, and speed become more difficult to control, which may lead to lower etching precision.

[0141] Maintaining manufacturing precision while increasing the complexity of structural units is difficult. This embodiment reduces the complexity requirements of the structural units in a single modulation layer by using multi-layer modulation. It should be understood that the precision of the structural units in a single modulation layer can be achieved with existing manufacturing processes, while multi-layer modulation allows for relatively flexible adjustment of the overall modulation structure complexity of the spectral chip according to actual needs.

[0142] In detail, taking a two-layer optical modulation layer as an example, the spectral chip 10B includes a filter structure 12B and an image sensor 11B, with the filter structure 12B disposed along the photosensitive path of the image sensor 11B. The filter structure 12B of the spectral chip 10B includes a first optical modulation layer 124B and a second optical modulation layer 125B, wherein the first optical modulation layer 124B and the second optical modulation layer 125B are used to modulate the incident light. The first optical modulation layer 124B and the second optical modulation layer 125B are stacked vertically to form an optical modulation layer 120B of the filter structure 12B. It is understood that in this preferred embodiment of the present invention, the optical modulation layer 120B may further include a third modulation layer or a fourth modulation layer; that is, the number of layers of the optical modulation layer 120B is merely an example and not a limitation.

[0143] The image sensor 11B is used to receive the modulated light signal and process the modulated light signal to obtain the spectral information of the target. The first light modulation layer 124B and the second light modulation layer 125B work together to modulate the incident light. It is worth mentioning that the transmission spectrum matrix A corresponding to the spectral chip 10B in this preferred embodiment of the present invention cannot be simply understood as the convolution of the transmission spectrum A1 of the first light modulation layer 124B and the transmission spectrum A2 of the second light modulation layer 125B, but is a transmission spectrum matrix A formed by the combined action of the first light modulation layer 124B and the second light modulation layer 125B.

[0144] As an example, in this preferred embodiment of the present invention, both the first optical modulation layer 124B and the second optical modulation layer 125B can be implemented as having a modulation aperture structure. Specifically, the first optical modulation layer 124B further includes a plurality of first structural units 1241B, and the second optical modulation layer 125B further includes a plurality of second structural units 1251B, wherein at least one first structural unit 1241B and at least one second structural unit 1251B correspond to each other; that is, the incident light from the object under test is modulated by the first structural unit 1241B and then by the second structural unit 1251B to improve the optical modulation effect of the optical modulation layer 120B. Each first structural unit 1241B further has at least one first modulation aperture 1240B, and each second structural unit 1251B further has at least one second optical modulation aperture 1250B. The first modulation aperture 1240B of the first optical modulation layer 124B differs from the corresponding second modulation aperture 1250B of the second optical modulation layer 125B.

[0145] It is understood that the difference between the first modulation aperture 1240B and the second modulation aperture 1250B may be due to differences in structure (e.g., shape, type) and / or structural parameters (e.g., structural dimensions, structural depth). In one example of the present invention, one of the first modulation apertures 1240B of the first structural unit 1241B is a circular aperture, and the second optical modulation aperture 1250B of the second structural unit 1251B corresponding to the first structural unit 1241B is a square aperture. In another example of the present invention, one of the first modulation apertures 1240B of the first structural unit 1241B is a circular aperture, and the second optical modulation aperture 1250B of the second structural unit 1251B corresponding to the first structural unit 1241B is also a circular aperture, but with different diameters and / or aperture depths.

[0146] To further demonstrate the advantages of this application, such as Figure 5A and Figure 5BThe illustration shows the first structural unit 1241B and the second structural unit 1251B of the optical modulation layer 120B being implemented as a first circular aperture and a second circular aperture, and the transmission spectrum corresponding to the multilayer structure after the first and second circular apertures are combined. Figure 5A In the intended effect, the structural unit shapes corresponding to the first and second curves are both circular holes, but their sizes differ; Figure 5B As shown in the diagram, the curve represents a new modulation effect produced by the combination of the first and second circular holes. It is evident that the combination of two simple graphics can complicate the transmission spectrum, thereby improving the final recovery accuracy.

[0147] like Figure 4A and Figure 4B As shown, the spectral chip 10B further includes a dielectric layer 13B, wherein the dielectric layer 13B is formed between the image sensor 11B and the filter structure 12B of the spectral chip 10B to bond the filter structure 12B and the image sensor 11B. As an example, the dielectric layer 13B can be silicon dioxide, and the dielectric layer 13B has a flat upper surface, thereby improving the bonding performance between the filter structure 12B and the image sensor 11B.

[0148] The spectral chip 10B further includes a connecting layer 14B, which is located between the first optical modulation layer 124B and the second optical modulation layer 125B of the filter structure 12B, and serves to connect the first optical modulation layer 124B and the second optical modulation layer 125B. Preferably, the connecting layer 14B is made of a low refractive index material, such as silicon oxide, which is beneficial for improving the complexity of the transmission spectrum of the spectral chip 10B. It is worth mentioning that the refractive index of the connecting layer 14B differs significantly from that of the optical modulation layer 120B.

[0149] like Figure 6 Another optional embodiment of the spectral chip 10B of the present invention is shown, wherein the spectral chip 10B further includes at least one filling structure 15B, wherein the filling structure 15B is formed in the first light modulation layer 124B and / or the second light modulation layer 125B of the light filtering structure 12B, and light can be transmitted through the filling structure 15B of the spectral chip 10B. It is worth mentioning that, in this preferred embodiment of the present invention, the filling structure 15B of the spectral chip 10B is formed within the modulation aperture of the first light modulation layer 124B and / or the second light modulation layer 125B to improve modulation complexity.

[0150] As an example, the first optical modulation layer 124B fills the filling structure 15B, or the second optical modulation layer 125B fills the filling structure 15B. Alternatively, both the first optical modulation layer 124B and the second optical modulation layer 125B may have filling structures. The corresponding filling structures 15B may be the same or different. Preferably, in this embodiment, the first optical modulation layer 124B and the second optical modulation layer 125B are made of high refractive index materials, such as silicon nitride or single-crystal silicon; the filling structure 15B is formed of a low refractive index material, such as metal or silicon oxide. Further, the spectral chip 10B further includes a capping layer 16B, which is located on the upper surface of the first optical modulation layer 124B of the filter structure 12B. Therefore, it can be understood that the incident light of the object under test first passes through the cover layer 16B, enters the first light modulation layer 124B of the filter structure 12B, that is, passes through the first structural unit 1241B, then enters the connection layer 14B, and then enters the second light modulation layer 125B, that is, after passing through the second structural unit 1251B, the incident light is modulated, and then received by the image sensor 11B.

[0151] Example 4

[0152] As image sensor technology advances, the size of the physical pixels becomes smaller, making it difficult to focus incident light onto the corresponding physical pixels, leading to interference between them. This interference causes deviations between the matrix A and output bi of the interfering pixel unit and the actual result, resulting in inaccurate spectral reconstruction results that do not match reality.

[0153] To address the aforementioned technical issues, such as Figure 7 and Figure 8 As shown, a spectral chip 10C according to another aspect of the present invention is illustrated in the following description. The spectral chip 10C includes an image sensor 11C, a filter structure 12C located on the photosensitive path of the image sensor 11C, and a plurality of grids 17C for preventing crosstalk of incident light at the image sensor 11C. Accordingly, the image sensor 11C includes a substrate layer 111C and at least one physical pixel formed on the substrate layer 111C. In this embodiment, the physical pixels are arranged in an array on the substrate layer 111C to form a physical pixel array. The filter structure 12C includes at least one structural unit 121C having a specific transmission spectrum for modulating incident light, and the grids 17C are located between the structural units 121C. Each structural unit 121C of the filter structure 12C and at least one physical pixel group of the image sensor 11C constitute a structural pixel 102C.

[0154] The spectral chip 10C can prevent crosstalk between incident light entering the structural pixels 102C by using grids 17C disposed between the structural pixels 102C. It is worth noting that in this embodiment, the structural pixels 102C can be divided into two cases: one where a group of structural units 121C corresponds to one physical pixel, in which case the grids 17C can be understood as being disposed between adjacent structural units 121C and surrounding the corresponding physical pixel. Preferably, in this application, a group of structural units 121C corresponds to multiple physical pixels, such as 4, 9, or 16 physical pixels, etc., and the multiple physical pixels are square, such as 2*2, 3*3, or 4*4 physical pixels. The grids 17C are disposed with the structural pixels 102C as units, that is, the grids 17C are disposed between adjacent filter structure units 12C and surround the corresponding multiple physical pixels.

[0155] Furthermore, the grid 17C can be made of metallic or non-metallic materials, such as copper or aluminum, or it can be made of a low-n material, wherein the low-n material can be a low-refractive-index material. It is worth mentioning that metallic or low-n materials can cause incident light incident on the surface of the grid 17C to be reflected into the corresponding physical pixel, which, in addition to preventing crosstalk, can also improve the corresponding QE value.

[0156] Optionally, in other alternative embodiments of the present invention, the light modulation layer 120C of the spectral chip 10C further includes a plurality of modulation regions 122C and at least one non-modulation region 123C for spacing adjacent modulation regions 122C, wherein the modulation regions 122C modulate the incident light, and the modulated incident light is received by the image sensor 11C, and the corresponding spectrum can be recovered by calculation. In this preferred embodiment of the present invention, the modulation regions 122C of the light modulation layer 120C have structural pixels 102C composed of structural units 121C and physical pixels, so the grid 17C is disposed between the structural units 121C, with the structural pixels 102C as units; the non-modulation regions 123C of the light modulation layer 120C are disposed between the physical pixels, with the grid 17C as units.

[0157] Example 5

[0158] Current spectral imaging technology is mainly based on spectrometers combined with mechanical scanning structures. This approach requires precise control of the mechanical scanning and a trade-off between scanning step size, leading to increased costs and reduced temporal resolution. In contrast, spectrometers utilizing filters and photodetector arrays, due to their inherent two-dimensional photosensitive structure, can directly achieve spectral imaging through arraying. This approach has irreplaceable advantages in cost, temporal resolution, and integration. Combined with computational spectroscopy methods, the spatial resolution of this approach can be significantly improved, resulting in a substantial overall advantage. However, this approach has significant data storage and logical processing requirements, especially under conditions of high spectral resolution, high spatial resolution, and high frame rate, posing new challenges to the system architecture.

[0159] Referring to the accompanying drawings of this invention Figure 9 As shown, a spectral analysis device according to another preferred embodiment of the present invention will be described below. Unlike the first preferred embodiment described above, the spectral analysis device includes a spectral chip 10D, which comprises an image sensor 11D, a filter structure 12D located on the optical path of the image sensor 11D, multiple memories 18D, and a logic processing component 19D. The image sensor 11D, the memories 18D, and the logic processing component 19D are arranged in a stacked structure to realize the transmission and processing of data and / or signals. The memories 18D are typically RAM, such as DRAM, SRAM, etc. The logic processing component 19D consists of multiple first-level logic processors 191D and multiple second-level logic processors 192D. Furthermore, the logic processors can be processing units such as ISP, CPU, GPU, or NPU, or logic computing units customized for specific algorithms, i.e., computing units that have specific operators embedded in their memory.

[0160] The spectral chip 10D is divided into a first stacked layer 101D and a second stacked layer 103D. The first stacked layer 101D includes the image sensor 11D, a plurality of memories 18D, and a plurality of first-level logic processors 191D with physical pixels as the smallest unit. The image sensor 11D, the plurality of memories 18D, and the logic processors 19D are stacked sequentially, wherein one physical pixel of the image sensor 11D corresponds to one memory 18D and one first-level logic processor 191D. The second stacked layer 103D includes at least one second-level logic processor 192D, which is located below the first stacked layer 101D and connected to the first-level logic processors 191D. Photoelectric conversion, signal storage, and traditional image processing, such as signal scanning phase difference and color difference processing, can be performed through the first stacked layer. This layer processes the signal read from each physical pixel. Based on this, the signals read from the physical pixels constituting the spectral pixel 1020 are then transmitted to the secondary logic processor 192D of the corresponding second stacked layer 103D. The second stacked layer 103D performs logic operations related to spectral recovery, such as using the aforementioned artificial neural network, least 2 norm, etc.

[0161] In this embodiment, the secondary logic processor 192D is connected to at least one primary logic processor 191D, enabling direct or indirect data transmission. Both processors perform different processing on the received signals, allowing each secondary logic processor 192D of each spectral pixel 1020 to directly perform spectral recovery. By further arraying and expanding the spectral pixels 1020, a spectral image can be obtained. It is worth noting that the secondary logic processor 192D can be configured in units of spectral pixels 1020. For example, if each spectral pixel 1020 contains 10*10 physical pixels, then the secondary logic processor 192D can be connected to 10*10 primary logic processors 191D.

[0162] From the perspective of the stacking structure, the image sensor 11D of the spectral analysis device in this embodiment has an integration function, both in terms of physical structure and data flow. That is, in terms of physical aspect, the secondary logic processor is integrated (close to) its corresponding physical pixel as much as possible to reduce the data transmission distance. In terms of data aspect, the data of the physical pixels that constitute the spectral pixel 1020 are uniformly transmitted to the corresponding secondary logic processor for calculation.

[0163] Example 6

[0164] Figure 10A and Figure 10BThe imaging method of the spectral analysis device according to any of the preferred embodiments of the present invention is further explained. In this preferred embodiment of the present invention, taking one physical pixel corresponding to a group of structural units 121 as an example, it is further explained how to recover spectral information using m groups of physical pixels (that is, pixels on the image sensor) and their corresponding m groups of structural units 121, denoted as spectral pixel 1020. It is worth noting that in the embodiments of this application, multiple physical pixels may also correspond to a group of the aforementioned structural units 121. In this preferred embodiment of the present invention, a structural pixel 1201 is constituted by a group of the aforementioned structural units 121 and at least one corresponding physical pixel, and at least one of the aforementioned structural pixels 1201 constitutes a spectral pixel 1020.

[0165] Based on the above theoretical foundation, a snapshot-type spectral imaging device can be realized by arraying the spectral pixels 1020.

[0166] As an example, such as Figure 10A As shown, an image sensor with 1896*1200 pixels is used. Figure 10A (A portion of the image sensor area is shown). Simultaneously, m=4 is selected, meaning 4x4 pixel units are chosen to form a spectral pixel 1020. This results in 474x300 independent spectral pixels 1020, each of which can have its spectral curve calculated individually using the method described above. The spectral analysis device can perform snapshot-style spectral imaging of the object under test, thus obtaining the spectral information of the object in a single exposure.

[0167] Based on this, the selection method of spectral pixels can be rearranged according to actual needs without making any adjustments to the image sensor, thereby improving spatial resolution. For example... Figure 10B As shown, you can select a close arrangement of solid and dashed boxes to increase the spatial resolution in the example above from 474*300 to nearly 1896*1200.

[0168] Furthermore, in this preferred embodiment of the present invention, the image sensor 11 can be rearranged according to the requirements of spatial resolution and / or spectral resolution. For example, when higher spectral resolution is required, 8*8 unit pixels can be used to form one spectral pixel; when higher spatial resolution is required, 3*3 physical pixels can be used to form one spectral pixel.

[0169] It's worth noting that this approach can be combined with edge detection algorithms. The edge detection algorithm identifies the boundaries of the object under test, and then the spectral pixels are reconstructed. At this point, the spectral pixels are reconstructed based on the boundaries of the object in the image. This involves preprocessing the image data from the spectral chip to identify sensor-level sensitivity edges and dynamically adjusting the filter group for edge restoration. Edge detection operators are used to detect edges in the image, avoiding areas with large pixel gradients to ensure consistent pixel intensity for spectral restoration. When selecting pixels for spectral restoration, edge regions are avoided; the nearest, connected, and a specific number of pixels are selected. Notably, the positions of the pixels used for spectral restoration can be dynamically selected to ensure consistent intensity, improve the accuracy of spectral restoration, and prevent spectral distortion at edges.

[0170] Example 7

[0171] Referring to the accompanying drawings of this invention Figure 11 As shown, the imaging method of the spectral analysis device according to any of the preferred embodiments of the present invention will be explained in the following description. In this preferred embodiment of the invention, high spatial resolution is achieved by moving the filter structure 12 of the spectral analysis device.

[0172] In detail, in this embodiment, the filter structure 12 is located on the optical path of the image sensor 11 but is not fixed to the image sensor 11. The spectral analysis device further includes a driving mechanism 110, which is driveably connected to the filter structure and can drive the filter structure 12 to move on a horizontal plane (the horizontal plane refers to a plane perpendicular to the optical axis). Preferably, in this embodiment, the horizontal area of ​​the filter structure 12 is greater than or equal to the horizontal area of ​​the image sensor 11 (or the imaging surface area of ​​the image sensor). Therefore, the filter structure 12 and the image sensor 11 are relatively movable. By moving them relative to each other, the structural unit 121 of the filter structure 12 corresponding to the physical pixels on the image sensor 11 can be changed. Taking one physical pixel as one structural unit as an example, where physical pixel P(3,3) corresponds to structural unit T(3,3), when the filter structure 12 moves relative to the image sensor 11 along the positive X-axis by one period distance of structural unit 121 (i.e., moves by one physical pixel size), then physical pixel P(3,3) corresponds to structural unit T(2,3); the corresponding P(4,3) changes to correspond to T(3,3); if it moves along the positive Y-axis, then physical pixel P(3,3) corresponds to structural unit T(3,4).

[0173] It should be noted that the moving distance of the filter structure 12 can be based on the periodic distance of the structural unit in the X / Y axis direction, or it can be based on the length of the physical pixel in the X / Y axis direction, or it can be an integer multiple of the periodic distance of the structural unit.

[0174] It is worth mentioning that, in this preferred embodiment of the present invention, the spectral chip 10 has multiple different shooting positions, that is, the filter structure 12 of the spectral chip 10 can be driven to move between the multiple shooting positions, and the spectral analysis device can take pictures at each of the different shooting positions of the filter structure 12 to obtain at least one spectral information. The data processing unit of the spectral analysis device can synthesize a final spectral image based on the spectral information obtained at each of the shooting positions. It can be understood that by moving the filter structure 12, the light signal corresponding to the test object or region is modulated by different structural units to obtain different modulation effects, and then the same region after multiple modulations is synthesized to obtain the final spectral image, thereby improving the resolution. Those skilled in the art will understand that the spectral image can contain two-dimensional images to a certain extent, that is, conventional multi-channel imaging.

[0175] As an example, in this preferred embodiment of the present invention, the filter structure 12 is driven by the driving mechanism 110, so that the filter structure 12 and the image sensor 11 move relative to each other at a periodic distance of the structural unit 121, and at the same time, spectral information is captured at each position to increase the spatial resolution.

[0176] In detail, the filter structure 12 and the image sensor 11 are located at position A, where the spectral analysis device captures and acquires first spectral information. Upon completion of the capture at position A, the driving mechanism 110 drives the filter structure 12 to move right by one periodic distance of the structural unit 121 to position B, where the spectral analysis device captures and acquires second spectral information. Then, the driving mechanism 10 drives the filter structure 12 to move backward by one periodic distance of the structural unit 121 to position C, where the spectral analysis device captures and acquires third spectral information. The driving mechanism 10 then drives the filter structure 12 to move left by one periodic distance of the structural unit 121 to position D, where the spectral analysis device captures and acquires fourth spectral information. After the capture is complete, the driving mechanism 10 drives the filter structure 12 to move forward by one periodic distance of the structural unit from position D back to position A, reaching the initial position. The spectral analysis device can capture spectral data at each of these locations (A, B, C, D) and synthesize the acquired spectral data to achieve higher resolution spectral imaging. It is important to note that the positional relationship between the object or region under test, the optical system, and the image sensor remains unchanged throughout this process.

[0177] In this way, the timing of moving the filter structure 12 is consistent with a reference time that is synchronized with the image sensor readout timing, such as the readout frame time for pixel output.

[0178] Each time the filter structure 12 is driven to move a small distance, the image sensor 11 can acquire spectral data for each physical pixel. Furthermore, by combining spectral data obtained from various types of combined patterns for the amount of movement, a spectral image can be synthesized.

[0179] It is worth mentioning that there is a gap between the filter structure 12 and the image sensor 11, the gap being less than or equal to 20µm and greater than or equal to 0.5µm. Preferably, the gap between the filter structure 12 and the image sensor 11 is 2µm. It is understood that if the gap is too large, it is easy to cause light crosstalk, which will lead to inaccurate information received by the structure pixels, resulting in poor recovery accuracy. On the other hand, if the gap is too small, it will directly cause interference to the movement of the filter structure.

[0180] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, wherein the imaging method includes the following steps:

[0181] (201) Drive a filter structure 12 of a spectral chip 10 to move, such that the filter structure 12 moves at least one position relative to the image sensor 11, and captures spectral information at corresponding positions; and

[0182] (202) The spectral information captured at each location is synthesized into a spectral image.

[0183] It should be noted that the spectral information described in this invention can be either a spectral image or spectral data. The spectral image can also be output as a two-dimensional image, depending on the requirements.

[0184] In the imaging method of the present invention, the filter structure 12 of the spectral chip 10 is connected to a driving mechanism 110 in a driving manner, and the driving mechanism 110 drives the filter structure 12 to move on a surface parallel to the image sensor 11. There is a gap between the filter structure 12 of the spectral chip 10 and the image sensor 11, wherein the gap is less than or equal to 20 μm and greater than or equal to 0.5 μm.

[0185] In step (201) of the above imaging method, the filter structure 12 is driven to move by a periodic distance of the structural unit 121 of the filter structure 12 in the X / Y axis direction; or by a unit of length of the physical pixel of the image sensor 11 in the X / Y axis direction; or by an integer multiple of the periodic distance of the structural unit.

[0186] Example 8

[0187] Referring to the accompanying drawings of this invention Figure 12 As shown, the imaging method of the spectral analysis device according to any of the preferred embodiments of the present invention will be explained in the following description. In this preferred embodiment of the invention, high spatial resolution is achieved by moving the spectral chip 10 of the spectral analysis device.

[0188] The difference from the preferred embodiment described above is that, in this preferred embodiment of the present invention, the spectral chip 10 is connected to the driving mechanism 110 in a driving manner, the driving mechanism 110 drives the spectral chip 10 to move as a whole, and the spectral analysis device captures spectral information at different moving positions of the spectral chip 10.

[0189] It is worth mentioning that, in this preferred embodiment of the present invention, the spectral chip 10 has multiple different shooting positions, that is, the spectral chip 10 can be driven to move between the multiple shooting positions, and the spectral analysis device can take pictures at each of the shooting positions to obtain at least one spectral information. The data processing unit of the spectral analysis device can combine the spectral information obtained at each of the shooting positions into a spectral image.

[0190] In detail, the spectral chip 10 is located at position A, where the spectral analysis device captures and acquires first spectral information. Upon completion of the capture at position A, the driving mechanism 110 moves the spectral chip 10 to the right by one periodic distance of the structural unit 121 to position B, where the spectral analysis device captures and acquires second spectral information. The driving mechanism 10 then moves the spectral chip 10 backward by one periodic distance of the structural unit 121 to position C, where the spectral analysis device captures and acquires third spectral information. The driving mechanism 10 then moves the spectral chip 10 to the left by one periodic distance of the structural unit 121 to position D, where the spectral analysis device captures and acquires fourth spectral information. After capturing all information, the driving mechanism 10 moves the spectral chip 10 forward by one periodic distance of the structural unit from position D back to position A, returning to the initial position. The spectral analysis device can capture images at each of these positions (A, B, C, D) and synthesize the spectral information acquired at each position, thereby achieving higher resolution spectral imaging. Spectral information can be understood as images, that is, images are captured at various locations; or it can be spectral data, that is, spectral data is acquired at various locations and then processed.

[0191] As an example, in this preferred embodiment of the invention, the spectral chip 10 can be driven by the driving mechanism 110 and has the function of moving a small distance relative to the plane containing the two-dimensional physical pixels. The timing of moving the spectral chip 10 is consistent with a reference time, which is synchronized with, for example, the readout frame time for physical pixel output or the readout timing of an image sensor. The single movement distance of the spectral chip 10 can be in units of the size of the physical pixel in the X / Y axis direction (in integer multiples) or in units of structural pixels.

[0192] Each time the spectral chip moves a tiny distance, the spectral analysis device can obtain spectral data for each pixel. Furthermore, by combining spectral data obtained from various types of combined patterns for different amounts of movement, a high spatial resolution spectral image can be synthesized.

[0193] Optionally, in other alternative embodiments of the present invention, the driving mechanism 110 is driveably connected to the image sensor 11 of the spectral chip 10, and the driving mechanism 110 drives the image sensor 11, causing the image sensor 11 to move relative to the filtering mechanism 12. That is, in some cases, the image sensor can also be moved to capture multiple images, which are then combined. It is understood that the movement of the image sensor 11 is in the same direction as the movement of the filtering mechanism 12 in the preferred embodiment described above.

[0194] Optionally, spectral information under different conditions can be acquired by moving the lens, and then synthesized to improve spatial resolution.

[0195] The main technical principle of this invention is to obtain at least two sets of spectral information by keeping the object under test and the optical system unchanged, and by changing at least a portion of the structure of the spectral chip 10 (such as the entire spectral chip 10, the image sensor 11, or the filter structure 12), or by relatively moving the optical system 20 and the spectral chip 10 to obtain at least two sets of spectral information. Then, the spectral information is integrated to obtain a higher resolution spectral image. Preferably, it can be the synthesis of four, nine, or other sets of spectral information. Here, the spectral information can be an image or spectral data.

[0196] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, wherein the imaging method includes the following steps:

[0197] (301) Drive a spectral chip 10 to move, and capture at least one spectral information at each corresponding position; and

[0198] (302) The spectral information obtained at each location is synthesized into a spectral image.

[0199] In the imaging method described above in this invention, the spectral chip 10 is connected to a driving mechanism 110 in a driving manner, and the driving mechanism 110 drives the spectral chip 10 to move a small distance on a horizontal plane.

[0200] In step (a) of the imaging method described above, the spectral chip 10 is driven to move by a periodic distance of the structural unit 121 of the spectral chip 10 in the X / Y axis direction; or by a length dimension of the physical pixel of the image sensor 11 of the spectral chip 10 in the X / Y axis direction; or by an integer multiple of the periodic distance of the structural unit.

[0201] Example 9

[0202] Referring to the accompanying drawings of this invention Figure 13As shown, the imaging method of the spectral analysis device according to any of the preferred embodiments of the present invention will be explained in the following description. In this preferred embodiment of the present invention, high spatial resolution is achieved by moving the filter structure 12 of the spectral analysis device. Unlike the preferred embodiment described above, which synthesizes a high-resolution image by capturing multiple spectral information, the spectral analysis device of this preferred embodiment of the present invention improves resolution through other means. The filter structure 12 is transmissively connected to the driving mechanism 110, and the driving mechanism 110 drives the filter structure 12 to move, so that the filter structure 12 of the spectral chip 10 and the image sensor 11 produce a small movement distance along the horizontal plane where the two-dimensional physical pixels are located.

[0203] Specifically, taking one structural unit 121 of the filter structure 12 corresponding to one physical pixel as an example, the structural unit 121 of the filter structure 12 is denoted as T. i，j The physical pixels on the image sensor 11 are denoted as P. i，j By moving the filter structure 12, different structural units 121 correspond to the same physical pixel, thereby obtaining signals with different modulation effects of the same incident light for the same physical pixel, and then restoring them to improve the resolution.

[0204] When the number of corresponding structural units in the filter structure is limited, high resolution can be achieved by moving the filter structure. That is, each physical pixel of the image sensor 11 corresponds to a different structural unit 121 during the movement of the filter structure 12, while the object under test and the optical system remain unchanged, thus the incident light information incident on the physical pixel remains unchanged. Since the change in the structural unit 121 causes a change in the modulated signal b, n+1 modulated signals b can be obtained by moving the filter structure n times. These n+1 signals can be used to recover the spectral curve, and then arrayed according to their corresponding positions to obtain the spectral information. It is important to note that for n moves, the lower the correlation coefficient of the transmission spectrum curves corresponding to the n+1 structural units for the same physical pixel, the better; that is, the smaller the correlation coefficient.

[0205] As an example, the number of structural units 121 is 200*200, and the image sensor 121 has 1896*1200 physical pixels. When in the initial position, the structural unit T... 1，1 Corresponding to P 848 , 500 After obtaining signal b1, the filter structure is moved one grid position so that T 2，1 Corresponding to P 848，500 Given signal b2, assuming 24 moves, then T 25，1Corresponding to P 848，500 By obtaining signal b25, we can obtain vector b = [b1, b2, ..., b25]. We can then use vector b to recover the corresponding pixel P. 848，500 The spectral curves are obtained. During each movement, different physical pixels will generate a corresponding set of signals b with different structural units, thus obtaining the signal vectors corresponding to 200*200 physical pixels, which can then be used to recover 200*200 spectral curves. Spectral information can be obtained by arraying these curves.

[0206] Preferably, one structural unit 121 of the filter structure 12 corresponds to multiple physical pixels to form a structural pixel 102. The moving distance of the filter structure 12 is measured in units of the period length of the structural unit 121 corresponding to the structural pixel, thereby enabling the same multiple physical pixels and different structural units to form a structural pixel with a new transmission spectrum. It can be understood that the movement can also be measured in units of the side length of the physical pixel in the X / Y direction.

[0207] The difference between this embodiment and Embodiment Seven is that in Embodiment Seven, the moving filter structure 12 can be understood as each spectral pixel moving, allowing the corresponding image sensor to obtain different signals for synthesis. In this embodiment, however, the moving filter structure 12 allows one or more physical pixels to correspond to different modulation units to form different structural pixels, further achieving the effect of spectral pixels for spectral restoration or imaging. This embodiment can also improve spectral resolution for spectral curve restoration.

[0208] According to another aspect of the present invention, the present invention further provides an imaging method for a spectral analysis device, wherein the imaging method includes the following steps:

[0209] (401) Drive a filter structure 12 of a spectral chip 10 to move, such that each physical pixel of an image sensor 11 of the spectral chip 10 corresponds to a plurality of different structural units 121 of the filter structure 12, and each physical pixel of the spectral chip 10 obtains a set of signals based on the corresponding structural units 121; and

[0210] (402) Based on the obtained signal, the spectral curve corresponding to each pixel is obtained, and the spectral curve is arrayed to obtain spectral information.

[0211] In the imaging method of the spectral analysis device of the present invention, if one structural unit 121 of the filter structure 12 corresponds to multiple physical pixels to form a structural pixel 102, the moving distance of the filter structure 12 is measured in units of the period length of the structural unit 121 corresponding to the structural pixel. This allows the same multiple physical pixels and different structural units to form a structural pixel with a new transmission spectrum. Optionally, the filter structure 12 is driven to move in units of the side length of the physical pixel in the X / Y direction.

[0212] The spectral recovery method of the spectroscopic device according to any of the preferred embodiments of the present invention will be explained in the following description. In this preferred embodiment of the invention, high spectral resolution is achieved by zooming the optical system 20.

[0213] It is worth mentioning that the spectral analysis device of this preferred embodiment of the present invention corresponds to the broadband spectral recovery principle, denoting the intensity signal of the incident light at different wavelengths λ as x(λ), and the transmission spectrum curve of the filter structure 12 as T(λ). The filter structure 12 has m groups of structural units 121, and the transmission spectrum of each group of structural units 121 is different. Overall, the filter structure can be denoted as T. i (λ)(i=1,2,3,…,m). Each group of structural units 121 has a corresponding physical pixel below it, which detects the light intensity b modulated by the filter structure. i As can be seen from the principle, the more structural units 121 in the filter structure 12, the higher the spectral resolution and the higher the recovery accuracy. However, an excessive number of structural units 121 can lead to increased processing difficulty, higher costs, and may also require a larger spectral chip size. Therefore, this embodiment proposes a method to improve spectral resolution without increasing the number of structural units, addressing these issues.

[0214] When the optical system 20 zooms, the principal angle and / or the receiving cone angle of the incident light reaching the spectral chip change, further causing a change in the corresponding transmission spectrum matrix A. The same incident light reaching the altered transmission spectrum matrix A' will produce a different light intensity b in the image sensor 11 compared to the original intensity. i Therefore, while keeping the incident light constant, different transmission spectrum matrices A can be generated by the optical system 20, thereby obtaining the light intensity corresponding to different transmission spectrum matrices A. Then, the spectrum can be recovered from all the light intensities, which can improve the accuracy.

[0215] As an example, the optical system of the spectral device may be, but is not limited to, a zoom lens. For example, the zoom lens has three focal lengths, and the transmission spectrum matrices A1, A2, and A3 corresponding to the three focal lengths are obtained through calibration. When the same incident light enters the zoom lens, the first focal length is taken, and the light intensity b1 is obtained through the corresponding transmission spectrum matrix A1. Then, the focal length is changed to the second focal length, keeping the incident light unchanged, and the light intensity b2 is obtained through the transmission spectrum matrix A2. The focal length is further changed to the third focal length, keeping the incident light unchanged, and the light intensity b3 is obtained through the transmission spectrum matrix A3. Then, the three sets of vector light intensities b1, b2, and b3 are processed to obtain the vector light intensity b, and the spectral curve is recovered through the light intensity b.

[0216] The spectral device of the present invention further includes a light homogenizing component 90, wherein the light homogenizing component 90 may be, but is not limited to, a frosted glass, a diffraction element, a light homogenizer, or other devices with a light homogenizing effect. The light homogenizing component 90 is disposed at the front end of the optical system 20, that is, after the incident light is homogenized by the light homogenizing component 90, it enters the optical system 20, and then is modulated by the filter structure 12 before being received by the image sensor 11.

[0217] Furthermore, for the recovery of the spectral curve, in order to improve the resolution, it is preferable that the incident light can fully cover the filter structure 12 at the point where the effective focal length (or effective back focal length) is the smallest. Thus, during zooming, the effective focal length increases and the incident light coverage area exceeds the range of the filter structure, but it can ensure that all structural units 121 participate in the modulation of the incident light.

[0218] The optical system of the above-described variable focal length spectrometer is further illustrated in Examples 10 to 13 below.

[0219] Example 10

[0220] Figures 14 to 15B The following further illustrates a specific implementation of an optical system 20 of the spectral analysis apparatus according to any of the preferred embodiments of the present invention. As an example, in this preferred embodiment of the present invention, the optical system 20 is implemented as a vertical lens.

[0221] In a detailed embodiment of the present invention, the optical system 20 can be implemented as a zoom lens group. The optical system 20 includes at least one lens assembly 21. Based on the characteristics of the analyte, by adjusting the position of the at least one lens assembly 21, the effective focal length of the optical system 20 is changed, thereby altering the transmission spectrum matrix corresponding to the principal angle and / or the receiving cone angle of the incident light from the analyte onto the spectral chip 10.

[0222] like Figure 14As shown, the lens assembly 21 of the optical system 20 further includes a first lens group 21a, a second lens group 21b, and a third lens group 21c, wherein the first lens group 21a, the second lens group 21b, and the third lens group 21c are arranged along the same optical axis, and the second lens group 21b is located between the first lens group 21a and the third lens group 21c.

[0223] Preferably, the first lens group 21a and the third lens group 21c are relatively fixed in position on the optical axis, while the second lens group 21b can be driven and moved along the optical axis, thereby achieving zoom (changing the effective focal length). It is worth mentioning that, in other optional embodiments of the present invention, the first lens group 21a can also be movable, and the zoom magnification can be increased by moving the first lens group 21a. Further, the second lens group 21b further includes at least one zoom lens 211b and at least one compensation lens 212b, and zoom is achieved by moving the zoom lens 211b and the compensation lens 212b.

[0224] The spectral chip 10 includes a filter structure 12 and an image sensor 11. The filter structure 12 is located on the optical path of the image sensor 11. The filter structure 12 includes multiple structural units 121, wherein each structural unit 121 can modulate the incident light after passing through the optical system 20 before it is received by the image sensor 11. Each structural unit 121 has a corresponding transmission spectrum curve, which can modulate the incident light.

[0225] Based on the above-mentioned spectral recovery and transmission spectrum matrix, the working principle of the spectral device is as follows: the incident light containing the object to be measured first enters the optical system 20. After adjustment by the optical system 20, it will be incident on the surface of the filter structure 12 with different specific principal beam angles and receiving beam angles. Then, it is modulated by different transmission spectrum matrices of the filter structure 12 and then received by the image sensor 11 to obtain spectral information modulated by different transmission spectrum matrices. The spectral information is then processed, and the corresponding spectral curve is recovered or calculated through an algorithm to improve the spectral resolution.

[0226] The optical system 20 further includes at least one moving mechanism 22, wherein the second lens group 21b of the optical system 20 is tractively connected to the moving mechanism 22, and the moving mechanism 22 drives the second lens group 21b to move. Specifically, the moving mechanism 22 is tractably connected to the zoom lens 211b and the compensation lens 212b of the second lens group 21b, and the moving mechanism 22 drives the zoom lens 211b and the compensation lens 212b of the second lens group 21b to adjust the focal length of the optical system 20.

[0227] The transmission spectrum curve of the structural unit 121 of the spectral chip 10 corresponding to the optical system 20 changes due to variations in the principal angle and / or cone angle of the incident light. Since the transmission spectrum matrix A is composed of the transmission spectrum curves of multiple structural units, the zooming of the optical system 20 further causes changes in the transmission spectrum matrix A, thereby improving the spectral resolution. The moving mechanism 22 can be implemented as a motor, piezoelectric ceramic, or other device that moves the lens.

[0228] Furthermore, the spectral analysis device further includes at least one focusing mechanism 40, wherein the focusing mechanism 40 is connected to the first lens group 21a, and the focusing is achieved by driving the first lens group 21a through the focusing mechanism 40. Optionally, in other alternative embodiments of the present invention, the focusing mechanism 40 may also operate on the entire optical system 20, that is, the optical system 20 is connected to the focusing mechanism 40, and focusing is achieved by moving the optical system 230.

[0229] The optical system 20 further includes an aperture stop 23, which is disposed at the front end of the first lens group 21a.

[0230] Example 11

[0231] Figures 16 to 17 Further explanation is provided regarding another specific embodiment of the optical system 20A of the spectral analysis apparatus described in any of the preferred embodiments of the present invention. As an example, in this preferred embodiment of the present invention, the optical system 20A is implemented as a liquid lens.

[0232] The optical system 20A described in this embodiment includes at least one liquid lens assembly 25A and at least one lens assembly 21A. The liquid lens assembly 25A and the lens assembly 21A are arranged one after the other along the same optical axis. The liquid lens assembly 25A can change its curvature, thereby changing the focal length of the optical system 20A.

[0233] The liquid lens assembly 25A may include at least one deformable lens body 251A, a flexible transparent cover member 252A, and an actuator 253A, wherein the flexible transparent cover member 252A is attached to the surface of the at least one deformable lens body 251A to provide mechanical stability to the at least one deformable lens body 251A. The actuator 253A is used to shape the flexible transparent cover member 252A into a desired shape. The actuator 253A is located on the upper surface of the flexible transparent cover member 252A, and the desired shape is defined by a configuration pattern of the actuator 253A and a respective voltage amplitude applied to the configuration pattern of the actuator 253A. Work is performed on the deformable lens body 251A by the actuator 253A, causing the deformable lens body 251A to deform, thereby causing the optical system 20A to zoom. Preferably, the at least one deformable lens body 251A has an elastic modulus greater than 300 Pa, thereby avoiding deformation caused by attraction in the flexible transparent cover component 252A during normal operation. Preferably, the lens body 251A has a refractive index as high as possible, such as in the range of 1.35-1.90. Accordingly, the refractive index of the lens body should be at least 1.35, such as in the range of 1.35-1.75, such as in the range of 1.35-1.55. The absorptivity of the deformable lens body 251A in the visible light region is less than 10% per millimeter of thickness, and the deformable lens body 251A comprises a polymer network of cross-linked or partially cross-linked polymers, and further comprises mixed oils or bound oils, thereby increasing the refractive index of the polymer network of cross-linked or partially cross-linked polymers.

[0234] Similar to the preferred embodiment described above, the spectral analysis device further includes at least one focusing mechanism 40A, wherein the focusing mechanism 40A is connected to the first lens group 21a, and the focusing is achieved by driving the first lens group 21a through the focusing mechanism 40A. The optical system 20A also includes an aperture stop 23A, which is disposed at the front end of the first lens group 21a.

[0235] The spectral analysis device further includes at least one image stabilization mechanism 50A, wherein the image stabilization mechanism 50A is connected to the liquid lens assembly 25A and / or the at least one lens assembly 21A of the optical system 20A, or the image stabilization mechanism 50A is connected in a driving manner to the spectral chip 10, thereby driving the optical system 20A and / or the spectral chip 10 to achieve the image stabilization function of the spectral analysis device. Preferably, the image stabilization mechanism is disposed on the lens assembly 21A.

[0236] Example 12

[0237] Figures 18A to 19 Further explanation is provided regarding another specific embodiment of the optical system 20B of the spectral analysis apparatus described in any of the preferred embodiments of the present invention. As an example, in this preferred embodiment of the present invention, the optical system 20B is implemented as a periscope lens.

[0238] The optical system 20B is a zoom lens, comprising at least one lens assembly 21B, at least one moving mechanism 22B, and at least one deflector 26B. The at least one moving mechanism 22B is connected to the at least one lens assembly 21B and drives the at least one lens assembly 21B to move, thereby changing the focal length of the optical system 20B. The deflector 26B is disposed at the front end of the at least one lens assembly along its optical axis, deflecting the transmission direction of light incident on or exiting the at least one lens assembly 21B. The lens assembly 21B further comprises a first lens group 21a, a second lens group 21b, and a third lens group 21c, wherein the first lens group 21a, the second lens group 21b, and the third lens group 21c are arranged along the same optical axis, and the second lens group 21b is located between the first lens group 21a and the third lens group 21c.

[0239] The deflector 26B, the first lens group 21a, the second lens group 21b, and the third lens group 21c are arranged sequentially. After the incident light enters the deflector 26B and is deflected, it passes through the first lens group 21a, the second lens group 21b, and the third lens group 21c in sequence to reach the filter structure 12. After being modulated by the filter structure 12, it is received by the image sensor 11.

[0240] It is worth mentioning that at least one of the first lens group 21a, the second lens group 21b, and the third lens group 21c of the lens assembly 21B is tractably connected to the moving mechanism 22B. The moving mechanism 22B drives the first lens group 21a, the second lens group 21b, or the third lens group 21c to move, thereby changing the optical power of the optical system 20B. That is, the first lens group 21a can also be moved, and the zoom magnification is increased by moving the first lens group 21a. Preferably, the moving mechanism 22B is connected to the second lens group 21b, wherein the second lens group 21b includes at least one zoom lens 211b and at least one compensation lens 212b, and zoom is achieved by moving the zoom lens 211b and the compensation lens 212b. The at least one zoom lens 211b and the at least one compensation lens 212b of the second lens group 21b are tractably connected to the moving mechanism 22B, and the moving mechanism 22B drives the zoom lens 211b to move along the optical axis to adjust the overall focal length of the optical system 20B.

[0241] In this embodiment, the turning element 26B redirects the incident light from a vertical direction to a horizontal direction, thereby reducing the height of the spectral analysis device. The turning element 26B can be implemented as a prism or a mirror. Notably, at least one front lens group (not shown in the figure) is provided at the front end of the turning element 26B. This front lens group can increase the field of view (FOV) or the light flux of the spectral analysis device. The front lens group can be implemented as a light homogenizing system.

[0242] Furthermore, since the optical system 20B typically corresponds to a long optical path, its size in one direction becomes excessively large. For example... Figure 18B Another optional embodiment of the present invention is shown, wherein the turning element 26B of the optical system 20B further includes a first turning element 261B and a second turning element 262B, wherein the first turning element 261B turns the incident light in the height direction (which can be defined as the Z-axis) to propagate along the X-axis, and the second turning element 262B turns the incident light propagating along the X-axis to the Y-axis, wherein the X-axis and Y-axis are perpendicular, and the Z-axis is perpendicular to the plane formed by the X and Y axes. The first turning element 261B, the first lens group 21a, the second lens group 21b, the second turning element 262B, the third lens group 21c, and the spectral chip 10 are arranged sequentially.

[0243] It is worth mentioning that, to ensure sufficient light intake, the lens size corresponding to the first lens group 21a is often the largest, which directly determines the height of the spectral analysis device. Therefore, the first lens group 21a can include at least one first lens 211a, wherein the at least one lens 211a is chamfered along a direction perpendicular to the Z-axis. The first lens 211a includes an effective area and an ineffective area. The height of the lens can be controlled by removing the ineffective area, thereby reducing the height of the spectral analysis device. It is understood that by chamfering, the height of the first lens 211a can be controlled to be less than or equal to 6mm, further ensuring that the height of the spectral analysis device is less than or equal to 6.5mm.

[0244] Preferably, the first lens element 211a is less than or equal to 5.5 mm, and the spectral analysis device is less than or equal to 5.9 mm. Furthermore, to compensate for the reduced light intake, the first lens element 211a of the first lens group 21a is made of glass, thereby reducing light loss and increasing the amount of light entering the lens.

[0245] Similar to the preferred embodiment described above, the spectral analysis device further includes at least one focusing mechanism 40B, wherein the focusing mechanism 40B is connected to the first lens group 21a, and the focusing mechanism 40B drives the first lens group 21a to achieve focusing. The optical system 20B also includes an aperture stop 23B, which is disposed at the front end of the first lens group 21a.

[0246] Example 14

[0247] Figure 20A and Figure 20B Further illustrating an exemplary structure of the spectral analysis device according to any of the preferred embodiments of the present invention, the spectral analysis device further includes a circuit board 70, to which the spectral chip 10 is electrically connected. The spectral chip 10 can be implemented as a chip-on-board (COB), a chip-scale package (CSP), or a flip-chip package. It is worth noting that the circuit board 70 can be, but is not limited to, a PCB, an F-PCB, or a ceramic substrate. When the spectral analysis device is used for imaging or video recording, the spectral chip 10 generates significant heat. Therefore, preferably, the circuit board 70 is implemented as a ceramic substrate. Furthermore, the spectral analysis device may also include a heat sink 60, which can be attached to the circuit board 70 or to the spectral chip 10 to improve heat dissipation of the spectral chip 10.

[0248] The spectral analysis device further includes a bracket 80, which is disposed on the circuit board 70. The optical system 20 is disposed on the bracket 80. The bracket 80 has a light-transmitting hole to allow light entering the optical system 20 to pass through and be received by the spectral chip 10. Preferably, the bracket 80 is formed from an opaque material such as plastic through a process such as injection molding, and then fixed to the circuit board 70 with an adhesive. Further, the bracket 80 can also be integrally formed on the circuit board 70. For example, the circuit board with the spectral chip 10 attached is placed in a mold, the mold is closed, molding material is injected, cured, and the mold is removed. An integrally formed molded body is formed on the circuit board and covers the non-imaging area of ​​the spectral chip, which can effectively improve the reliability of the spectral chip and the circuit board, and further reduce the size of the spectral analysis device to a certain extent.

[0249] For specific applications, the spectral analysis device may further include a filter (not shown in the figure), which is disposed between the optical system 20 and the spectral chip 10, located on the optical path of the spectral chip 10. The filter is used to filter incident light in unwanted wavelengths, thereby improving image quality. Preferably, the filter is attached to the support.

[0250] It is worth mentioning that, in this preferred embodiment of the present invention, the spectral chip 10 of the spectral analysis device can be driven. By driving the position of the spectral chip 10, the spectral analysis device can capture different spectral information at different positions of the spectral chip 10, and obtain a spectral image with higher spectral resolution through the acquired spectral information.

[0251] This embodiment provides a method for inverse design of structural unit 121 based on a deep neural network, including the following steps:

[0252] Step 101: Obtain the initial data of the structural unit 121 according to the structural unit 121 to be reverse designed.

[0253] In this invention, based on the structural unit 121 to be reverse-designed, firstly, a polygonal structural unit 121 with a structure closely similar to the structural unit 121 is generated. Then, based on this polygonal structural unit 121, a set of initial parameters, i.e., the initial data of the structural unit 121, is generated. This application can predict optical parameters based on any random polygonal structural unit 121, thereby optimizing the previous polygonal structural unit 121 based on the optical parameters obtained from each prediction, so that the structural data of the final polygonal structural unit 121 satisfies the target optical parameters.

[0254] Step 102: Input the initial data of the structural unit 121 into the trained optical parameter prediction model to obtain optical prediction parameters. The trained optical parameter prediction model is obtained by training a deep neural network with sample micro-nano data labeled with optical attribute parameters. The sample micro-nano data includes sample structural unit 121 data and sample micro-nano optical property data.

[0255] Step 103: Based on the evaluation function and optical target parameters, evaluate the optical prediction parameters. If the evaluation result does not meet the preset conditions, optimize the initial data of the structural unit 121 using the optimization algorithm and the evaluation result to obtain optimized data of the structural unit 121. Input the optimized data of the structural unit 121 into the trained optical parameter prediction model, and execute steps 102 to 103 again until the evaluation result of the optical prediction parameters obtained in the current iteration meets the preset conditions. Then, perform inverse design of the structural unit 121 based on the optimized data of the structural unit 121 corresponding to the optical prediction parameters in the current iteration.

[0256] In this embodiment, an optical parameter prediction model trained by a deep neural network predicts the corresponding electromagnetic response of the device (such as transmission spectrum and Q value) based on the initial parameters of structural unit 121. Then, the evaluation value (Figure of merit) of the device's electromagnetic response is calculated using an evaluation function and optical target parameters. In this embodiment, the evaluation function can be arbitrarily chosen according to the actual design goals, which include, but are not limited to: resonance at a preset frequency, increasing the resonant Q value, preset pass spectrum shape, preset electric field amplitude, and preset phase response. Then, through an optimization algorithm, based on the obtained evaluation value, a set of optimized parameters is generated for the initial parameters of structural unit 121. The process of neural network prediction, prediction result evaluation, and parameter optimization and updating continues until the parameters of structural unit 121 corresponding to a value close to the global optimum are obtained, so as to realize inverse design based on the parameters of structural unit 121.

[0257] The reverse design method for structural unit 121 based on deep neural networks provided in this embodiment uses deep neural networks to predict the electromagnetic response corresponding to the structural parameters. Specifically, it trains the neural network to predict the electromagnetic characteristics of structural unit 121 and obtains the optimal structural parameters that meet the target through iterative optimization based on preset optical target parameters. Since the calculation principle is based on prediction, the calculation time for the electromagnetic response is significantly reduced (up to 105 times faster) compared to directly calculating the electromagnetic response using simulation software, allowing for iterative optimization using optimization algorithms. The final design result, compared to forward design, not only obtains parameters that approach global optimum but also significantly shortens the design time and saves considerable human resources.

[0258] Based on the above embodiments, the trained optical parameter prediction model is obtained through the following steps:

[0259] According to the optical property parameters, each sample micro-nano data is labeled with a corresponding tag, and a training sample set is constructed based on the labeled sample micro-nano data and the corresponding sample optical parameters. The sample micro-nano data includes sample structural unit 121 data and sample micro-nano optical property data.

[0260] The training sample set is input into a deep neural network for training to obtain a trained optical parameter prediction model.

[0261] In this invention, the number of hidden layers in the deep neural network is approximately 3-20. The data dimension of the input layer varies depending on the actual structural complexity, roughly ranging from 3 to 10,000 dimensions. The output parameters, i.e., the optical prediction parameters obtained by the deep neural network, may include, but are not limited to, resonant wavelength, resonant Q value, pass spectrum, amplitude, and phase response, with the output parameter dimension roughly ranging from 1 to 1,000 dimensions. In this invention, the training and testing samples of the deep neural network can be calculated using commercial software such as FDTD, FEM, or Rsoft; alternatively, they can be calculated programmatically using the Fourier modal analysis method (also known as the strictly coupled mode analysis method).

[0262] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

Claims

1. A spectroscopic analysis apparatus, comprising: A spectral chip; the spectral chip is equipped with multiple transmission spectrum matrices; and An optical system, wherein the optical system is located in the optical path of the spectral chip; The optical system has a variable focal length, and this variable focal length corresponds to the plurality of transmission spectral matrices of the spectral chip. By adjusting the focal length of the optical system, the principal angle and / or the receiving cone angle of the incident light from the analyte reaching the spectral chip are adjusted, thereby configuring a specific transmission spectral matrix for the spectral chip to match the analyte. The spectral chip obtains at least one spectral information based on each focal length position of the optical system using a specific transmission spectral matrix, and integrates the spectral information to recover the spectral curve. Wherein, the principal light angle of the spectral chip represents the angle between the principal ray and the normal that are guided to the spectral chip, the principal ray represents the line connecting the point where the light signal emitted from the object under test is transmitted to the point where the corresponding structural pixel of the spectral chip is received, the normal represents the line perpendicular to the photosensitive surface of the spectral chip, and the light-receiving cone angle is the incident angle of the light signal to the structural pixel.

2. The spectral analysis apparatus according to claim 1, wherein the optical system comprises at least one lens assembly and at least one moving mechanism, wherein the at least one lens assembly is transversely connected to the at least one moving mechanism, and the at least one lens assembly is driven to move by the at least one moving mechanism to adjust the focal length of the optical system.

3. The spectral analysis apparatus according to claim 2, wherein the optical system further includes at least one deflector, wherein the deflector is disposed in the optical axis direction of the at least one lens assembly, and the deflector deflects the transmission direction of light incident on or exiting the at least one lens assembly.

4. The spectral analysis apparatus according to claim 1, wherein the optical system comprises at least one liquid lens assembly and at least one lens assembly, the liquid lens assembly and the lens assembly being arranged one after the other along the same optical axis, and the liquid lens assembly being capable of changing its own curvature.

5. The spectral analysis apparatus according to claim 1 further includes at least one data processing unit, wherein the spectral chip is electrically connected to the at least one data processing unit, and the data processing unit integrates and processes the spectral information to obtain a spectral curve.

6. The spectral analysis device according to claim 1, wherein the spectral chip records the zoom position of the optical system corresponding to each of the transmission spectrum matrices.

7. The spectral analysis apparatus according to any one of claims 2-4, wherein the spectral chip further comprises an image sensor and at least one filter structure disposed on the photosensitive side of the image sensor, wherein the filter structure is located above the image sensor, and the filter structure is a broadband filter structure in the frequency domain or wavelength domain.

8. The spectral analysis apparatus according to any one of claims 1-4, further comprising a light homogenizing component, wherein the light homogenizing component is located at the front end of the optical system.

9. A terminal device, comprising: A terminal device host; and The spectral analysis device according to any one of claims 1 to 8, wherein the spectral analysis device is electrically connected to the host of the terminal device.

10. An imaging method applied to the spectral analysis apparatus according to any one of claims 1-8, the method comprising: (501) The principal angle and / or the receiving cone angle of the incident light reaching the spectral chip are modulated by adjusting the focal length of an optical system, thereby configuring a specific transmission spectrum matrix for the spectral chip to be adapted to the test object, and obtaining multiple spectral information of the incident light based on the transmission spectrum matrices of the spectral chip corresponding to different focal lengths of the optical system. and (502) Integrate the spectral information and recover the spectral curve using the integrated spectral information. Wherein, the principal light angle of the spectral chip represents the angle between the principal ray and the normal that are guided to the spectral chip, the principal ray represents the line connecting the point where the light signal emitted from the object under test is transmitted to the point where the corresponding structural pixel of the spectral chip is received, the normal represents the line perpendicular to the photosensitive surface of the spectral chip, and the light-receiving cone angle is the incident angle of the light signal to the structural pixel.

11. The imaging method according to claim 10 further includes the following steps: matching a corresponding transmission spectrum matrix to the spectral chip based on the focal length of the optical system, and calculating the spectral information of the incident light based on the transmission spectrum matrix.

Images