Spectral information acquisition system

Abstract

The invention relates to the technical field of spectral imaging, in particular to a spectral information acquisition system. The spectral information acquisition system comprises a transmitting end used for transmitting mid-infrared light which is used for irradiating an object to be measured; the generation module is used for generating near-infrared light according to the mid-infrared light, and the time of the near-infrared light is synchronous with that of the mid-infrared light; the sum frequency module is used for performing sum frequency on the near-infrared light and the mid-infrared light passing through the object to be measured to obtain visible light; and the receiving end is used for receiving the visible light and generating spectral information of the to-be-measured object according to the visible light.

Description

Technical Field

[0001] The present disclosure relates to the technical field of spectral imaging, and in particular to a spectral information acquisition system. Background Art

[0002] Spectral imaging technology can simultaneously extract the spatial and spectral information of the object to be measured.

[0003] In the related art, infrared light can be used to illuminate the object to be measured, and an infrared light sensor can be used to receive the light carrying the spectral information of the object to be measured, and then an infrared spectral image of the object to be measured can be obtained in combination with a spectral reconstruction algorithm.

[0004] However, infrared light sensors have long imaging times and small pixel arrays, making it difficult to meet the requirements of high-speed, wide-angle, and multi-pixel infrared spectral imaging. In addition, infrared light sensors are expensive and difficult to miniaturize. Summary of the Invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a spectral information acquisition system that can solve the above problems.

[0006] According to a first aspect of an embodiment of the present disclosure, a spectral information acquisition system is provided, comprising: a transmitting end for emitting mid-infrared light, wherein the mid-infrared light is used to irradiate an object to be measured; a generating module for generating near-infrared light based on the mid-infrared light, wherein the near-infrared light is synchronized with the mid-infrared light in time; a sum frequency module for sum-frequency-generating the near-infrared light and the mid-infrared light passing through the object to be measured to obtain visible light; and a receiving end for receiving the visible light, and generating spectral information of the object to be measured based on the visible light.

[0007] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0008] The spectral information acquisition system proposed in this disclosure includes a transmitter, a generation module, a sum frequency module, and a receiver. The transmitter emits mid-infrared light to illuminate the object under test, the generation module generates synchronized near-infrared light, and the sum frequency module combines the mid-infrared light and near-infrared light after irradiating the object under test to generate visible light. The receiver can be a visible light sensor that receives the visible light and generates spectral information about the object under test.

[0009] The receiving end of the present invention can receive visible light, and the visible light sensor has the advantages of a larger field of view, more pixels, and a faster response rate compared to the infrared light sensor. In addition, since the manufacturing process of the visible light sensor is relatively mature, the manufacturing cost is also lower than that of the infrared light sensor. It is small in size, simple to operate, and conducive to equipment integration. In order to be able to use the visible light sensor as the receiving end, the spectral information acquisition system of the present invention can, after irradiating the object to be measured with mid-infrared light, perform sum frequency on the mid-infrared light carrying the spectral information of the object to be measured through a generation module and a sum frequency module to generate visible light, so that the receiving end of the spectral information acquisition system of the present invention can use a visible light sensor that can receive visible light.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0012] Figure 1 It is a structural diagram of a spectral information acquisition system according to an exemplary embodiment of the present disclosure.

[0013] Figure 2 It is a structural diagram of a spectral information acquisition system according to an exemplary embodiment of the present disclosure.

[0014] Figure 3 3 is a schematic structural diagram of a spectrum modulator according to an exemplary embodiment of the present disclosure.

[0015] Figure 4 This is a schematic diagram of generating a metasurface unit according to an exemplary embodiment of the present disclosure.

[0016] Figure 5 It is a schematic diagram of a visible light coding reconstruction neural network structure according to an exemplary embodiment of the present disclosure.

[0017] Figure 6 1 is a diagram showing the relationship between mid-infrared up-conversion and visible down-conversion according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0019] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0021] Spectral imaging technology can extract both spatial and spectral information about the object being measured. Compared to the three-channel data of traditional imaging, spectral imaging technology can reach dozens or even hundreds of channels, significantly improving the information utilization of the object being measured, thereby extracting more accurate and multi-dimensional features of the object being measured. By leveraging the high differences between the object being measured and the background in specific spectral bands, the object being measured, which is difficult to distinguish from the background using traditional imaging technology, can be easily extracted. This enables detection and identification of the object under test in more complex backgrounds and multi-target conditions, facilitating applications in scenarios such as soil monitoring, target identification, and material detection.

[0022] Traditional spectral imaging systems rely on scanning in the temporal or spatial dimensions to acquire spectral information. These typically involve mechanical delay lines or push-broom methods, which are time-consuming, incapable of observing high-speed moving objects, and have difficulty observing instantaneous spectral changes. Their relatively large size also hinders miniaturization and integration. Temporal or spatial scanning systems can also cause data distortion due to scene changes during the integration period, severely limiting the ability to monitor dynamic objects.

[0023] In the related art, infrared light can be used to illuminate the object to be measured, and an infrared light sensor can be used to receive the light carrying the spectral information of the object to be measured, and then an infrared spectral image of the object to be measured can be obtained in combination with a spectral reconstruction algorithm.

[0024] However, most current infrared light sensors use cooled mercury cadmium telluride or indium antimonide detectors, which have complex manufacturing processes, high costs, slow response times, and low frame rates, and cannot meet the needs of wide-viewing-angle, large-area, and high-speed imaging.

[0025] In order to solve the above technical problems, the present disclosure proposes a spectral information acquisition system.

[0026] Figure 1 This is a schematic diagram of a spectral information acquisition system according to an embodiment of the present disclosure. The spectral information acquisition system can be used to perform spectral imaging on an object to be measured to obtain spectral information of the object to be measured.

[0027] like Figure 1 As shown, the spectrum information acquisition system includes:

[0028] An emitting end, configured to emit mid-infrared light, wherein the mid-infrared light is used to illuminate an object to be measured;

[0029] a generating module, configured to generate near-infrared light according to the mid-infrared light, wherein the near-infrared light is synchronized with the mid-infrared light in time;

[0030] a sum frequency module, configured to perform sum frequency processing on the near-infrared light and the mid-infrared light passing through the object to be measured to obtain visible light;

[0031] The receiving end is used to receive the visible light and generate spectrum information of the object to be measured according to the visible light.

[0032] In some embodiments, the transmitting end is used to emit mid-infrared light, and the mid-infrared light is used to illuminate the object to be measured.

[0033] Mid-infrared (MIR) light can be electromagnetic waves with a wavelength range of 2.5 to 25 μm. By illuminating an object with MIR light, the structure of the object can be identified based on its absorption characteristics.

[0034] The mid-infrared light after irradiating the object to be measured will carry the spectral information of the object to be measured. After being received and imaged by the receiving end, the spectral information of the object to be measured can be determined based on algorithm processing.

[0035] In some embodiments, the generating module may be configured to generate near infrared light according to the mid infrared light, wherein the near infrared light is synchronized with the mid infrared light in time.

[0036] Near-infrared (NIR) light can be an electromagnetic wave with a wavelength range of 780 to 2500 nm. The generation module can generate NIR light corresponding to the mid-infrared light at the transmitter, so that the NIR light can be combined with the mid-infrared light to generate visible light.

[0037] To ensure that near-infrared light can be sum-frequency-modulated with mid-infrared light, the near-infrared light generated by the generation module also needs to be synchronized with the mid-infrared light so that the two laser pulses overlap precisely in the time dimension and the pulse peaks can reach the sum-frequency module at the same time to ensure that the sum-frequency effect can generate visible light and ensure sufficiently high conversion efficiency.

[0038] Since the optical path of the near-infrared light and the mid-infrared light reaching the sum frequency module is fixed and the time required is also fixed after the near-infrared light and the mid-infrared light are emitted, the generation module needs to generate near-infrared light that can be synchronized with the mid-infrared light when generating the near-infrared light.

[0039] In some embodiments, a sum frequency module is used to perform sum frequency on the near-infrared light and the mid-infrared light passing through the object to be measured to obtain visible light.

[0040] Sum-Frequency Generation (SFG) is a second-order nonlinear optical process that allows two beams of light with different frequencies to interact, generating a new beam with the sum of their frequencies. In this paper, the SFG module uses this effect to generate visible light by summing near-infrared light with mid-infrared light passing through the object under test.

[0041] After irradiating the object to be measured, mid-infrared light can carry the spectral information of the object to be measured. Visible light is obtained by summing the mid-infrared light passing through the object to be measured. Therefore, the spectral information of the object to be measured carried in the mid-infrared light can be retained on the visible light. Then, the spectral information of the object to be measured can be determined by processing the visible light.

[0042] Based on the generation module and the sum frequency module, the present disclosure realizes the conversion of mid-infrared light carrying the spectral information of the object to be measured into visible light carrying the spectral information of the object to be measured, so that the receiving end does not need to be limited to the infrared light sensor.

[0043] In some embodiments, the receiving end is used to receive the visible light and generate spectrum information of the object to be measured based on the visible light.

[0044] The receiving end may include a visible light sensor for receiving visible light and determining spectral information of the object to be measured based on the received visible light.

[0045] Compared to infrared sensors, visible light sensors are more mature, less expensive, and feature high pixel counts, small size, and ease of integration. For example, visible light sensors can be used in silicon-based visible light cameras. Furthermore, due to their high frame rates, reaching several kilohertz, and fast response rates, visible light sensors can significantly improve the monitoring of transient states and high-speed processes. For example, they can better monitor chemical reactions in biological samples and diffusion in solutions.

[0046] Compared with the related art that uses infrared light sensors for receiving, the spectral information acquisition system proposed in the present disclosure uses a receiving end that can receive visible light. It is more mature and has a low production cost. It can have more than one million pixels, a fast response rate, a small size, and is easy to integrate.

[0047] The reason why the present disclosure can use a visible light receiving end instead of the infrared light sensor in the related art is that the present disclosure generates near-infrared light that is time-synchronized with the mid-infrared light through a generation module, and then performs sum frequency addition on the mid-infrared light and the near-infrared light through the sum frequency effect of the sum frequency module to convert them into visible light, which is then received by the receiving end to determine spectral information based on the visible light.

[0048] Figure 2 It is a structural diagram of a spectral information acquisition system according to an embodiment of the present disclosure.

[0049] In some embodiments, the system further includes: a first dichroic mirror 101, for dividing the mid-infrared light emitted by the transmitting end into a first mid-infrared light in a first direction and a second mid-infrared light in a second direction; wherein the first mid-infrared light irradiates the object to be measured 104, and the second mid-infrared light is input into the generation module; the generation module is used to generate the near-infrared light based on the second mid-infrared light.

[0050] like Figure 2 As shown, the main optical axis direction of the spectrum information acquisition system is the first direction, and the second direction is different from the first direction.

[0051] The first dichroic mirror 101 can be used to split or combine light.

[0052] As described above, the generation module needs to generate near-infrared light that can be synchronized with the mid-infrared light in time. In the present disclosure, the first dichroic mirror 101 can divide the mid-infrared light emitted by the transmitting end into two beams of light in different directions: a first mid-infrared light and a second mid-infrared light without changing the frequency of the mid-infrared light.

[0053] The first mid-infrared light is transmitted along the main optical axis to illuminate the object 104 to be measured, carrying the spectral information of the object. The second mid-infrared light is transmitted along a second direction and input into the generation module, so that the generation module can generate near-infrared light synchronized with the first infrared light based on the second mid-infrared light synchronized with the first mid-infrared light. This near-infrared light is subsequently used for frequency summing with the first mid-infrared light.

[0054] In other embodiments, after determining the mid-infrared light, the near-infrared light can be determined based on the mid-infrared light, and the phase of the near-infrared light generated by the generation module can be adjusted to synchronize the time of the near-infrared light with the mid-infrared light.

[0055] The phase and frequency of the near-infrared light generated by the generation module are adjustable. After determining the phase, frequency and other information of the mid-infrared light, the near-infrared light that can be synchronized with the mid-infrared light can be directly generated.

[0056] Alternatively, near-infrared light is first generated based on mid-infrared light, and after discovering that the time is not synchronized, the phase of the near-infrared light is adjusted until the near-infrared light is synchronized with the mid-infrared light.

[0057] like Figure 2 As shown, in some embodiments, the generation module includes: a photoelectric conversion unit 114, a pulse generator 115 and a laser emitter 116; wherein the photoelectric conversion unit 114 is used to receive the mid-infrared light and convert the mid-infrared light into an electrical signal; the pulse generator 115 is used to generate a synchronous pulse signal according to the electrical signal; and the laser emitter 116 is used to emit the near-infrared light according to the synchronous pulse signal.

[0058] After being split by the first dichroic mirror, the second mid-infrared light can be input to a photoelectric converter 114 (e.g., a photodiode). Upon receiving the second mid-infrared light, the photoelectric converter 114 can generate a synchronization electrical signal and input the generated synchronization electrical signal to a pulse generator 115 to generate a synchronization pulse. The synchronization pulse then controls the laser emitter 116 to emit near-infrared light synchronized with the first mid-infrared light.

[0059] For example, the laser emitter 116 may be a 1064 nm laser, and the wavelength of the emitted near-infrared light is 1064 nm.

[0060] The photoelectric conversion unit 114 in the generation module can be a pre-calibrated and synchronized photodetector (PD), which is excited by a second mid-infrared light continuous pulse laser in the same beam as the first mid-infrared light to ensure that the near-infrared light energy generated by the laser emitter 116 and the mid-infrared light carrying the spectral information of the object to be measured have the maximum sum frequency efficiency at the sum frequency module, thereby increasing the intensity of the generated visible light and improving the signal-to-noise ratio of the system.

[0061] In some embodiments, the system further includes: a first refraction module, configured to adjust the irradiation range of the mid-infrared light through optical refraction, so that the mid-infrared light can irradiate and cover the object to be measured.

[0062] When the object to be measured is large, the first refraction module can enlarge the irradiation range of the mid-infrared light so that the mid-infrared light can completely irradiate and cover the object to be measured, thereby obtaining the complete spectral information of the object to be measured in one irradiation, without the need to irradiate multiple areas of the object to be measured separately, obtaining the spectral information of the corresponding area each time, and obtaining the complete spectral information of the object to be measured in multiple times.

[0063] When the object to be measured is small, the first refraction module can narrow the irradiation range of the mid-infrared light. While ensuring that the mid-infrared light can fully irradiate and cover the object to be measured, it increases the intensity of the mid-infrared light within the unit irradiation range, which is conducive to reducing noise interference and obtaining clearer and more specific spectral information.

[0064] like Figure 2 As shown, in some embodiments, the first refraction module may include a first mid-infrared lens 102 and a second mid-infrared lens 103 .

[0065] exist Figure 2 In the example shown, based on the first mid-infrared lens 102 and the second mid-infrared lens 103, the irradiation range of the first mid-infrared light obtained after being split by the first dichroic mirror 101 can be amplified, and after amplification, the first mid-infrared light is still kept parallel to the main optical axis to illuminate the object to be measured 104.

[0066] After the first mid-infrared light passes through the first refraction module, the light field area increases and the irradiation range is magnified. After passing through the object to be measured 104 , the light field changes, thereby carrying the spectral information of the object to be measured.

[0067] In some embodiments, the system further includes: a third mid-infrared lens 105 for focusing the mid-infrared light passing through the object to be measured, wherein the sum frequency module is located at the focus of the third mid-infrared lens 105 .

[0068] The third mid-infrared lens 105 can focus the mid-infrared light passing through the object to be measured onto the sum frequency module, thereby generating a sum frequency effect in the sum frequency module.

[0069] In some embodiments, the sum frequency module includes: a second dichroic mirror 106 and a nonlinear crystal 107; wherein the second dichroic mirror 106 is used to combine the mid-infrared light and the near-infrared light in different directions, and after combining, the visible light is generated through the sum frequency effect of the nonlinear crystal 107.

[0070] Second dichroic mirror 106 is used to combine mid-infrared light along the main optical axis with near-infrared light in a non-main optical axis direction. The second dichroic mirror can redirect the near-infrared light so that it also propagates along the main optical axis, thereby combining it with the mid-infrared light that has passed through the second dichroic mirror.

[0071] The nonlinear crystal 107 may include a CPPLN (Chirped Periodically Poled LiNbO3) crystal. If the system includes a third mid-infrared lens 105, the nonlinear crystal 107 may be disposed at the image focus of the third mid-infrared lens 105, such as Figure 2 shown.

[0072] After the near-infrared light is combined with the mid-infrared light carrying the spectral information of the object to be measured at the second dichroic mirror 106 , a nonlinear sum frequency effect may occur in the nonlinear crystal 107 , thereby generating visible light.

[0073] In some embodiments, the system further includes: a visible lens 108, configured to convert the visible light generated by the sum frequency module after sum frequency conversion into visible light parallel to the main optical axis.

[0074] like Figure 2 As shown, the nonlinear crystal 107 can be set at the object focus of the visible lens 108, so that the visible light generated by the sum frequency effect in the nonlinear crystal 107 can be collimated by the visible lens 108 and become visible light emitted parallel to the main optical axis.

[0075] The combination of the visible lens 108 and the third mid-infrared lens 105 can correct the light beam and ensure that the size of the visible light beam is compatible with the aperture of the receiving end.

[0076] In some embodiments, the system further includes: an optical filter 109 , which is disposed between the sum frequency module and the receiving end and is configured to filter out light in the mid-infrared band in the visible light.

[0077] After passing through the sum frequency module, the mid-infrared and near-infrared light are summed to generate visible light. However, to avoid excess noise and ensure the accuracy of the visible light used for spectral imaging, a filter 109 (e.g., a low-pass filter) can be used to filter out the mid-infrared light before it is irradiated to the receiving end. Filtering out infrared light mixed with visible light can also reduce the beam intensity, preventing sensor saturation at the receiving end.

[0078] In some embodiments, the receiving end may be a visible light camera 113 .

[0079] The visible light camera 113 may be configured in an object-side telecentric manner, and the visible light beam may be imaged on the sensor after passing through the camera window 110 .

[0080] In some embodiments, the receiving end includes: a spectral modulator 111, a visible light sensor 112, and a processing module; wherein the spectral modulator 111 is used to spectrally modulate the visible light passing through the object to be measured; the visible light sensor 112 is used to receive the spectrally modulated visible light to generate a spectral image; the processing module is used to generate spectral information of the object to be measured based on the spectral modulator 111 and the spectral image.

[0081] After the visible light carrying the spectral information of the object to be measured passes through the window 110, it can be received by the visible light sensor 112 (for example, it can be a visible camera focal plane array), and a layer of spectral modulator 111 is attached in front of the visible light sensor 112. Therefore, the visible light received by the visible light sensor 112 is the visible light that has passed through the spectral modulator 111, and the spectral modulator 111 can spectrally modulate the visible light, so that based on the received visible light and the spectral modulator 111, the spectral information of the object to be measured carried in the visible light can be determined.

[0082] Each subunit of the spectral modulator 111 corresponds to a pixel on the visible light sensor 112. The subunits of the spectral modulator 111 can have different spectral response characteristics, so that they can reflect the spectral characteristics of visible light carrying spectral information in different aspects, so that multiple spectral information of the object to be measured can be reflected through one spectral imaging.

[0083] The spectrum modulator 111 can also be tightly fitted with the visible light sensor 112 before being packaged to the receiving end to ensure that the gap between the spectrum modulator 111 and the visible light sensor 112 is small enough to reduce crosstalk between adjacent subunits of the spectrum modulator 111.

[0084] The receiving end also includes a processing module. The processing module can convert the image output by the visible light sensor 112 into a two-dimensional matrix, and divide the two-dimensional matrix into blocks through a pre-divided hypersurface, and can convert the value of each hypersurface into an observation vector Y i And input the visible observation vector into the pre-trained convolutional neural network to obtain the up-converted visible band spectrum X i Then the visible band spectrum X can be converted into i Down-converted to mid-infrared spectrum M i Finally, the mid-infrared spectrum vector M corresponding to the mid-infrared band spectrum is obtained i Rearranged in wavelength order to form a mid-infrared spectrum cube.

[0085] Spectral modulator 111 can modulate the spectrum of a visible light beam, eliminating the need for complex time-space spectrometers and scanning functions. This allows for full collection of spectral imaging three-dimensional cube data within a single exposure, resulting in a compact, lightweight mid-infrared spectrometer while also enabling high-speed reading of spectral images. Furthermore, because the device does not require sophisticated scanning mechanisms, the mid-infrared imaging spectrometer is suitable for a variety of applications and environments, significantly improving its stability and reliability in harsh environments such as high temperature and humidity, and ensuring the accuracy of the device's reading and processing of data.

[0086] The spectrum modulator 111 will be introduced in detail below.

[0087] Figure 3 It is a structural schematic diagram of a spectrum modulator according to an embodiment of the present disclosure.

[0088] like Figure 3 As shown, in some embodiments, the spectral modulator 111 includes an arrayed metasurface unit 1112, which contains a plurality of micro-nanostructures 1113; wherein the patterns of the plurality of micro-nanostructures 1113 in a metasurface unit 1112 are the same, and different metasurface units have different spectral transmittances.

[0089] The spectrum modulator 111 can be composed of arrayed and periodically distributed metasurface units 1112. Each metasurface unit 1112 corresponds to a pixel of the visible light sensor 112. The metasurface unit 1112 is composed of a micro-nanostructure 1113 with sub-wavelength periodicity, such as Figure 3 As shown, a metasurface unit 1112 can contain nine micro-nanostructures 1113 with the same pattern. Each micro-nanostructure 1113 is composed of a randomly generated pattern with C4 symmetry. Different metasurface units 1112 can have different spectral transmittances due to the different micro-nanostructures 1113 they contain, thereby producing different spectral modulation effects on visible light carrying spectral information of the object to be measured.

[0090] In some embodiments, a first number of metasurface units 1112 arranged in an array constitute a metasurface 1111, and the spectrum modulator is composed of multiple metasurfaces.

[0091] The spectral modulator 111 may be composed of a periodically distributed metasurface 1111 .

[0092] In some embodiments, the arrangement of the metasurface units 1112 on the spectral modulator is determined by optimizing at least one of the following evaluations: randomness evaluation; correlation evaluation; spectral feature density evaluation; and transmission loss evaluation.

[0093] The micro-nano structure 1113 is a randomly generated pattern, and thus based on the differences in the micro-nano structure 1113, 4096 kinds of metasurface structures can be randomly generated, and the most suitable metasurface unit can be screened out through evaluation and optimization.

[0094] The evaluation optimization may include at least one of randomness evaluation, correlation evaluation, spectral feature density evaluation, and transmission loss evaluation.

[0095] In some embodiments, the evaluation optimization may be a comprehensive evaluation of randomness evaluation, correlation evaluation, spectral feature density evaluation, and transmission loss evaluation.

[0096] Compared with the related art which determines the spectrum modulator 111 only based on correlation evaluation, the present disclosure combines comprehensive evaluation of randomness evaluation, correlation evaluation, spectrum feature density evaluation and transmission loss evaluation, so that the generated spectrum modulator 111 can better reflect the spectrum characteristics.

[0097] Regarding randomized evaluation:

[0098] For example, the transmission spectra (metasurface units 1112) that make up the transmission matrix (spectral modulator 111) should be as random as possible to provide spectral information covering the entire observation frequency band. In other words, the higher the randomness between metasurface units 1112, the higher the priority of the metasurface structure.

[0099] The entropy of the transmission matrix can be obtained by summing up the transmission spectra. The greater the entropy, the stronger the randomness of the transmission spectrum. In order to quantify the evaluation index, the autocorrelation function of the metasurface unit 1112 is calculated respectively. The autocorrelation function is a measure of the correlation degree of the response of the metasurface unit 1112 to different frequency bands. In order to have different responses to different frequency bands as much as possible, the autocorrelation function of the metasurface unit 1112 should be close to the delta function. The correlation coefficient between the transmittance function after a certain time delay and the transmittance function with zero time delay is defined as the autocorrelation coefficient of the metasurface unit 1112, which is defined as follows:

[0100]

[0101] Where E represents the mathematical expectation and D represents the variance. The autocorrelation coefficient is set as the optimization target α, the average autocorrelation coefficient of the designed metasurface 1111 is calculated, and the optimization method (particle swarm optimization, simulated annealing algorithm, etc.) is used for iterative optimization to obtain the maximum average autocorrelation coefficient α of the designed encoder. max and α min , iterate multiple times and take α max and α min The respective averages are used to reduce the iteration error.

[0102] Regarding relevance assessment:

[0103] For example, the correlation of the transmission matrix should be as low as possible to reduce the interference of redundant information on valid information. The correlation can be calculated by the correlation coefficient matrix, and the smaller the sum of the absolute values ​​of the matrix elements, the better. Similarly, the cross-correlation coefficient of the transmittance function is a measure of the response correlation between different metasurface units 1112. In order to obtain more spectral feature information as much as possible, the difference in the transmittance function between different metasurface units 1112 should be large enough, and the average cross-correlation coefficient between metasurface units should not exceed 0.3. The cross-correlation coefficient is defined as follows:

[0104]

[0105] Where X and Y are the transmission matrices of different metasurface units 1112. The mutual correlation coefficient is set as the optimization target β, and optimization methods such as particle swarm optimization are used to optimize, and the average value is taken after multiple iterations to obtain β max and β min .

[0106] Regarding the evaluation of spectral feature density:

[0107] For example, the width and number of spectral features should be appropriate. Excessively wide or too few characteristic peaks will lead to the loss of fine spectral information. Conversely, this will increase the difficulty of experimental testing or cause overfitting of the spectral reconstruction algorithm. In order to improve the stability of the reconstruction algorithm, the condition number of the metasurface 1111 transfer matrix is ​​calculated. The smaller the condition number, the smaller the output change caused by a small change in the input value of the transfer matrix, that is, the more stable the reconstruction result of the neural network. The condition number is defined as follows:

[0108] γ(A)=||A||·||A -1 ||

[0109] Where ||·|| is the norm of the matrix. The optimization target can be set as the condition number γ of the hypersurface 1111, and γ can be obtained by iterative calculation. max and γ min γ measures the anti-noise capability of the metasurface 1111. The smaller its value is, the higher the signal-to-noise ratio of the encoder group is.

[0110] Regarding transmission loss assessment:

[0111] For example, the energy transfer efficiency of the transmission spectrum should be as high as possible to ensure a good signal-to-noise ratio during detection. This can be evaluated by calculating the average transmission loss over the entire observed spectrum. The average loss of the metasurface 1111 is defined as follows:

[0112]

[0113] Among them, T iis the average transmittance of the ith metasurface unit 1112. Setting the average loss coefficient Γ of the metasurface 1111 as the optimization target, we get Γ max and Γ min .

[0114] In some embodiments, the above four evaluations may be comprehensively considered, and the above four optimization objectives may be normalized and summed up to serve as the final optimization objective Opt.

[0115] The definition of Opt is as follows:

[0116]

[0117] Taking Opt as the optimization target, we use optimization methods such as simulated annealing to optimize and get the minimum Opt min and the corresponding metasurface 1111, thereby ensuring the accuracy and stability of the spectral results reconstructed using the metasurface-neural network.

[0118] It should be noted that the spectral modulator 111 in the related art only considers that the transmission matrix must have low correlation. However, the present disclosure can comprehensively evaluate the spectral modulator 111 and metasurface 1111 based on low correlation, high randomness, appropriate spectral feature density, and low transmission loss, thereby determining the most appropriate metasurface 1111 and ensuring the accuracy and stability of the reconstructed spectral results.

[0119] Figure 4 Schematic diagram of generating a metasurface unit according to an embodiment of the present disclosure.

[0120] The metasurface unit 1112 has C4, or four-fold rotational symmetry, which ensures that the transmittance function of the metasurface unit 1112 is polarization-independent.

[0121] like Figure 4 As shown, the micro-nano structure 1113 can be divided into 13 grids, each grid length is 0.2um, and the period of a single structure is 2.6um. The grids within one eighth are randomly assigned values ​​of 0 and 1, where 0 represents no structure and 1 represents structure. The other grids are then filled in through mirroring, rotation, and other operations. There are a total of 2 28 This combination method greatly increases the number of structures of the metasurface unit 1112.

[0122] In some embodiments, the system further includes: an adjustment module, which is used to adjust the number of metasurface units 1112 that constitute the metasurface 1111.

[0123] The number of pixels in the visible light sensor 112 is W×H, so the number of metasurface units 1112 in the spectral modulator 111 is also W×H. m*n metasurface units 1112 can be combined into a metasurface 1111, resulting in a total of (W / m)×(H / n) metasurfaces 1111. The number of pixels in the image of the mid-infrared up-conversion spectral modulation imaging system is (W / m)×(H / n), and each metasurface 1111 corresponds to a spatial object point of the sample.

[0124] In order to cope with different application scenarios and targets to be measured, different sizes m and n of the metasurface 1111 can be set, that is, the number of metasurface units 1112 that make up the metasurface 1111 can be adjusted, thereby changing the number of metasurfaces 1111 on the spectral modulator 111, and then controlling the number of pixels and spectral resolution of spectral imaging.

[0125] For example, for an object to be measured that is large in size and has a relatively simple material composition, m and n can be appropriately reduced to increase the spatial resolution of the target and sacrifice some spectral resolution; conversely, for an object to be measured that is small in size and has a complex material composition, m and n can be appropriately increased to increase the spectral resolution and sacrifice some spatial resolution.

[0126] In some embodiments, it is necessary to train corresponding neural networks for metasurfaces 1111 of different sizes.

[0127] The size of the metasurface 1111 can be adjusted by changing the sizes of m and n. However, different m and n require training different neural networks to adapt to different metasurface 1111 configurations.

[0128] Assume m*n=p, that is, the number of metasurface units 1112 contained in the metasurface 1111 is p. After the visible light beam passes through the metasurface 1111, the corresponding y=Ax, where y is the light intensity received by the visible light sensor 112, A is the transmission matrix (determined by the spectrum modulator 111), and x is the input spectrum (determined by the spectral information of the object to be measured). The matrix expression of the measurement equation is:

[0129]

[0130] Where k is the number of spectral channels for reconstructing the spectrum, p is the number of modulation units, and A ij represents the transmittance of the i-th modulation unit to the j-th spectral channel, y i represents the light intensity response of the i-th modulation unit, x j represents the intensity of the jth spectral channel. Typically, A is not a square matrix, and p is much smaller than k, making the equation underdetermined. In this case, it is impossible to find the exact solution to x using rigorous mathematical methods. However, based on the principles of compressed sensing, an approximate solution to x can be obtained using dictionary learning or neural network methods.

[0131] Figure 5 It is a schematic diagram of a visible light coding reconstruction neural network structure according to an embodiment of the present disclosure.

[0132] like Figure 5 As shown, the image output by the visible light sensor 112 is divided into a series of m*n two-dimensional matrices according to the division of the metasurface 1111, which are normalized and passed into the port of the neural network as input. Subsequently, after the expansion, pooling, and convolution operations of the multi-layer neural network, 1*k spectral data are obtained at the output port of the neural network. The above operations are repeated until all metasurface spectra are reconstructed. Finally, the obtained spectral data are divided according to the spectral channel, that is, the sample image under each spectral channel is obtained.

[0133] It should be noted that different metasurface 1111 configurations determine different neural network parameters, that is, the neural networks required to reconstruct the spectra of metasurfaces 1111 at different locations are not necessarily the same. Different neural network structures need to be set according to the transmission matrix of the metasurface 1111, and different activation functions and optimization algorithms are used to iteratively optimize the network. For example, the mean square error (MSE) between the spectra reconstructed by neural networks with different parameters and the spectra obtained using an infrared Fourier spectrometer is calculated respectively, and the network activation function and optimization algorithm corresponding to the minimum MSE are used as the network architecture to achieve network optimization. The size of the reconstructed and rearranged spectral image is (W / m)×(H / n)*k, where (W / m) is the number of horizontal pixels of the image, (H / n) is the number of vertical pixels of the image, and k is the number of spectral channels.

[0134] The spectral image obtained by neural network reconstruction is a visible spectrum image after up-conversion by the CPPLN crystal, which is not the mid-infrared spectrum image we need. Therefore, it is necessary to use the sum-frequency relationship of the nonlinear crystal to down-convert the visible spectrum into the mid-infrared spectrum.

[0135] Figure 6 1 is a diagram showing the relationship between mid-infrared up-conversion and visible down-conversion according to an embodiment of the present disclosure.

[0136] like Figure 6 As shown, the spectrum range of the original mid-infrared beam is 2300nm ~ 4100nm, and it interacts nonlinearly with the near-infrared light (such as 1064nm laser) in the CPPLN crystal. Using the sum frequency formula:

[0137] ω3=ω1+ω2

[0138] It can be calculated that the spectrum range of the visible light beam after upconversion is 728nm~845nm. After reconstructing the 728~845nm spectrum image using a neural network, the inverse of the sum frequency formula is obtained:

[0139] ω1=ω3-ω2

[0140] Where ω3 is the frequency of visible light, ω1 is the frequency of mid-infrared light, and ω2 is the frequency of near-infrared light (e.g., 1064nm laser). By reconstructing ω3 and the given parameters of ω2, the distribution of ω1 can be obtained, i.e., the mid-infrared spectral image.

[0141] Corresponding to the embodiments of the spectrum information acquisition system disclosed herein, the present disclosure also provides embodiments of corresponding spectrum information acquisition methods, which are executed by the receiving end in any of the above embodiments, specifically by a processing module in the receiving end.

[0142] The method for obtaining spectral information is as follows:

[0143] Get the two-dimensional measurement value received by the receiver; normalize the pixel values ​​in each hypersurface separately, and splice them into a one-dimensional vector row by row to obtain the visible measurement vector Y i , where i is the metasurface number, ranging from 1 to m*n.

[0144] Set the visible measurement vector Y i Input into the pre-trained neural network corresponding to the superpixel to obtain the output visible spectrum X i .

[0145] According to the sum-frequency relationship, the output visible spectrum X i Convert to mid-infrared spectrum M i .

[0146] The mid-infrared spectrum M i The mid-infrared three-dimensional spectral image R is obtained by splicing in spatial order. ajλ , where a is the number of pixel rows, j is the number of pixel columns, and λ is the number of spectral channels.

[0147] Then, through the mid-infrared three-dimensional spectral image R ajλ Determine the spectral information of the object to be measured.

[0148] The spectral information acquisition system proposed in the present disclosure can convert mid-infrared light into visible light, and then perform spectral imaging based on the receiving end. While acquiring mid-infrared spectral images, the imaging frame rate and image spatial resolution are improved. At the same time, the receiving end is a visible light sensor, which replaces the mid-infrared sensor in related technologies, which can reduce development costs. The high-speed dynamic measurement of the spectral imaging system can monitor the rapid changes of the object to be tested and obtain details of the instantaneous changes of the object to be tested. It is very suitable for the application of samples in scenarios such as biological testing and chemical reactions. At the same time, the adjustable encoder configuration means that the system has more flexible and changeable detection modes to adapt to objects of different sizes and compositions to be tested.

[0149] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0150] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0151] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0152] The above is a detailed introduction to the methods and devices provided in the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the methods and core ideas of the present disclosure. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present disclosure.

Claims

1. A spectral information acquisition system, characterized in that: The system comprises: An emitting end, configured to emit mid-infrared light, wherein the mid-infrared light is used to illuminate an object to be measured; a generating module, configured to generate near-infrared light according to the mid-infrared light, wherein the near-infrared light is synchronized with the mid-infrared light in time; a sum frequency module, configured to perform sum frequency processing on the near-infrared light and the mid-infrared light passing through the object to be measured to obtain visible light; The receiving end is used to receive the visible light and generate spectrum information of the object to be measured according to the visible light.

2. The system according to claim 1, wherein: The system further comprises: a first dichroic mirror, configured to split the mid-infrared light emitted by the transmitting end into a first mid-infrared light in a first direction and a second mid-infrared light in a second direction; wherein the first mid-infrared light is used to illuminate the object to be measured, and the second mid-infrared light is input into the generating module; The generating module is configured to generate the near-infrared light according to the second mid-infrared light.

3. The system according to claim 1, wherein: The sum frequency module includes: a second dichroic mirror, and a nonlinear crystal; The second dichroic mirror is used to combine the mid-infrared light and the near-infrared light in different directions, and after combining, generate the visible light through the sum frequency effect of the nonlinear crystal.

4. The system according to claim 1, wherein: The system further comprises: A filter is provided between the sum frequency module and the receiving end, and is used to filter out light in the mid-infrared band in the visible light.

5. The system according to claim 1, wherein: The receiving end includes: Spectral modulator, visible light sensor, processing module; Wherein, the spectrum modulator is used to perform spectrum modulation on the visible light passing through the object to be measured; The visible light sensor is used to receive spectrally modulated visible light to generate a spectral image; The processing module is used to generate spectral information of the object to be measured according to the spectral modulator and the spectral image.

6. The system according to claim 5, characterized in that The spectral modulator includes arrayed metasurface units, each of which contains multiple micro-nano structures; wherein the patterns of the multiple micro-nano structures in one metasurface unit are the same, and different metasurface units have different spectral transmittances.

7. The system according to claim 6, characterized in that The arrangement of the metasurface units on the spectrum modulator is determined by evaluating and optimizing at least one of the following: Randomness assessment; correlation assessment; spectral feature density assessment; transmission loss assessment.

8. The system according to claim 6, wherein: A first number of metasurface units arranged in an array constitute a metasurface, and the spectrum modulator is composed of multiple metasurfaces.

9. The system according to claim 8, characterized in that The system further comprises: An adjustment module is used to adjust the number of metasurface units that constitute the metasurface.

10. The system according to claim 1, wherein: The generation module includes: Photoelectric conversion unit, pulse generator and laser emitter; wherein, The photoelectric conversion unit is used to receive the mid-infrared light and convert the mid-infrared light into an electrical signal; The pulse generator is used to generate a synchronous pulse signal according to the electrical signal; The laser emitter is used to emit the near-infrared light according to the synchronization pulse signal.

Images