Adjustable coding and deep learning reconstruction inversion method and system for infrared spectral detection chip

Abstract

The application discloses an adjustable coding and deep learning reconstruction inversion method for an infrared spectrum detection chip, and comprises the following steps: constructing a spectral response matrix of a target infrared spectrum detector; forming a training set by combining a real spectrum curve and photocurrent data under different voltage channels; constructing an initial model based on the spectral response matrix and a deep learning network framework, wherein the initial model comprises a data preprocessing module, a feature extraction module, a feature fusion module and a prediction module; training the initial model by using the training set to obtain a reconstruction model used for reconstructing a spectrum curve; and inputting photocurrent data collected by the target infrared spectrum detector into the reconstruction model to output a spectrum curve corresponding to the photocurrent data. The application further provides an adjustable coding and deep learning reconstruction inversion system. The method provided by the application can significantly improve the resolution and noise resistance of spectrum reconstruction of an infrared spectrum detector.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of optoelectronic information processing, and particularly relates to an adjustable coding and deep learning reconstruction inversion method and system for an infrared spectrum detection chip. BACKGROUND

[0002] Traditional infrared spectrum reconstruction technology mainly faces three technical bottlenecks: first, although the inversion algorithm based on a physical model (such as ridge regression and compressed sensing) has a clear physical interpretation, the reconstruction resolution is insufficient when processing narrow-interval double-peak spectra, and it is difficult to effectively analyze the fine structure with a peak interval of less than 5 nm. In some published patents or documents, although the regularization strategy is used to improve stability, the spectrum solving capability is still limited by the bandwidth and non-orthogonality of the response function matrix. Second, although the data-driven method based on deep learning (such as a convolutional neural network and a fully connected network) has the ability to automatically extract spectral features and tolerate certain nonlinear noise, it is usually strongly dependent on large-scale, high-quality labeled training data, and it is difficult to adapt to actual scenarios with insufficient training samples or distribution drift. In addition, such models are mostly “black box structures”, and there is still a lack of physical interpretability, which is difficult to unify with the spectral transmission process in the actual spectrum system. Third, existing methods face the challenge of strong noise interference in the mid-infrared band (3000-5000 nm). For example: thermal noise, detector dark current fluctuation, and atmospheric absorption edge effect.

[0003] The reconstruction error of the traditional reconstruction model (such as wavelet transform + ridge regression) increases significantly, the MSE rises by more than 30%, the spectral peak shifts increase, and weak peaks are easily covered, especially under low-channel sampling conditions. Specifically, the most optimal scheme in the prior art improves the noise resistance through wavelet denoising and ridge regression, but its double-peak resolution capability is still limited to 10 nm; while the deep learning scheme proposed by other methods reduces the data requirement, but does not optimize the strong noise environment specific to the mid-infrared band, which limits its application in practice. In addition, the contradiction between high spectral resolution and limited hardware resources also seriously restricts the development of existing technologies. Especially in the high-demand application scenarios that require single-peak resolution accuracy of 1 nm, double-peak interval resolution better than 5 nm, and strong noise resistance (which can maintain reconstruction stability under >60% noise level), the existing algorithm system cannot be considered.

[0004] Patent document CN103207015A discloses a spectrum reconstruction method, comprising: a filter initialization step, obtaining the transmission spectrum curve of each filter channel on the filter; a spectrum acquisition step, acquiring initial spectrum information after light transmits through the filter; a spectrum reconstruction step, reconstructing the initial spectrum information using the transmission spectrum curve with a non-negative matrix full rank decomposition method to obtain reconstructed spectrum information. The present application reconstructs the spectrum information collected by the image sensor, thereby realizing the spectrum information of the detected light.

[0005] Patent document CN120063487A discloses a single-pixel-based visible-infrared fusion compressed spectrum imaging system and method, wherein the imaging method comprises the following steps: step 1, signal encoding of the target signal to be measured, obtaining the encoded target infrared signal and the encoded target visible light signal; step 2, inputting the target infrared signal to be measured into the infrared imaging light path to collect the mid-wave infrared spectrum intensity value, and inputting the target visible light signal to be measured into the visible light imaging light path to collect the visible light spectrum intensity value; step 3, compressing and sampling the visible light spectrum intensity value and the mid-wave infrared spectrum intensity value, and performing reconstruction operation to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured; step 4, image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fusion image with rich details. SUMMARY

[0006] The present application aims to provide a tunable coding and deep learning reconstruction inversion method and system for infrared spectrum detection chips, which can significantly improve the resolution and noise resistance of infrared spectrum detector spectrum reconstruction.

[0007] To achieve the first object of the present application, the following technical solution is provided: a tunable coding and deep learning reconstruction inversion method for infrared spectrum detection chips, comprising the following steps:

[0008] Constructing a spectrum response matrix of the target infrared spectrum detector;

[0009] Irradiating the target infrared spectrum detector with different real spectrum curves, collecting corresponding photocurrent data under different bias voltage channels, and pairing the real spectrum curve with the photocurrent data to form a spectrum data set, which contains a training set in the data set;

[0010] Based on the spectrum response matrix and the deep learning network framework, an initial model is constructed, which includes a data preprocessing module, a feature extraction module, a feature fusion module and a prediction module.

[0011] The data preprocessing module is used to convert the input photocurrent data into a photocurrent response curve, which is in the form of a spectrum signal tensor that can be processed by a deep learning model.

[0012] The feature extraction module includes a local feature extractor and a global feature extractor. The local feature extractor is used to extract local features of the waveform in the photocurrent response curve. The global feature extractor is used to perform position encoding on the input photocurrent data and to extract features from the position sequence obtained by position encoding to obtain the corresponding global sequence features.

[0013] The feature fusion module is used to discretize local features into global sequence features to output the corresponding fused features;

[0014] The prediction module makes predictions based on the input fusion features and outputs the prediction results.

[0015] The initial model is trained using the training set to obtain a reconstruction model for reconstructing the spectral curve;

[0016] The photocurrent data collected by the target infrared spectrometer is input into the reconstruction model to output the spectral curve corresponding to the photocurrent data.

[0017] Specifically, the process of constructing the spectral response matrix is ​​as follows:

[0018] The spectral response curves corresponding to different bias voltages were acquired using a target infrared spectrometer.

[0019] All spectral response curves were stacked by wavelength and by voltage channel to construct a response matrix of voltage channel number minus wavelength sampling point number.

[0020] Specifically, the position code is generated using a fixed sine-cosine function, with the following formula:

[0021] ;

[0022] ;

[0023] in, For wavelength position index, For feature dimensions.

[0024] Specifically, the conversion relationship between the spectral data and the photocurrent data is as follows:

[0025] ;

[0026] in, Indicates responsiveness. Represents wavelength, For spectral data intensity, and For infrared spectroscopy detectors to detect wavelengths The upper and lower limits, The photoelectric current data is obtained.

[0027] Specifically, the local feature is automatically extracted from the photoelectric current response curve by a deep learning network, and the feature is a hidden representation and is not predefined as a specific physical parameter.

[0028] Specifically, the global feature extractor outputs a global feature sequence matching the length of the position sequence by performing layer-by-layer self-attention calculation on the position sequence and residual connection.

[0029] Specifically, in the training process, a multi-index function is used to update the parameters of the initial model.

[0030] Specifically, the multi-index function includes mean square error and spectral matching degree.

[0031] In order to realize the second object of the application, the technical scheme is provided as follows: an adjustable coding and deep learning reconstruction inversion system is used to perform the steps of the adjustable coding and deep learning reconstruction inversion method of the infrared spectrum detection chip to reconstruct a high-resolution spectrum curve.

[0032] Compared with the prior art, the beneficial effects of the present application are:

[0033] In the limited channel light response data, the complete spectrum curve of the target in the mid-infrared band is recovered or inferred, the spectral resolution and noise robustness of the reconstruction are significantly improved, and the key bottlenecks of the prior art in the three aspects of "spectrum information missing", "resolution limited" and "strong noise interference" are effectively broken through. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 The schematic diagram of the adjustable coding and deep learning reconstruction inversion method provided in the embodiment is shown in the figure.

[0035] Figure 2 The schematic diagram of the model reconstruction provided in the embodiment is shown in the figure.

[0036] Figure 3 The reconstruction result schematic diagram provided in the embodiment is shown in the figure. DETAILED DESCRIPTION

[0037] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0038] As shown in the present embodiment, an adjustable coding and deep learning reconstruction inversion method for an infrared spectral detection chip is provided, which includes the following steps: Figure 1

[0039] The spectral response matrix of the target infrared spectral detector is constructed, and the specific steps in the present embodiment are as follows:

[0040] Different bias voltages are applied to the dynamic tunable filter at the front end of the dynamic infrared focal plane detector, so that the filter obtains different spectral transmittance curves under voltage channels.

[0041] The filter has selective transmittance ability for different wavelengths of infrared light under different voltages, so that the detector corresponds to a specific group of spectral responsivity curves under each channel.

[0042] The responsivity curves under all channels are arranged in the wavelength dimension and stacked in the voltage channel dimension, and a response matrix with a size of ( is the number of voltage channels, is the number of wavelength sampling points) can be constructed. The response matrix describes the spectral modulation characteristics of the dynamic infrared focal plane.

[0043] The target infrared spectral detector is irradiated with different real spectral curves, and corresponding photocurrent data is collected under different bias voltage channels. The real spectral curve and the photocurrent data are paired to form a spectral data set, and the data set includes a training set. In the present embodiment, the conversion relationship between the spectral curve and the photocurrent data is as follows: ;

[0044] wherein, represents the responsivity, represents the wavelength, is the spectral data intensity, and ​The upper and lower limits of the wavelength that can be detected by the infrared spectrum detector, are the photocurrent data.

[0045] Since the data of the actual environment is discrete, the above integral form can be converted to :

[0046] ;

[0047] Since , where U is the voltage, I is the current, and r is the resistance, it can also be converted to , that is . Because some detectors receive light signals and are processed by readout circuits to output in the form of photovoltage data (rather than photocurrent data), the deep learning algorithm can also use photovoltage data under different voltage channels and corresponding raw spectra as the training set for model training, and use the actually measured photovoltage data for spectrum reconstruction.

[0048] The training set in this embodiment mainly comes from three aspects: (1) the spectrum data and corresponding light response data measured by the actual spectrometer; (2) the spectrum data and corresponding light response data simulated by the computer through the model; (3) the spectrum data in the existing public data set.

[0049] As Figure 2 shown, it is a schematic diagram of the model reconstruction provided in this embodiment. First, the dual-feature fusion driven deep learning spectrum reconstruction algorithm reads the training data and test data from the spectrum data set for model training, wherein the deep learning spectrum reconstruction process combines the local feature extraction capability of the residual network and the global dependence modeling capability of the Transformer. The processing of the light response data and the spectrum data is mainly divided into two paths, one of which uses a residual network to process local features, and the other uses a Transformer module to process global features, to hybridly fuse the dual-path feature output.

[0050] The dual-feature fusion driven deep learning spectrum reconstruction algorithm has the following technical advantages:

[0051] Dual-modal feature fusion: simultaneously capturing local features (convolution path) and global dependencies (Transformer path);

[0052] Position-aware capability: preserving wavelength position information through sine / cosine encoding;

[0053] Deep optimization: residual connections ensure stable training of 20+ layer networks;

[0054] Computational efficiency: adaptive pooling replaces fully connected layers, reducing parameters by 80%;

[0055] Strong generalization: Dropout and weight decay effectively prevent overfitting;

[0056] Based on Python and Qt5 framework, a graphical interface application program integrating deep learning spectral reconstruction algorithm is developed. The system adopts modular design concept, consisting of model training module and spectral reconstruction module, realizing the functions of parameter configuration, training process visualization of deep learning model and rapid reconstruction of high-resolution spectral data.

[0057] The model includes a data preprocessing module, a feature extraction module, a feature fusion module, and a prediction module. The feature extraction module includes a local feature extractor and a global feature extractor. The local feature extractor includes four residual blocks. The first residual block does not change the channel number, the second residual block increases the channel number from 64 to 128, the third residual block does not change the channel number, and the fourth residual block further increases the channel number from 128 to 256. Each of the above residual blocks is composed of two one-dimensional convolution layers and a shortcut, which can effectively capture the local features of the input spectrum. The global feature extractor includes self-attention and feedforward neural network. The feature extractor is stacked by 6 encoder blocks. Each encoding block contains 8 self-attention mechanisms and feedforward neural networks. The expansion factor of the feedforward neural network is 4, and then a multi-head self-attention mechanism is used to model the input sequence globally to learn the long-distance dependency between different wavebands of the spectrum. The global feature is a hidden representation, including but not limited to the relative relationship between spectral peaks and the overall spectral profile.

[0058] The initial model constructed above is trained using the training set constructed to obtain a reconstruction model for reconstructing the spectral curve.

[0059] In this embodiment, mean square error (MSE) and spectral matching degree (normalized cross correlation NCC) are used, and the MSE calculation formula is:

[0060] .

[0061] The spectral matching degree (normalized cross correlation NCC) calculation formula is:

[0062] .

[0063] The photocurrent data collected by the dynamic adjustable infrared focal plane detector is input into the reconstruction model to output the spectral curve corresponding to the photocurrent data,

[0064] In this embodiment, the reconstruction model first projects the m-dimensional input optical response data to a high-dimensional embedding space with embed_size=256 through linear mapping, and adds position encoding based on the sine function to introduce sequence information in the spectral dimension. Subsequently, the Transformer path performs layer-by-layer self-attention calculation and residual connection on the embedded sequence, outputting a global feature sequence matching the sequence length; by averaging pooling the sequence, its overall representation is extracted. At the same time, the original input data is also sent to the residual convolution path, which goes through four residual blocks and global average pooling to obtain a local feature vector with a length of 256.

[0065] In the feature fusion module, the global features output by the Transformer branch and the local features output by the residual convolution branch are spliced and fused (forming a fusion vector with a length of 512), and then the final spectral reconstruction output is completed through a two-layer fully connected network.

[0066] The entire model architecture uses residual connection, layer normalization, and multi-head attention mechanism to improve training stability and expression ability, and builds an infrared spectral reconstruction framework that complements global and local features. Compared with traditional models, this structure has stronger robustness and generalization ability while improving reconstruction accuracy, and is especially suitable for high-resolution, strong noise reconstruction scenarios in the mid-infrared band.

[0067] The embodiment also provides an adjustable coding and deep learning reconstruction inversion system for executing the steps of the adjustable coding and deep learning reconstruction inversion method for infrared spectral detection chips provided in the above embodiments.

[0068] As shown in Figure 3 , the system provided in the above embodiments is used to carry out spectral reconstruction tests on wide-band single-peak, narrow-band single-peak, double-peak, and triple-peak optical response data.

[0069] Among them, the left graph is the single-peak reconstruction effect, and the right graph is the double-peak reconstruction effect. The full width at half maximum of the single-peak and double-peak is 50 nm. In the single-peak and double-peak spectral reconstruction scenarios, the algorithm can achieve an MSE less than 0.001 and a spectral matching degree of more than 0.96, showing excellent spectral reconstruction accuracy. For triple-peak and complex multi-peak spectral reconstruction tasks, the algorithm still maintains an MSE of less than 0.002 and a spectral matching degree of more than 0.93, showing excellent multi-peak spectral reconstruction adaptation ability.

[0070] The experimental results show that the algorithm not only has high-precision representation characteristics in single / double-peak spectral reconstruction, but also has excellent generalization reconstruction performance for triple-peak and complex multi-peak systems, effectively verifying its robustness and applicability in different spectral complexity scenarios.

[0071] The spectral reconstruction experiment is carried out on the single-peak spectral light response data with a peak height of 1.0 and a full width at half maximum of 100 nm, and the difficulty of resolving narrow-interval single-peak is solved, with a center wavelength interval of only 1 nm. The experimental data show that the algorithm realizes MSE<0.001 in single-peak and double-peak reconstruction scenes, and the reconstructed spectrum is highly consistent with the original spectrum, which is intuitively verified, and fully demonstrates the strong resolution capability and high-precision spectral reconstruction performance of the algorithm under ultra-narrow wavelength interval.

[0072] The spectral reconstruction experiment is carried out on the double-peak spectral light response data with a peak height of 1.0, a full width at half maximum of 3 nm and a peak interval of 5 nm, and the bottleneck of spectral resolution of narrow-interval double-peak is successfully solved. The experiment realizes high-precision reconstruction with a mean square error (MSE) of 0.000987 and a spectral matching degree of 0.937834. The above quantitative results show that even in the extreme scene of narrow-interval (5 nm) and narrow line width (FWHM 3 nm) double-peak, the algorithm still shows excellent spectral reconstruction accuracy and spectral shape reproduction capability, effectively verifying its robustness in high-resolution spectral analysis tasks.

[0073] The anti-noise reconstruction experiment is carried out by injecting 2%, 5%, 10%, 20%, 40% and 60% Gaussian noise into the double-peak limit spectrum with a peak height of 1.0, a full width at half maximum of 3 nm and a peak interval of 5 nm, and the robustness of the algorithm in the noise environment is explored. The data show that under the interference of different noise levels, the algorithm can accurately reproduce the spectral shape characteristics of the limit resolution double-peak, and shows excellent stable reconstruction performance. Especially when facing high noise interference such as 40% and 60% which exceeds the threshold of conventional application scene, the algorithm can still effectively extract the double-peak characteristics. The results fully verify the engineering applicability and anti-interference potential of the algorithm in the extreme noise environment, and provide technical support for spectral analysis in complex measurement scenes.

[0074] The above description (including the drawings and specific embodiments) is only for explaining the technical idea and preferred embodiments of the present application, and the description cannot be used to limit the protection scope of the present application. Any equivalent replacement, modification or variation of the technical solution made by any person skilled in the art based on the technical idea of the present application, as long as it does not deviate from the function and structure principle of the present application, shall fall within the protection scope defined by the claims of the present application.

Claims

1. An adjustable coding and deep learning reconstruction inversion method for an infrared spectral detection chip, characterized in that, The method comprises the following steps: constructing a spectral response matrix of a target infrared spectral detector; irradiating the target infrared spectral detector with different real spectral curves, collecting corresponding photocurrent data under different bias voltage channels, and pairing the real spectral curves with the photocurrent data to form a spectral data set; constructing an initial model based on the spectral response matrix and a deep learning network framework, the initial model comprising a data preprocessing module, a feature extraction module, a feature fusion module, and a prediction module; the data preprocessing module is configured to convert the input photocurrent data into photocurrent response curves; the feature extraction module is configured to extract local features of the photocurrent response curves and global feature sequences of the photocurrent data, the feature extraction module comprising a local feature extractor and a global feature extractor, the local feature extractor being configured to extract local features of waveforms in the photocurrent response curves, and the global feature extractor being configured to perform position encoding on the input photocurrent data and extract features from a position sequence obtained by the position encoding to obtain corresponding global sequence features; The position coding is generated by a fixed sine-cosine function, and the formula is: ; ; wherein, is a wavelength position index, is a feature dimension; the global feature extractor outputs a global feature sequence matching the length of the position sequence through layer-by-layer self-attention calculation and residual connection on the position sequence; the feature fusion module is configured to discretize the local features into the global sequence features to output corresponding fusion features; the prediction module is configured to perform prediction according to the input fusion features to output a prediction result; the initial model is trained using the spectral data set to obtain a reconstruction model for reconstructing spectral curves; the photocurrent data collected by the target infrared spectral detector is input into the reconstruction model to output a spectral curve corresponding to the photocurrent data.

2. The adjustable coding and deep learning reconstruction inversion method for the infrared spectrum detection chip according to claim 1, characterized in that, The construction process of the spectral response matrix is as follows: collecting corresponding spectral responsivity curves under different bias voltages by the target infrared spectral detector; stacking all the spectral responsivity curves in the order of wavelength arrangement and voltage channel to construct a response matrix with the number of voltage channels and the number of wavelength sampling points.

3. The adjustable coding and deep learning reconstruction inversion method for the infrared spectrum detection chip according to claim 1, characterized in that, The conversion relationship between the spectral data and the photocurrent data is as follows: ; wherein, represents the responsivity, represents the wavelength, is the spectral data intensity, and is the upper and lower limits of the wavelength that can be detected by the infrared spectrum detector, is the photocurrent data.

4. The adjustable coding and deep learning reconstruction inversion method for the infrared spectrum detection chip according to claim 1, characterized in that, In the training process, a multi-index function is used to update the parameters of the initial model.

5. The adjustable coding and deep learning reconstruction inversion method for the infrared spectrum detection chip according to claim 4, characterized in that, The multi-index function comprises a mean square error and a spectral matching degree.

6. A tunable coded and deep learning reconstruction inversion system, characterized in that, The method is used to perform the steps of the adjustable coding and deep learning reconstruction inversion method for the infrared spectral detection chip as claimed in any one of claims 1-5 to reconstruct a high-resolution spectral curve.

Images