Spectrum reconstruction method for broadband filtering modulation spectrum detection system

Abstract

The invention relates to the field of spectrum reconstruction, in particular to a spectrum reconstruction method for a broadband filtering modulation spectrum detection system, which comprises the following steps of: converting a hyperspectral data set into a training data set according to parameters of the broadband filtering modulation spectrum detection system; building a spectrum reconstruction network model based on deep learning and carrying out training; and pixel splitting point-by-point scanning reconstruction is carried out on a reconstruction target, a hyperspectral image is constructed according to original arrangement, space-spectrum joint optimization is carried out, and finally high-precision spectral image reconstruction is realized. According to the method, the demand of a spectrum reconstruction process on a data set scale is reduced through a pixel splitting point-by-point scanning reconstruction strategy, a Transform architecture is introduced into a reconstruction network, the feature information extraction capability of the network is improved, the strong nonlinear characterization capability of a KAN architecture is combined, the spectrum curve reconstruction precision is improved, and spatial spectrum combined processing is combined, so that the spectrum reconstruction efficiency is improved. The error is further reduced, and high-quality reconstruction of the spectral image is realized.

Description

Technical Field

[0001] This invention relates to the field of spectral reconstruction, specifically to a spectral reconstruction method for broadband filtered modulation spectral detection systems, which realizes the reconstruction of broadband filtered modulation signals into original hyperspectral signals. Background Technology

[0002] Spectroscopic detection, as a key technology for obtaining information on the composition and state of matter, has irreplaceable application value in fields such as materials science, environmental monitoring, and astrophysics. Traditional hyperspectral detection systems are mainly based on three physical spectral separation principles: dispersive spectral separation, filter spectral separation, and Fourier transform spectral separation. Among them, dispersive spectral separation achieves spectral separation based on the dispersion effect of prisms or gratings. However, due to the physical scale of optical devices, achieving high resolution requires increasing the optical path length, resulting in a large device size. In addition, the dispersion process causes light energy to be dispersed throughout the detector array, resulting in weak energy reception per pixel. Filter spectral separation typically uses a large number of narrow-bandpass filters for spectral selection. If mechanical moving parts are used to switch filters, there are limitations on spectral and temporal resolution. If multi-aperture systems or filter arrays similar to Bayer arrays are used, there are limitations on spectral and spatial resolution. Furthermore, even for a high-performance narrow-bandpass filter, its narrow half-width at half-maximum (HWHM) also leads to lower light energy received by the detector pixels. Fourier spectroscopic methods based on the Michelson interferometry principle offer advantages such as high luminous flux, high output, and high signal-to-noise ratio. However, they suffer from long measurement times, high optomechanical precision requirements, complex structures, and high sensitivity to environmental vibrations. Due to physical limitations, the aforementioned three types of systems struggle to simultaneously achieve miniaturization, high spatiotemporal resolution, and high signal-to-noise ratio. To address this issue, broadband filtering modulation computational spectral detection systems exhibit unique advantages. These systems integrate micro / nano optical filter arrays with detectors, employing filter units with low correlation between transmission spectra to achieve spectral encoding and compressed sensing. This significantly reduces system size while maintaining detection accuracy, and broadband filtering substantially increases the luminous flux per pixel, resulting in a high signal-to-noise ratio. However, while broadband filtering modulation offers these advantages, it inevitably leads to optical signal aliasing due to dimensionality reduction sampling, necessitating spectral reconstruction to obtain the original target spectral information. In practical applications, super-resolution processing of the spectral dimension must be performed simultaneously with reconstruction, further increasing the difficulty. Therefore, there is an urgent need for spectral reconstruction algorithms that efficiently utilize known spatial and spectral information to expand the number of spectral channels and achieve high-quality reconstruction of the original spectral signal. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a spectral reconstruction method for broadband filtered modulation spectral detection systems. This method effectively utilizes known spatial and spectral information, expanding the number of spectral channels while ensuring spectral reconstruction accuracy. Combined with the hardware of the broadband filtered modulation spectral detection system, this method can solve the problem of the mutual constraints between the number of spectral channels, spectral resolution, and detection sensitivity while achieving miniaturization of the spectral detection system, thus possessing high practicality.

[0004] To achieve the above objectives, the present invention provides the following solution: A spectral reconstruction method for a broadband filtered modulation spectral detection system includes the following steps: Step S1: Perform data preprocessing and construct a training dataset for the spectral reconstruction network: Select one or more hyperspectral images containing all types of detected targets, split the hyperspectral images into spectral curves in units of pixels, process the spectral curves according to the transmission curves of the filter arrays of each channel used in the broadband filter modulation spectral detection system and the detector response curves, add noise, and convert the spectral curves into multi-channel aliasing signals to simulate the response of the spectral signal of the detected target to the pixels of the multi-channel detector. The training dataset is composed of the spectral intensity of the band to be reconstructed at each pixel and the corresponding detector response. Step S2, build a deep learning-based spectral reconstruction network model: First, construct a Transformer-based regression network, use an embedding mapping layer to map the input signal to a high-dimensional space to extract feature information, combine a multi-layer Transformer encoder to dynamically calculate the correlation between different features, automatically focus on key information, replace the FFN module in the Transformer encoder with a KAN network, and finally use a dimension-compressed KAN network as the regression head to form a KAN regression head. Through the local adaptability of the spline basis function of the KAN regression head, accurately regress the spectral data and output the predicted values ​​of the spectral intensity of each band. Step S3: Train the spectral reconstruction network. Use the detector response in the dataset as the network input and the spectral intensity of each band as the network output. Before training, initialize the data and divide the initialized dataset into training set, test set, and validation set for training, testing, and validation of the neural network, respectively. Train the built model using the mean squared error (MSE) as the network loss function. Use the AdamW optimizer based on cosine annealing learning rate combined with early stopping mechanism and dynamic batch size growth as the network training strategy. Store the network model parameters with the minimum loss function on the validation set and the dataset initialization parameters. Step S4, spectral reconstruction: After processing the multi-channel response obtained by the detector using the saved initialization parameters, the input is scanned point by point into the stored spectral reconstruction network, and the normalized hyperspectral information corresponding to the pixel is calculated and output. After performing inverse normalization on the output, the reconstructed spectral curves are stitched together according to the pixel arrangement of the detector to obtain the hyperspectral image data of the detected target. Step S5 involves performing spatial-spectral joint optimization on the hyperspectral image data, applying spatial bilateral filtering and total variational regularization to each spectral channel to obtain a high-precision spectral image.

[0005] Furthermore, in step S1, the broadband filtered modulation spectral detection system uses broadband random filters, each of which transmits throughout the entire operating wavelength band, and the transmission curves between channels have low correlation. The overall correlation coefficient of the transmission curves between channels of the broadband filtered modulation spectral detection system is set to be less than 0.2, wherein the correlation coefficient of the transmission curves of the i-th and j-th channels is... Defined as: , In the formula The transmission spectra of the i-th and j-th channels are respectively. Let i be the covariance of the transmission spectra i and j. for The expansion form of covariance operation, Let i and j be the standard deviations of the transmission spectra. Let be the mean values ​​of transmission spectra i and j, and be the mean correlation value of the i-th channel among the n channels of the broadband filtered modulated spectral detection system. for: , Total correlation coefficient of n channels Defined as: .

[0006] Furthermore, in step S1, the hyperspectral image, the transmission curve of each channel filter array, and the detector response curve used to construct the training dataset all contain the interval where the target band is located. When there are differences in the bands where the sampling points of the three are located, spline interpolation is used to unify the bands of the sampling points.

[0007] Furthermore, the noise added in step S1 includes photon noise and circuit readout noise, wherein the photon noise N photon The circuit follows a Poisson distribution, and the readout noise N readout It follows a Gaussian distribution.

[0008] Furthermore, the formula for converting the spectral curve into a multi-channel aliased signal in step S1 is expressed as follows: , After image registration The pixel response value corresponding to the same object point in each channel is , These are the spectral channel numbers; the spectral information of the target point in each band is represented as follows: For broadband filtered modulation spectral detection systems, the broadband filter array in The broadband filter spectral transmission response corresponding to each channel is expressed as follows: The detector's spectral response efficiency is expressed as The noise generated by each channel of the detector is , The minimum wavelength at which the detector operates. Let M be the maximum operating wavelength of the detector. The spectral response integral equation is discretized with a degree of discretization of M and an equal-separation discretization method. The original equation is then expressed as: , Where m is the number of the band to be reconstructed. The spectral intensity of each pixel in the original data corresponding to the wavelength of the band to be reconstructed is selected and together with the corresponding multi-channel detector response to form the training dataset of the spectral reconstruction network.

[0009] Furthermore, the embedding mapping layer is composed of fully connected layers, with the number of input nodes equal to the number of detector response channels, mapping the input data to a 128-dimensional high-dimensional space to achieve high-dimensional mapping; The Transformer encoder contains more than three encoder units, each equipped with a 4-head self-attention mechanism. The standard FFN feedforward network is replaced with a KAN network, which uses cubic spline basis functions and a 5×5 interpolation grid. Feature enhancement is achieved through residual connections and layer normalization. The extracted 128-dimensional Transformer features are further processed by the KAN regression head to achieve decoding and dimensionality reduction. The number of output nodes is equal to the number of reconstructed bands.

[0010] Furthermore, step S3 includes the following stages: 1) Data preprocessing Before training, the dataset is first initialized to improve model convergence speed and reduce the impact of parameter value range on parameter weights. The mean-standard deviation standardization method is used, and the calculation formula is as follows: , in, As input to the training set, For the output of the training set, These are the mean and standard deviation of the input to the training set, respectively. These are the mean and standard deviation of the training set outputs, respectively. It is a very small constant. The input values ​​of the training set after standardization. To standardize the output values ​​of the training set after computation, the initialized dataset is divided into a training set, a test set, and a validation set for training, testing, and validation of the neural network, respectively. 2) Construct the loss function Mean squared error (MSE) is used as the loss function for network training. The calculation formula is: , Where B is the number of samples processed in the same batch, and K represents the feature dimension of each sample. Let be the predicted value of the j-th dimension for the i-th sample. For the corresponding true value; Add a KAN regularization term to the loss function calculation. The formula is: , in, The coefficients of the spline basis functions are represented. To find the square of the L2 norm, N is the total number of coefficients; Total loss function The calculation formula is: , 3) Training methods Training employs the AdamW optimizer, combined with a cosine annealing strategy to dynamically adjust the learning rate. Initial learning rate and weight decay parameters are set based on task type and dataset size. Simultaneously, training efficiency and model stability are balanced by dynamically adjusting the batch size, which gradually doubles from the initial value as training progresses and has an upper limit. During training, model parameters and dataset initialization parameters that minimize the total loss of the validation set are saved and ultimately used for deployment in the spectral reconstruction task.

[0011] Furthermore, the point-by-point scanning in step S4 is a single response data of the multi-channel sensor. After ensuring pixel spatial alignment through image registration, the multi-channel response value of each pixel is extracted sequentially from left to right according to the row scanning order. This value is then input into the trained spectral reconstruction network, which outputs the standardized hyperspectral information corresponding to that pixel. After performing an inverse standardization operation on the output, the spectral reconstruction results of each pixel are finally rearranged into a hyperspectral image according to the pixel spatial arrangement of the original image, thus completing the spectral reconstruction of the sensor response.

[0012] Furthermore, the accuracy of the reconstructed spectral curve in step S4 is represented by the mean square error, peak signal-to-noise ratio, or spectral similarity, as shown in the following formulas: , , , in, Mean square error, Peak signal-to-noise ratio, For spectral similarity, For predicted values, and These are the transposes of the true value and the true value, respectively. Number of bands It is the maximum value in the data. This is an inverse cosine operation.

[0013] Furthermore, step S5 specifically includes: First, total variational regularization is performed on each spectral channel to minimize the total variation of the image gradient to smooth noise while preserving edges. The objective function of total variational regularization is: , in, This indicates taking the minimum value, where dx is the differential symbol. This is the image after denoising. It is the original image. It is the image gradient magnitude. Control the regularization strength; After performing total variational regularization, a bilateral filter is further applied to the image of each spectral channel. The bilateral filter formula is as follows: , in, Pixels after filtering The new value of , where e is the natural constant. It is a pixel The neighborhood, For pixels in the neighborhood, For pixels The original value, For pixels The original value, Controlling the size of the space kernel, The width of the color kernel is controlled by W, which is the normalization factor. The parameters of total variation regularization and bilateral filtering are automatically explored through Bayesian optimization, with mean squared error as the optimization objective, to automatically search for the total variation regularization weights. bilateral filter kernel parameters and The optimal combination.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] 1) This invention enables high-precision reconstruction of the spectral aliasing signal obtained by the broadband filtered modulation spectral detection system into the original target hyperspectral information, and achieves spectral super-resolution in the process;

[0016] 2) This invention significantly reduces the data set requirements for deep learning-based hyperspectral image reconstruction by combining pixel-by-pixel splitting and point-by-point scanning reconstruction with joint processing of spatial-spectral information;

[0017] 3) This invention introduces the Transformer architecture into the reconstruction network, which improves the network's ability to extract feature information. Combined with the powerful nonlinear representation ability of the KAN network, it reduces the error of the reconstructed spectrum compared with the conventional spectral reconstruction method based on fully connected layers. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments and descriptions of the invention and are intended to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0019] Figure 1 This is a flowchart of a spectral reconstruction method for a broadband filtered modulation spectral detection system according to the present invention;

[0020] Figure 2 This is a schematic diagram of a broadband filter-modulated spectral detection system in a spectral reconstruction method for a broadband filter-modulated spectral detection system according to the present invention;

[0021] Figure 3 This is a schematic diagram of the spectral reconstruction network model structure in a spectral reconstruction method for a broadband filtered modulation spectral detection system according to the present invention.

[0022] Figure 4 This is a comparison diagram of the reconstructed spectrum and the original target spectrum using curve fitting in a spectral reconstruction method for a broadband filtered modulation spectral detection system according to the present invention.

[0023] Figure 5 This is a schematic diagram comparing the response of the stitched hyperspectral image after spatial-spectral joint optimization with that of the original target in a single spectral channel in a spectral reconstruction method for a broadband filtered modulation spectral detection system according to the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0025] To make the above-mentioned objects, features and advantages of the invention more apparent and understandable, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Reference Figures 1-5 As shown, a spectral reconstruction method for a broadband filtered modulation spectral detection system includes the following steps:

[0027] Step S1: Perform data preprocessing and construct a training dataset for the spectral reconstruction network. Select one or more hyperspectral images containing all types of detected targets, and split the hyperspectral images into spectral curves in pixels. Process the spectral curves according to the transmission curves of the filter arrays and the detector response curves used in the broadband filter modulation spectral detection system, and add noise to convert the spectral curves into multi-channel aliasing signals to simulate the response of the spectral signal of the detected target to the pixels of the multi-channel detector. The training dataset is composed of the spectral intensity of the band to be reconstructed at each pixel and the corresponding detector response.

[0028] The hyperspectral image dataset selected in this example is Salinas, which was acquired by the AVIRIS sensor. It consists of 512×217 pixels and 224 spectral bands with wavelengths ranging from 0.4µm to 2.5µm, containing object samples from 16 categories. In practical use, the responses of 20 bands covering the water absorption area (bands 108-112, 154-167, and 224) were removed. A Salinas-A scene containing 86×83 pixels was used for spectral image reconstruction testing, yielding a total of 111,104 spectral curves after pixel splitting.

[0029] In step S1, the broadband filtered modulation spectral detection system is as follows: Figure 2 As shown, this system uses broadband random filters, each of which transmits across the entire operating wavelength band, and the transmission curves between channels exhibit low correlation. Specifically, the transmission curves of channels i and j show low correlation. Defined as:

[0030]

[0031] In the formula The transmission spectra of the i-th and j-th channels are respectively. Let i be the covariance of the transmission spectra i and j. for The expansion form of covariance operation, Let i and j be the standard deviations of the transmission spectra. Let be the mean values ​​of transmission spectra i and j, and be the mean correlation value of the i-th channel among the n channels of the broadband filtered modulated spectral detection system. for:

[0032]

[0033] Total correlation coefficient of n channels Defined as:

[0034]

[0035] The transmission spectra of 120 filters were calculated using the correlation calculation formula. Nine broadband random filters with a total correlation coefficient of 0.1683, covering the transmission bands from 0.4µm to 2.5µm, were selected and placed in the nine channels. The spectral response curve of the mercury cadmium telluride detector was used as the detector response efficiency of the system. Using the spectral curves of the dataset as a benchmark, the consistency of the sampling point bands of the filter transmission curves and detector response curves in each channel was compared. In this example, the sampling points of the three types of filters differed in band. Cubic spline interpolation was used to unify the spectral sampling point bands.

[0036] In step S1, the formula for converting the spectral curve into a multi-channel aliased signal is expressed as follows:

[0037]

[0038] After image registration The pixel response value corresponding to the same object point in each channel is , These are the spectral channel numbers; the spectral information of the target point in each band is represented as follows: For broadband filtered modulation spectral detection systems, the broadband filter array in The broadband filter spectral transmission response corresponding to each channel is expressed as follows: The detector's spectral response efficiency is expressed as The noise generated by each channel of the detector is , The minimum wavelength at which the detector operates. For the maximum operating wavelength of the detector, the integral equation of the spectral response is discretized with a degree of discretization of M and an equal-separation discretization method. The discretized form of the equation is then:

[0039]

[0040] Where m is the number of the band to be reconstructed. This corresponds to the wavelength of that band.

[0041] For each spectral channel, the spectral intensity of the dataset, the filter transmittance, and the sensor response efficiency are multiplied according to the above equation and summed to obtain the ideal detector response. Poisson noise, calculated as the desired ideal detector response, is then added to simulate photon noise N. photon Add Gaussian noise with a mean of 0 and a standard deviation of 5 to surround the analog circuit readout noise N. readout The detector's true response is obtained by summing the ideal response of the detector with the noise.

[0042] The above calculations were performed on 111,104 spectral curves from the Salinas dataset to obtain 111,104 sets of multi-channel detector responses. Thirty-two bands with detector response efficiencies greater than 0.8 were selected as the reconstructed spectral bands. The dataset consists of the spectral intensities of the 111,104 spectral curves in these 32 bands and the corresponding 9-channel detector responses.

[0043] Step S2: Construct a deep learning-based spectral reconstruction network model. First, construct a Transformer-based regression network. Use an embedding mapping layer to map the input signal to a high-dimensional space to extract feature information. Combine a multi-layer Transformer encoder to dynamically calculate the correlation between different features and automatically focus on key information. Replace the FFN module in the Transformer encoder with a KAN network. Finally, use a dimensionally compressed KAN network as the regression head to form a KAN regression head. Through the local adaptability of the spline basis functions of the KAN regression head, accurately regress the spectral data and output the predicted values ​​of the spectral intensity of each band.

[0044] In step S2, the deep learning-based spectral reconstruction network model is as follows: Figure 3 As shown, a three-level architecture of embedded mapping layer-Transformer encoder-KAN regression head is adopted. The embedded mapping layer consists of fully connected layers, with the number of input nodes equal to the number of detector response channels. The 9-dimensional detector response is nonlinearly mapped to a 128-dimensional feature space through the fully connected network. Subsequently, the data is input into the Transformer encoder, which contains 5 levels of encoder units. Each encoder unit consists of a self-attention mechanism with 4 independent attention heads, residual connections and layer normalization modules, and a KAN network. The KAN network uses cubic spline basis functions and a 5×5 interpolation grid to replace the traditional feedforward neural network, and enhances the spectral detail modeling capability through differentiable spline interpolation. Finally, the KAN regression head outputs 32-dimensional data, corresponding to the responses of 32 reconstructed bands, through two-level progressive dimensionality reduction of the extracted 128-dimensional Transformer features.

[0045] Step S3: Train the spectral reconstruction network. Use the detector response in the dataset as the network input and the spectral intensity of each band as the network output. Before training, initialize the data and divide the initialized dataset into training, testing, and validation sets for training, testing, and validation of the neural network, respectively. Train the constructed model using the mean squared error (MSE) as the network loss function. Use the AdamW optimizer based on cosine annealing learning rate combined with early stopping mechanism and dynamic batch size growth as the network training strategy. Store the network model parameters with the minimum loss function on the validation set along with the dataset initialization parameters.

[0046] In step S3, the dataset is first standardized by mean-standard deviation to improve model convergence speed and reduce the impact of parameter value range on parameter weights. The calculation formula is as follows:

[0047]

[0048] in, As input to the training set, For the output of the training set, The mean and standard deviation are input to the training set. The mean and standard deviation of the training set output. It is a very small constant to prevent division by zero errors. The input values ​​of the training set after standardization. This represents the output value of the training set after standardization. 7138 data sets corresponding to the Salinas-A scene in the Salinas dataset were removed and used as the test set. The remaining 103966 data sets were initialized and used as the training dataset, divided into training and validation sets at a ratio of 90% and 10%, respectively, for training and validation of the neural network.

[0049] In step S3, the mean squared error (MSE) is used as the loss function for network training. The formula for calculating the mean squared error loss function is as follows:

[0050]

[0051] Where B is the number of samples processed in the same batch, and K represents the feature dimension of each sample. Let be the predicted value of the j-th dimension for the i-th sample. This corresponds to the actual value.

[0052] To prevent the KAN network from overfitting and resulting in poor model generalization ability, a KAN regularization term is added to the loss function calculation. The formula is:

[0053]

[0054] in The coefficients of the spline basis functions are represented. To find the square of the L2 norm, N is the total number of coefficients.

[0055] The overall loss function is a weighted average of the two, as shown in the formula:

[0056]

[0057] In step S3, the AdamW optimizer is used, and the learning rate is dynamically adjusted in conjunction with a cosine annealing strategy. The initial learning rate is 1×10⁻⁶. -3 The weight decay parameter is 1×10 -4 To balance training efficiency and model stability, a dynamic batch size is used. The initial batch size is 512, and it doubles every 10 training epochs, with a maximum of 8192. Model parameters and dataset initialization parameters that minimize the total loss of the validation set are saved over 200 training epochs for deployment in the spectral reconstruction task.

[0058] Step S4, spectral reconstruction: After processing the multi-channel response obtained by the detector using the saved initialization parameters, the input is scanned point by point into the stored spectral reconstruction network, and the normalized hyperspectral information corresponding to the pixel is calculated and output. After performing inverse normalization on the output, the reconstructed spectral curves are stitched together according to the detector pixel arrangement to obtain the hyperspectral image data of the detected target.

[0059] The point-by-point scanning process involves processing the response data of a multi-channel sensor. After ensuring pixel spatial alignment through image registration, the multi-channel response value of each pixel is extracted sequentially from left to right in a row scanning order. This value is then input into the trained spectral reconstruction network, which outputs the standardized hyperspectral information corresponding to that pixel. After performing an inverse standardization operation on the output, the spectral reconstruction results of each pixel are rearranged according to the pixel spatial arrangement of the original image to form a hyperspectral image, thus completing the spectral reconstruction of the sensor response.

[0060] In step 4, the detector responses corresponding to the 7138 spectral curves of the Salinas-A scene are used with the training set. The mean and standard deviation are standardized using mean-standard deviation. The trained spectral reconstruction network is input row by row from left to right, and the standardized hyperspectral information corresponding to each pixel is output, based on the training set. After performing inverse normalization on the output for the mean and standard deviation, the spectral intensities of the 32 bands corresponding to the detector response are obtained.

[0061] Error analysis was performed between the reconstructed spectral intensity and the true spectral intensity, and the mean square error, peak signal-to-noise ratio, and spectral similarity were calculated respectively, using the following formulas:

[0062]

[0063]

[0064]

[0065] in, Mean square error, Peak signal-to-noise ratio, For spectral similarity, For predicted values, and These are the transposes of the true value and the true value, respectively. Number of bands It is the maximum value in the data. This is an inverse cosine operation.

[0066] The mean squared error between the reconstructed spectral intensity and the true spectral intensity in the entire Salinas-A scene is 35.59, the peak signal-to-noise ratio is 38.70, and the spectral similarity is 0.9836. The 7138 spectral curves are arranged in the input order to form an 86×83 pixel hyperspectral image.

[0067] like Figure 4 As shown, this is a comparison between the reconstructed values ​​and the true values ​​of each spectral band of a randomly selected pixel.

[0068] Step S5 involves performing spatial-spectral joint optimization on the hyperspectral image data, applying spatial bilateral filtering and total variational regularization to each spectral channel to obtain a high-precision spectral image.

[0069] In step S5, for generating an 86×83 pixel hyperspectral image containing 32 spectral channels, total variational regularization is first performed on each spectral channel, followed by bilateral filtering.

[0070] Total variational regularization minimizes the total variation of the image's gradient to smooth noise while preserving edges. Its objective function is:

[0071]

[0072] That This indicates taking the minimum value, where dx is the differential symbol. This is the image after denoising. It is the original image. It is the image gradient magnitude. Control the regularization strength;

[0073] After performing total variational regularization, a bilateral filter is further applied to the image of each spectral channel. The bilateral filter formula is as follows:

[0074]

[0075] in Pixels after filtering The new value of , where e is the natural constant. It is a pixel The neighborhood, For pixels in the neighborhood, For pixels The original value, For pixels The original value, Controlling the size of the space kernel, The width of the color kernel is controlled by W, which is the normalization factor.

[0076] Weights of total variation regularization in each spectral channel bilateral filter kernel parameters and The parameter space was automatically explored using Bayesian optimization. The Bayesian optimizer initially explored 5 random points and iterated 20 times, obtaining the optimal parameters by minimizing the mean square error. Hyperspectral images processed with total variational regularization and bilateral filtering using the optimal parameters showed a reduced mean square error to 32.92, an improved peak signal-to-noise ratio to 39.24, and an increased spectral similarity of the spectral curves of each pixel to 0.9889.

[0077] like Figure 5 As shown, a spectral channel is randomly selected. (a) is the original target image; (b) is the predicted image obtained by model reconstruction of the detector response; (c) is the result after applying total variation regularization to the predicted image only; (d) is the result after applying bilateral filtering to the predicted image only; and (e) is the result obtained after the predicted image is jointly optimized by spatial-spectral processing of total variation regularization and bilateral filtering.

[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spectral reconstruction method for a broadband filtered modulation spectral detection system, characterized in that, Includes the following steps: Step S1: Perform data preprocessing and construct a training dataset for the spectral reconstruction network: Select one or more hyperspectral images containing all types of detected targets, split the hyperspectral images into spectral curves in units of pixels, process the spectral curves according to the transmission curves of the filter arrays of each channel used in the broadband filter modulation spectral detection system and the detector response curves, add noise, and convert the spectral curves into multi-channel aliasing signals to simulate the response of the spectral signal of the detected target to the pixels of the multi-channel detector. The training dataset is composed of the spectral intensity of the band to be reconstructed at each pixel and the corresponding detector response. Step S2, build a deep learning-based spectral reconstruction network model: First, construct a Transformer-based regression network, use an embedding mapping layer to map the input signal to a high-dimensional space to extract feature information, combine a multi-layer Transformer encoder to dynamically calculate the correlation between different features, automatically focus on key information, replace the FFN module in the Transformer encoder with a KAN network, and finally use a dimension-compressed KAN network as the regression head to form a KAN regression head. Through the local adaptability of the spline basis function of the KAN regression head, accurately regress the spectral data and output the predicted values ​​of the spectral intensity of each band. Step S3: Train the spectral reconstruction network. Use the detector response in the dataset as the network input and the spectral intensity of each band as the network output. Before training, initialize the data and divide the initialized dataset into training set, test set, and validation set for training, testing, and validation of the neural network, respectively. Train the built model using the mean squared error (MSE) as the network loss function. Use the AdamW optimizer based on cosine annealing learning rate combined with early stopping mechanism and dynamic batch size growth as the network training strategy. Store the network model parameters with the minimum loss function on the validation set and the dataset initialization parameters. Step S4, spectral reconstruction: After processing the multi-channel response obtained by the detector using the saved initialization parameters, the input is scanned point by point into the stored spectral reconstruction network, and the normalized hyperspectral information corresponding to the pixel is calculated and output. After performing inverse normalization on the output, the reconstructed spectral curves are stitched together according to the pixel arrangement of the detector to obtain the hyperspectral image data of the detected target. Step S5 involves performing spatial-spectral joint optimization on the hyperspectral image data, applying spatial bilateral filtering and total variational regularization to each spectral channel to obtain a high-precision spectral image.

2. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, In step S1, the broadband filtered modulation spectral detection system uses broadband random filters. Each filter transmits throughout the entire operating wavelength band, and the transmission curves between channels have low correlation. The overall correlation coefficient of the transmission curves between channels of the broadband filtered modulation spectral detection system is set to be less than 0.2, wherein the correlation coefficient of the transmission curves of the i-th and j-th channels is... Defined as: ， In the formula The transmission spectra of the i-th and j-th channels are respectively. Let i be the covariance of the transmission spectra i and j. for The expansion form of covariance operation, Let i and j be the standard deviations of the transmission spectra. Let be the mean values ​​of transmission spectra i and j, and be the mean correlation value of the i-th channel among the n channels of the broadband filtered modulated spectral detection system. for: ， Total correlation coefficient of n channels Defined as: 。 3. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, In step S1, the hyperspectral image, the transmission curve of each channel filter array, and the detector response curve used to construct the training dataset all contain the interval where the target band is located. When there are differences in the bands where the sampling points of the three are located, spline interpolation is used to unify the bands of the sampling points.

4. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, The noise added in step S1 includes photon noise and circuit readout noise, where the photon noise N photon The circuit follows a Poisson distribution, and the readout noise N readout It follows a Gaussian distribution.

5. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, The formula for converting the spectral curve into a multi-channel aliased signal in step S1 is expressed as follows: ， After image registration The pixel response value corresponding to the same object point in each channel is , These are the spectral channel numbers; the spectral information of the target point in each band is represented as follows: For broadband filtered modulation spectral detection systems, the broadband filter array in The broadband filter spectral transmission response corresponding to each channel is expressed as follows: The detector's spectral response efficiency is expressed as The noise generated by each channel of the detector is , The minimum wavelength at which the detector operates. Let M be the maximum operating wavelength of the detector. The spectral response integral equation is discretized with a degree of discretization of M and an equal-separation discretization method. The original equation is then expressed as: ， Where m is the number of the band to be reconstructed. The spectral intensity of each pixel in the original data corresponding to the wavelength of the band to be reconstructed is selected and together with the corresponding multi-channel detector response to form the training dataset of the spectral reconstruction network.

6. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, The embedded mapping layer consists of fully connected layers. The number of input nodes is equal to the number of detector response channels. It maps the input data to a 128-dimensional high-dimensional space to achieve high-dimensional mapping. The Transformer encoder contains more than three encoder units, each equipped with a 4-head self-attention mechanism. The standard FFN feedforward network is replaced with a KAN network, which uses cubic spline basis functions and a 5×5 interpolation grid. Feature enhancement is achieved through residual connections and layer normalization. The extracted 128-dimensional Transformer features are further processed by the KAN regression head to achieve decoding and dimensionality reduction. The number of output nodes is equal to the number of reconstructed bands.

7. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, Step S3 includes the following stages: 1) Data preprocessing Before training, the dataset is first initialized to improve model convergence speed and reduce the impact of parameter value range on parameter weights. The mean-standard deviation standardization method is used, and the calculation formula is as follows: ， in, As input to the training set, For the output of the training set, These are the mean and standard deviation of the input to the training set, respectively. These are the mean and standard deviation of the training set outputs, respectively. It is a very small constant. The input values ​​of the training set after standardization. To standardize the output values ​​of the training set after computation, the initialized dataset is divided into a training set, a test set, and a validation set for training, testing, and validation of the neural network, respectively. 2) Construct the loss function Mean squared error (MSE) is used as the loss function for network training. The calculation formula is: ， Where B is the number of samples processed in the same batch, and K represents the feature dimension of each sample. Let be the predicted value of the j-th dimension for the i-th sample. For the corresponding true value; Add a KAN regularization term to the loss function calculation. The formula is: ， in, The coefficients of the spline basis functions are represented. To find the square of the L2 norm, N is the total number of coefficients; Total loss function The calculation formula is: ， 3) Training methods Training employs the AdamW optimizer, combined with a cosine annealing strategy to dynamically adjust the learning rate. Initial learning rate and weight decay parameters are set based on task type and dataset size. Simultaneously, training efficiency and model stability are balanced by dynamically adjusting the batch size, which gradually doubles from the initial value as training progresses and has an upper limit. During training, model parameters and dataset initialization parameters that minimize the total loss of the validation set are saved and ultimately used for deployment in the spectral reconstruction task.

8. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, The point-by-point scanning in step S4 is a single response data of the multi-channel sensor. After ensuring pixel spatial alignment through image registration, the multi-channel response value of each pixel is extracted sequentially from left to right according to the row scanning order. This value is then input into the trained spectral reconstruction network, which outputs the standardized hyperspectral information corresponding to that pixel. After performing an inverse standardization operation on the output, the spectral reconstruction results of each pixel are finally rearranged into a hyperspectral image according to the pixel spatial arrangement of the original image, thus completing the spectral reconstruction of the sensor response.

9. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, The accuracy of the reconstructed spectral curve in step S4 is represented by the mean square error, peak signal-to-noise ratio, or spectral similarity, as shown in the following formulas: ， ， ， in, Mean square error, Peak signal-to-noise ratio, For spectral similarity, For predicted values, and These are the transposes of the true value and the true value, respectively. Number of bands It is the maximum value in the data. This is an inverse cosine operation.

10. The spectral reconstruction method for a broadband filtered modulation spectral detection system according to claim 1, characterized in that, Step S5 specifically includes: First, total variational regularization is performed on each spectral channel to minimize the total variation of the image gradient to smooth noise while preserving edges. The objective function of total variational regularization is: ， in, This indicates taking the minimum value, where dx is the differential symbol. This is the image after denoising. It is the original image. It is the image gradient magnitude. Control the regularization strength; After performing total variational regularization, a bilateral filter is further applied to the image of each spectral channel. The bilateral filter formula is as follows: ， in, Pixels after filtering The new value of , where e is the natural constant. It is a pixel The neighborhood, For pixels in the neighborhood, For pixels The original value, For pixels The original value, Controlling the size of the space kernel, The width of the color kernel is controlled by W, which is the normalization factor. The parameters of total variation regularization and bilateral filtering are automatically explored through Bayesian optimization, with mean squared error as the optimization objective, to automatically search for the total variation regularization weights. bilateral filter kernel parameters and The optimal combination.

Images