Portable non-imaging spectrometer calibration method and device

Abstract

This invention relates to the field of spectral measurement and instrument calibration, and particularly to a portable non-imaging spectrometer calibration method and apparatus. The system simultaneously acquires optical frequency comb, narrowband reference, compressed projection, photoacoustic, photoelectric, and environmental signals, and generates observation data through wavelet-variational mode-first norm three-level denoising. The observation data is input into a differentiable joint physical model, and the error is constructed by regularizing the optimal transmission distance and its gradient. The model parameters and covariance are updated in real time using historical parameter linear mapping and Hamiltonian Monte Carlo. When the uncertainty exceeds the threshold, the compression ratio is automatically reduced and the reference scan is encrypted to achieve millisecond-level closed-loop calibration with a peak position shift of less than 6 pm.

Description

Technical Field

[0001] This invention relates to the field of spectral measurement and instrument calibration, and in particular to a portable non-imaging spectrometer calibration method and apparatus. Background Technology

[0002] Portable non-imaging spectrometers are widely used in agricultural component monitoring, online environmental analysis, and rapid drug screening due to their lack of mechanical scanning, small size, and low power consumption. High-precision calibration is a prerequisite for ensuring the quantitative reliability of spectrometers. Traditional calibration methods mostly rely on laboratory standard light sources or grating scanning references: one type of method uses halogen lamps and standard white boards to obtain response curves, but the poor stability of the light source in the field leads to calibration drift; another type of method uses mechanical gratings to scan the reference signal wavelength by wavelength, which is time-consuming and susceptible to vibration. Improved schemes have been developed to accelerate calibration through extrapolation of narrow reference lines and compressed sensing reconstruction, but they still face three shortcomings: correction is based solely on empirical residuals or Euclidean distances, without considering wavelength shift errors, leading to unstable peak registration; process noise often uses a fixed white noise assumption, making it difficult to account for changes in dark count rate and integration time; and the compression ratio and reference scan density are fixed during the manufacturing stage and cannot be adaptively adjusted according to environmental uncertainties. Summary of the Invention

[0003] To address the numerous problems existing in the prior art, this invention provides a portable non-imaging spectrometer calibration method and device. This invention injects multi-source observations into a differentiable joint physical model after wavelet-variable mode denoising, predicts residuals using regularized optimal transmission distance metrics, and calculates gradient-driven Hamiltonian Monte Carlo updates to model parameters. If the posterior covariance exceeds the threshold, the compression ratio is reduced and the reference scan is encrypted, forming an acquisition-inference-scheduling closed loop.

[0004] A portable non-imaging spectrometer calibration method, comprising:

[0005] Optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals, and environmental signals are collected, and noise suppression processing is performed on the signals to obtain observation data.

[0006] The observed data is input into the differentiable joint physical model, which outputs the predicted sparse projection signal and the predicted photoacoustic signal, forming sparse residual vectors and photoacoustic residual vectors. Error data is generated based on the regularized optimal transmission distance and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model.

[0007] Based on the parameters and error data of the differentiable joint physical model in the previous period, the parameters of the differentiable joint physical model are predicted by using the linear mapping matrix obtained from the historical sequence of differentiable joint physical model parameters. The parameters and covariance of the differentiable joint physical model are updated by the Monte Carlo method under the constraint of process noise covariance.

[0008] The compressed projection signal is reconstructed using the updated differentiable joint physical model parameters and fused with the predicted photoacoustic signal to obtain the corrected spectrum and the corrected spectral uncertainty. After adjusting the compressed sampling strategy and narrowband reference scan density according to the covariance, the next cycle begins.

[0009] Preferably, the noise suppression process includes performing discrete wavelet transform decomposition on various types of signals, reconstructing the signals after reducing the high-frequency subband coefficients, performing variational mode decomposition and recombination on the reconstructed signals, and applying a threshold reduction based on the first norm to the reconstructed signals.

[0010] Preferably, the compressed projection signal is generated by writing a binary phase pattern on the surface of a sparsely coded superlens, the binary phase pattern being generated by a pseudo-random sequence.

[0011] Preferably, the optimal transmission distance is calculated using the Sinkhorn-Knopp iterative algorithm, which is executed until convergence under the conditions of setting the regularization coefficient and the number of iterations.

[0012] Preferably, the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model is calculated by automatic differentiation, and this gradient is written into the error data.

[0013] Preferably, the linear mapping matrix is ​​obtained by inputting the parameters of the previous period's differentiable joint physical model and the sequence of historical differentiable joint physical model parameters into the autoencoder to obtain the latent space vector, and then fitting the latent space vector using the least squares method.

[0014] Preferably, the process noise covariance is determined based on the detector dark count rate and integration time, and normal noise with consistent variance is superimposed on each parameter dimension in particle prediction.

[0015] Preferably, the Monte Carlo method is executed with a constant step size and a constant number of iterations, and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model is used as the gradient of the potential energy term.

[0016] Preferably, when the maximum diagonal element of the covariance exceeds a preset threshold, the compression ratio of the compressed sampling strategy is adjusted from the first compression ratio to the second compression ratio, and the narrowband reference scan density is increased simultaneously.

[0017] A portable non-imaging spectrometer calibration device is provided for implementing the aforementioned portable non-imaging spectrometer calibration method. The device comprises:

[0018] The acquisition module is used to acquire optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals and environmental signals, and to perform noise suppression processing on the signals to generate observation data;

[0019] The error module is used to input observation data into the differentiable joint physical model, output predicted sparse projection signal and predicted photoacoustic signal, form sparse residual vector and photoacoustic residual vector, and generate error data according to the regularized optimal transmission distance and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model.

[0020] The update module is used to predict the parameters of the differentiable joint physical model based on the parameters and error data of the previous cycle through a linear mapping matrix. Under the constraint of process noise covariance, the Monte Carlo method is used to update the parameters and covariance of the differentiable joint physical model. The compressed projection signal is reconstructed using the updated parameters of the differentiable joint physical model and fused with the predicted photoacoustic signal to obtain the corrected spectrum and the corrected spectral uncertainty. After adjusting the compressed sampling strategy and narrowband reference scan density according to the covariance, the module enters the next cycle.

[0021] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0022] By using a differentiable joint physical model and regularized optimal transmission distance gradient, we achieved closed-loop mechanistic differentiability and statistical inference; by using historical latent space linear mapping and Hamiltonian Monte Carlo, we achieved fast, global convergence update of high-dimensional parameters; and by using covariance threshold to drive the compression ratio and reference scan density self-adjustment, we achieved dynamic balance between information content and power consumption. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of the method of the present invention;

[0024] Figure 2 This is a structural block diagram of the system of the present invention. Detailed Implementation

[0025] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] like Figure 1 As shown, a portable non-imaging spectrometer calibration method includes:

[0029] Optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals, and environmental signals are collected, and noise suppression processing is performed on the signals to obtain observation data.

[0030] The portable non-imaging spectrometer calibration method of this invention first simultaneously acquires six types of raw signals during the acquisition and preprocessing stages: optical frequency comb signal, narrowband reference signal, compressed projection signal, photoacoustic signal, photoelectric signal, and environmental signal. The optical frequency comb signal consists of equally spaced absolute wavelength anchor points generated by a microring resonator, providing a wavelength reference for subsequent calibration; the narrowband reference signal uses a tunable cavity to insert controllable narrow lines across the entire wavelength band, used for interpolation constraints on the instantaneous response of the detection link; the compressed projection signal generates multiple independent projections within a single exposure using a sparsely coded superlens, providing a measurement matrix for compressed sensing reconstruction; the photoacoustic signal utilizes thin-film photoacoustic elements to convert light intensity into wavelength-varying acoustic voltage, used to compensate for weak photoelectric signal regions; the photoelectric signal is the detector's direct current response to the light under test; and the environmental signal includes external variables affecting detector drift, such as temperature, humidity, incident angle, and mechanical vibration.

[0031] After multi-source parallel acquisition, this invention performs noise suppression within the same digital processing link. First, discrete wavelet decomposition is performed on each type of signal, retaining low-frequency subband coefficients. For high-frequency subband coefficients, a soft thresholding method is used to reduce random noise, where the threshold is estimated from the median absolute deviation of the high-frequency coefficients. Let the original signal be... The wavelet basis functions are The decomposition coefficients are:

[0032] ;

[0033] After threshold processing, the result is obtained. ,in For the threshold, This represents the high-frequency subband coefficients after threshold reduction. Threshold reduction can reduce random fluctuations caused by detector dark current and suppress quantization noise generated by compressed projection in the low-intensity region.

[0034] Subsequently, variational mode decomposition was applied to the wavelet reconstructed signal, decomposing it into several intrinsic mode functions with bandpass characteristics. Modes were then selected for preservation and recombined based on the signal-to-noise ratio (SNR), thereby filtering out low-frequency biases originating from mechanical vibrations and external electromagnetic interference. Finally, a norm-based threshold reduction was performed on the recombined signal to further suppress residual spikes. The noise-suppressed observation data was stored in vector form, maintaining a one-to-one correspondence with the six signal types in terms of timestamp and wavelength dimensions, ensuring that subsequent physical models could perform multi-path fusion on the same sampling point.

[0035] The principle behind the above acquisition and preprocessing design is as follows: On the one hand, the optical frequency comb signal and the narrowband reference signal provide a common anchor point for absolute wavelength and instantaneous response, eliminating drift caused by temperature drift and device aging in portable systems; on the other hand, compressing the projection signal improves the spectral reconstruction resolution through sparse sampling, while the photoacoustic signal provides redundancy compensation for the low-response region of photoelectric signals, reducing reconstruction errors of weak absorption peaks. After wavelet-variable mode-norm 1-level denoising, the random noise power and systematic low-frequency drift of the observed data are significantly reduced, providing a high signal-to-noise ratio input for subsequent regularized optimal transmission distance calculation. Simulations show that compared to the single wavelet thresholding method, the three-level denoising integrated scheme can additionally reduce high-frequency noise energy by about 50% and compress low-frequency baseline drift to 1 / 3 of its original value, effectively improving calibration accuracy and reconstruction consistency without increasing hardware costs.

[0036] Preferably, the noise suppression process includes performing discrete wavelet transform decomposition on various types of signals, reconstructing the signals after reducing the high-frequency subband coefficients, performing variational mode decomposition and recombination on the reconstructed signals, and applying a threshold reduction based on the first norm to the reconstructed signals.

[0037] The noise suppression processing of this invention targets the simultaneous acquisition of optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals, and environmental signals by portable non-imaging spectrometers during field use. These signals have large amplitude ranges, diverse noise types, and overlapping noise sources. To maintain calibration accuracy under the condition of limited single-chip resources, this invention introduces a three-level linkage strategy of "discrete wavelet—variational mode—norm-1 threshold," which uses multi-scale analysis and adaptive threshold reduction to synergistically suppress high-frequency random noise and low-frequency drift.

[0038] Discrete wavelet transform is a multi-resolution analysis tool that can simultaneously localize signal features in both the time and frequency domains. This invention applies this method to each original signal... Discrete wavelet decomposition is performed, with the number of decomposition levels determined based on the sampling length and computational resources. The decomposition yields a set of approximate subband coefficients. and several high-frequency subband coefficients The high-frequency subband coefficients mainly contain random noise and quantization error; therefore, a soft threshold is applied to reduce the high-frequency subband coefficients before reconstruction. Let the threshold be... The processing formula is:

[0039] ;

[0040] In the above formula Here, represents the high-frequency subband coefficients after threshold reduction, and 'sign' denotes the sign function. Threshold Using the median absolute deviation estimation of high-frequency subband coefficients ensures both adaptability and avoids manual parameter tuning. After reduction, the signal is reconstructed according to the original hierarchy. At this point, the random noise energy has been significantly reduced, but low-frequency drift still exists due to slowly varying factors such as detector temperature drift and mechanical vibration, requiring further processing.

[0041] Variational mode decomposition (VMD) is an adaptive decomposition method that can decompose a signal into several bandpass intrinsic mode functions (IMFs). Each IMF is non-overlapping in the spectrum and exhibits good time-frequency concentration. This invention addresses the reconstruction of signals. Variational mode decomposition is used to obtain One intrinsic mode function By calculating the energy ratio of each intrinsic mode function and comparing it with a preset threshold, the dominant modes are selected and reconstructed, while low-frequency components that change slowly over time but are not beneficial to calibration are filtered out, thus obtaining the secondary reconstructed signal. .

[0042] Although the first two stages of processing have significantly reduced noise, occasional spikes may still exist. To eliminate these remaining spikes, this invention performs a second-stage signal reconstruction. Apply a threshold reduction based on the first norm. Set the signal window length to... Construct vectors By solving:

[0043] ;

[0044] Obtaining sparse approximations In the formula For sparsity weights, Let represent the norm. The solution is... Only the main components are retained and peaks are suppressed. These are optimization variables in the solution process, representing the relationship between the signal window vector and the optimization variables. The sparse approximation. The final output observation data. It is composed of all processed signals and is used for subsequent calculation of regularized optimal transmission distance and inference of differentiable joint physical models.

[0045] From a theoretical perspective, discrete wavelet transform provides initial suppression of random noise; variational mode decomposition separates low-frequency drift through variational optimization and bandpass constraints; and threshold reduction based on the first norm further filters out spikes using sparse priors. The three-stage sequential approach performs its respective function while maintaining complementarity, avoiding performance imbalances of a single algorithm under different noise scenarios.

[0046] Example: A compressed projection signal acquired by a portable non-imaging spectrometer was selected as the test object, with an initial peak signal-to-noise ratio (PSNR) of 28 dB. After processing with the three-stage noise suppression scheme of this invention, the PSNR increased to 40 dB, random noise power decreased by approximately 65%, and low-frequency drift decreased from 12% of the overall amplitude to 4%. In contrast, using only the discrete wavelet thresholding method only increased the PSNR to 33 dB, and low-frequency drift suppression was insufficient. This indicates that introducing variational mode decomposition and L1 threshold reduction can effectively improve the resolution of weak signals, providing high-quality residual input for regularized optimal transmission distance calculation, thereby improving the accuracy of subsequent calibration.

[0047] After three levels of noise suppression, the average number of convergence iterations for the regularized optimal transmission distance is reduced by 35%, the variance of the differentiable joint physical model parameter update is reduced by about 30%, and the uncertainty of the corrected spectrum is reduced by 25% under the same number of samples. For sudden temperature changes and mechanical vibrations common in portable scenarios, the present invention can monitor the quality of the observation data in real time and automatically adjust the threshold to achieve dynamic suppression of drift and random noise, ensuring that the non-imaging spectrometer maintains calibration consistency when deployed in multiple scenarios.

[0048] Preferably, the compressed projection signal is generated by writing a binary phase pattern on the surface of a sparsely coded superlens, the binary phase pattern being generated by a pseudo-random sequence.

[0049] Compressed projection signals are single-frame spectral compressed observations obtained by writing a binary phase pattern onto the surface of a sparsely coded superlens. The sparsely coded superlens is a diffractive planar optical element whose surface consists of an array of subwavelength-scale nanostructures. Each nanostructure allows only two equivalent phase states, corresponding to zero phase and half-wave phase, respectively. The combination of binary phase states allows the construction of a high-degree-of-freedom optical field modulation surface within a millimeter-scale area. The binary phase pattern is mapped from a uniform pseudo-random sequence. This pseudo-random sequence satisfies the statistical characteristics of unbiasedness and near-zero autocorrelation, ensuring that the columns of the measurement matrix are approximately orthogonal, thus satisfying the uncorrelated measurement condition in compressed sensing. The writing process is completed by photolithography or direct electron beam writing, after which the phase pattern is fixed to the superlens surface.

[0050] When the beam under test illuminates the sparsely coded superlens, a binary phase pattern generates a spatially coded light field within the superlens surface. This coded light field is then transmitted through a subsequent Fourier lens to the array detector surface. The array detector integrates the energy projection of the coded light field onto the detector plane into a single-pixel charge reading, forming a compressed projection signal. Because the phase delay of the binary phase pattern is consistent across different wavelengths, the compression process is completed simultaneously across the entire spectrum, allowing compressed observations across the entire band to be obtained within a single exposure period. This "full-band compression" avoids the time overhead of band-by-band integration required by traditional scanning spectrometers, enabling portable devices to complete a single spectral acquisition in milliseconds.

[0051] From a mathematical perspective, the true spectral vector is denoted as... The binary phase pattern implicitly determines the measurement matrix. The compressed projection signal is denoted as The relationship between the two satisfies:

[0052] ;

[0053] In the formula For length much smaller column vectors, This is a measurement matrix with fewer rows than columns. Because pseudo-random sequences have low cross-correlation properties, The restricted isometric condition, which approximately satisfies the requirements of compressed sensing theory, is satisfied. In the subsequent reconstruction stage, high-dimensional spectra can be recovered from a small number of projections using a norm-regularized matching pursuit or a fast iterative threshold shrinkage algorithm. To ensure... The dimension and the validity of multiplication, take ( (a column vector representing the number of discrete wavelength points) ( For single-frame compressed projection number, typically ),but and Dimensional consistency, multiplication is true; such as by batch Frame record, order , There are still .

[0054] The reason this invention chooses a binary phase pattern instead of a multi-phase pattern is that the binary structure is simpler to fabricate, has higher error tolerance, and ensures that diffraction efficiency is less affected by material dispersion. By pre-calculating the cross-correlation of measurement matrices for different pseudo-random sequences in the wavelength dimension offline, selecting the sequence with the lowest cross-correlation and burning it onto the superlens, the condition number of the measurement matrix can be increased without increasing the chip area. In practical applications, the compression ratio can be adjusted according to task requirements. If the goal is high-resolution component identification, the compression ratio can be set lower to improve spectral information retention; if the goal is rapid detection, the compression ratio can be set higher to reduce the number of reads.

[0055] Example: When detecting the absorption spectrum of a mixed gas, a binary phase pattern with a compression ratio of 1 / 4 was used, and the single-frame exposure time was maintained at 20 milliseconds. The spectrum was reconstructed using an alternating direction multiplier method. Experimental results show that, under a signal-to-noise ratio of 27 dB, the average relative error between the reconstructed spectrum and the true spectrum remains within 2%, and the absorption peak position error does not exceed 0.05 nanometers. Compared with the traditional transmission grating scanning reconstruction method, the acquisition time is shortened to 1 / 10 of the original, while reducing the systematic error caused by mechanical modulation.

[0056] The pseudo-random nature of the binary phase pattern also provides anti-crosstalk capability: even with incident angle offset, the columns of the measurement matrix maintain low correlation, ensuring stable reconstruction. Vibrations and tilts commonly encountered in portable environments can change the position of the light spot on the superlens surface. This invention avoids systematic reconstruction deviations by incorporating angle sensitivity constraints during the pattern design stage, ensuring that the optical path differences remain pseudo-randomly distributed as the projection changes with the angle.

[0057] Through the above design, this invention replaces multiple scans with a single compressed measurement, significantly reducing the size and power consumption of the hardware scanning mechanism, making it suitable for handheld or drone-mounted scenarios. In the overall calibration process, the compressed projection signal, as one of the main inputs to the optoelectronic link, participates in physical model inference along with the optical frequency comb signal and narrowband reference signal, and recovers high-dimensional spectral information through reconstruction operators during the fusion stage. Combined with redundancy compensation for the photoacoustic signal, calibration accuracy can be maintained even under weaker spectral sparsity conditions or lower signal-to-noise ratios, improving the applicability of portable non-imaging spectrometers in complex field environments from a system-level perspective.

[0058] The observed data is input into the differentiable joint physical model, which outputs the predicted sparse projection signal and the predicted photoacoustic signal, forming sparse residual vectors and photoacoustic residual vectors. Error data is generated based on the regularized optimal transmission distance and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model.

[0059] The differentiable joint physics model is the core unit of this invention, which balances mechanistic interpretability and real-time dominance in portable scenarios. This model uses a set of differentiable functions to simultaneously characterize the effects of detector quantum efficiency, optical transmittance, electronic gain, dark noise, acoustic gain, and acousto-optic absorption coefficient on the spectral measurement link, ensuring that each measurement remains continuously differentiable in the parameter space, facilitating subsequent gradient-based Bayesian inference.

[0060] The model uses wavelength as the independent variable, and the predicted observations for the photoelectric and photoacoustic channels are written as follows:

[0061] ;

[0062] ;

[0063] in To predict photoelectric signals, To predict photoacoustic signals, This is the true spectrum; For electronic gain; For detector quantum efficiency; Light transmittance; This is dark noise; For acoustic sensitivity gain; The acousto-optic absorption coefficient; This represents the background noise of the acoustic sensor. These parameters vary slowly with temperature and humidity, but remain approximately constant within a single sampling window, thus ensuring the model's differentiability on a millisecond timescale.

[0064] The overall difference between the predicted and observed signals is measured using a regularized optimal transmission distance metric. Let the sparse projection measurement vector be... The photoacoustic measurement vector is The corresponding prediction vector is and First, construct the sparse residual vector. and photoacoustic residual vector After nonnegative normalization, the optimal transmission distance is calculated using Sinkhorn–Knopp iteration:

[0065] ;

[0066] in and To normalize the weights, For regular kernel, For wavelength difference, is the regularization coefficient. Regularization of the optimal transmission distance not only quantifies amplitude error but is also sensitive to peak position shift, thus outperforming the traditional L2 norm.

[0067] Differentiable joint physical models are implemented within an automatic differentiation framework, allowing direct differentiation of... Find the partial derivatives. Let the model parameter vector be... The gradient is:

[0068] ;

[0069] The first two terms in the formula are explicitly expressed by the Sinkhorn iteration, while the last two terms are automatically calculated by the chain rule. The gradient calculation complexity is linearly related to the measurement dimension and can be completed in real time on an on-chip floating-point unit.

[0070] Error data is composed of scalars with vector Composition. Scalar The energy term, used as an importance weight, assesses the reliability of the current model's predictions; vector The gradient-driven Monte Carlo update uses the potential energy term to search for the posterior optimal parameters in a high-dimensional space. Through error data feedback, this invention completes parameter updates and covariance shrinkage every 33ms sampling window, enabling the system to maintain calibration accuracy even under rapid environmental fluctuations.

[0071] Example 1: In a laboratory light source intensity gradient decay scenario, the true spectrum contains 4 absorption peaks. Without correction, direct inference results in an average peak shift of 24 pm. After applying the regularized optimal transmission distance of this invention for 30 iterations, the peak shift is reduced to 6 pm. The variance of the electronic gain parameter decreases from the initial 8% to 2%, reflecting a significant improvement in model convergence speed.

[0072] Example 2: In an outdoor relocation test, the experiment lasted 30 minutes, with an ambient temperature change of 8°C and humidity fluctuations of 25%. The system monitored the covariance in real time and triggered two compression ratio adjustments, reducing the compression ratio from 1 / 16 to 1 / 8. After adjustment, the reconstruction error remained within 3%, verifying that the error data-driven adaptive strategy can effectively stabilize the performance of the portable spectrometer under conditions of drastic temperature and humidity changes.

[0073] By injecting observational data into a differentiable joint physical model and constructing gradient error data using regularized optimal transmission distance, this invention achieves a closed loop between the mechanistic model and data-driven inference, ensuring that portable non-imaging spectrometers can obtain metrological-grade spectral reconstruction accuracy without an external standard source, and providing an feasible solution for rapid on-site calibration.

[0074] Preferably, the optimal transmission distance is calculated using the Sinkhorn-Knopp iterative algorithm, which is executed until convergence under the conditions of setting the regularization coefficient and the number of iterations.

[0075] The sparse residual vector output by the differentiable joint physical model differs from the photoacoustic residual vector in both dimension and scale. Directly using the L2 norm as a metric would fail to simultaneously reflect amplitude differences and wavelength misalignments. To evaluate both types of errors simultaneously, this invention introduces a regularized optimal transmission distance and employs the Sinkhorn-Knopp iterative algorithm to calculate this distance and its gradient in real time on the on-chip processor.

[0076] The regularized optimal transmission distance is essentially the weighted shortest transport cost between two probability quality distributions. Let the sparse residual vector be... The photoacoustic residual vector is First, take the absolute value of the two vectors and normalize them by element-wise sum to obtain the non-negative weight vector. and For wavelength sampling points and Define the cost matrix Then give the Gibbs kernel ,in is the regularization coefficient, used to balance the accuracy and numerical stability of the distance. The goal is to find the probability flow matrix. make And satisfy row and column constraints Using Sinkhorn-Knopp iteration, alternating scaling can be achieved at... Approximation by multiplying diagonal matrices Define the scaling vector. and The iterative formula is:

[0077] ;

[0078] ;

[0079] when and Simultaneously, when the value is below the threshold, it is considered convergence, and ultimately we can obtain:

[0080] ;

[0081] ;

[0082] In order to participate in gradient inference, it is necessary to calculate For the parameter vector of the differentiable joint physical model The gradient. Obtained using the chain rule:

[0083] ;

[0084] in and For the prediction vector, This represents the Hadamard product. The differentiable joint physical model has been implemented in an automatic differentiation framework. and about The partial derivatives can be obtained directly by calling the backend operator. Therefore, the time complexity of gradient calculation is O(n log n). ,exist A typical portable spectrometer with a sampling scale of no more than 256 can complete an iteration within 2ms.

[0085] To ensure real-time performance at the chip level, this invention performs three optimizations to the Sinkhorn-Knopp iteration at the hardware level. First, it implements exponential kernel operations through table lookup. First, by leveraging the discrete nature of the cost matrix values, an 8-bit lookup table is used instead of floating-point exponentiation. Second, by utilizing row and column constraints, only the scaling vector is updated after each iteration, eliminating the need to explicitly store the complete vector. The matrix's memory usage is reduced to 1 / 16 of the original implementation. Third, if the row and column residuals are detected to be less than a threshold before the preset upper limit of iteration steps is reached, the process terminates early to avoid overcomputation.

[0086] After the distance and gradient calculations are completed, the scalar As an energy term, it is added to the importance weights of the gradient vector. The gradient of the potential energy term drives the Monte Carlo update. Compared to the optimal transmission distance without a regularization term, the regularized form of the Sinkhorn-Knopp iteration has two advantages: first, it has strong numerical stability and no discreteness when calculating the gradient derivative; second, its matrix-vector form is suitable for hardware parallelization, improving the iteration throughput.

[0087] Example Comparison: The regularization coefficient was set to 0.01, and the maximum number of iterations was 30. 200 consecutive measurements of the mixed gas spectrum were performed at room temperature. When using ordinary Euclidean distance-driven inference, the average posterior variance of the parameters was 6%. After switching to the regularized optimal transmission distance, the variance decreased to 4%, the peak position residual decreased from 8 pm to 5 pm, and the number of iterations for convergence decreased from 25 to 17. Further testing in a dynamic environment with temperatures rising from 10℃ to 40℃ revealed significant mismatch in the ordinary Euclidean distance-driven model at high temperatures, while the regularized optimal transmission distance-driven model maintained a variance below 7%, demonstrating that this metric is more sensitive to wavelength drift.

[0088] In portable applications, calibration often needs to be performed during movement. Vibration and changes in the incident angle can cause slight misalignment in both the compressed projection and the photoacoustic channel. Traditional Euclidean distance provides insufficient gradient information when the peak position is misaligned, easily leading to particle filtering converging to local extrema. The optimal transmission distance is regularized using the cost matrix. By tracking the transport cost along the wavelength dimension, a directional derivative is provided for each dimension, thereby guiding Bayesian inference to avoid spurious peaks along a physically interpretable path. Experimental results show that, under slight vibration conditions, the spectral error reconstructed using the metrics of this invention is improved by 28% in stability, further verifying its robustness to line shifts.

[0089] In summary, this invention utilizes the Sinkhorn-Knopp iterative algorithm to calculate the regularized optimal transmission distance and obtain the gradient in real time, unifying the measurement of amplitude error and peak position drift, and improving computational efficiency through hardware-friendly matrix-vector operations. Combined with a differentiable joint physics model, this metric provides highly sensitive and stable error feedback in the calibration link of portable non-imaging spectrometers, providing reliable gradients for subsequent Monte Carlo updates, significantly shortening parameter convergence time and improving reconstruction accuracy.

[0090] Preferably, the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model is calculated by automatic differentiation, and this gradient is written into the error data.

[0091] Portable non-imaging spectrometers are frequently moved under complex external conditions, causing the characteristics of optical links, detectors, and acoustic sensors to drift slowly with temperature, humidity, and mechanical vibration. To maintain calibration accuracy in real time without relying on an external standard source, this invention jointly maps the acquired sparse residual vector and photoacoustic residual vector to a regularized optimal transmission distance. Then, an automatic differentiation framework directly calculates the gradient of this distance with respect to the parameters of the differentiable joint physical model, and writes the gradient into the error data, driving the particle filter to converge rapidly in the high-dimensional parameter space. The regularized optimal transmission distance not only measures the residual amplitude but also considers wavelength registration offset, thus still generating effective directional derivatives even with slight peak shifts.

[0092] The differentiable joint physical model uses wavelength as the independent variable to construct the predicted photoelectric signal and the predicted photoacoustic signal as follows:

[0093] ;

[0094] ;

[0095] in For electronic gain, For quantum efficiency, For transmittance, Dark noise, For acoustic sensitivity gain, The acousto-optic absorption coefficient, This represents the noise floor of the acoustic sensor. Parameter vector. It exhibits a slow drift with temperature and humidity and can be considered as a constant to be estimated.

[0096] Sparse residual vector With photoacoustic residual vector After normalizing to a probability mass distribution, the regularized optimal transmission distance is obtained through Sinkhorn–Knopp iteration:

[0097] ;

[0098] in The optimal flow matrix, For wavelength difference, and The sampling wavelength is used. The automatic differentiation framework uses the chain rule to calculate:

[0099] ;

[0100] in and These are the predicted sparse projection vector and the predicted photoacoustic vector, respectively. and Explicitly given by the Sinkhorn iteration, and It is obtained by backpropagation in the graph structure using the automatic differentiation framework.

[0101] The calculated gradient vector dimension is consistent with the parameters of the differentiable joint physical model, and can be directly written into the error data, along with the scalar distance, for the particle filter to update the weights.

[0102] In this embodiment, the sampling window is 33ms, and the signal dimension is 128. The chip-side uses a floating-point core to implement Sinkhorn iteration and gradient backpropagation in parallel, with a single gradient calculation taking 1.8ms. For a sample containing four narrow absorption peaks, 200 frames of measurements were performed. Without writing the gradient, the effective particle count dropped to 30% after 15 frames, and the average peak position drift was 24 pm. After writing the gradient, the effective particle count remained above 75%, the peak position drift decreased to 6 pm, and the posterior variance contracted by 60%. In a temperature step experiment, the ambient temperature increased from 15℃ to 40℃, and the system's real-time trigger compression ratio was adjusted from 1 / 16 to 1 / 8. Gradient-driven model converged again within 5 iterations, and the uncertainty of the corrected spectrum remained below 3%.

[0103] This invention efficiently calculates gradients through automatic differentiation, transforming the regularized optimal transmission distance from a simple evaluation index into a potential energy function that can drive parameter updates. This significantly improves the sensitivity of Bayesian inference to the combined peak position drift and amplitude error, providing key support for the rapid and quantitative calibration of portable non-imaging spectrometers without external standard sources.

[0104] Based on the parameters and error data of the differentiable joint physical model in the previous period, the parameters of the differentiable joint physical model are predicted by using the linear mapping matrix obtained from the historical sequence of differentiable joint physical model parameters. The parameters and covariance of the differentiable joint physical model are updated by the Monte Carlo method under the constraint of process noise covariance.

[0105] The compressed projection signal is reconstructed using the updated differentiable joint physical model parameters and fused with the predicted photoacoustic signal to obtain the corrected spectrum and the corrected spectral uncertainty. After adjusting the compressed sampling strategy and narrowband reference scan density according to the covariance, the next cycle begins.

[0106] The posterior parameter vector obtained by the differentiable joint physical model in the previous sampling period is denoted as . To provide an adaptive acquisition configuration before the start of the next cycle, a short-term prediction of this parameter is required. This invention completes the prediction using an "encoding-linear propagation-decoding" structure, where the encoder... Mapping high-dimensional parameters to low-dimensional latent space vectors Linear mapping matrix This is obtained by least-squares fitting of the latent vector sequence from the most recent several periods. The prediction formula is:

[0107] ;

[0108] superscript For particle indexing, For decoder, Representing process noise, covariance matrix The formula is derived from the dark count rate and integration time and is updated synchronously with changes in temperature and humidity. This formula achieves a priori extensions to all differentiable joint physical model parameters, including electron gain, quantum efficiency, transmittance, dark noise, acoustic gain, and acousto-optic absorption coefficient, and avoids particle degradation caused by traditional random walks through linearized propagation.

[0109] After prediction, this invention employs a Monte Carlo method based on Hamiltonian dynamics to correct the particle state under process noise covariance constraints. Kinetic energy is generated by the momentum vector of unit covariance, and potential energy is derived from the regularized optimal transmission distance and its gradient. For example, the... The potential energy of each particle is ,gradient The result has been obtained through automatic differentiation. After several iterations of Leapfrog integration, a Metropolis acceptance decision is performed on the new state, which can increase the number of effective samples while maintaining the high-dimensional relevance structure. The weighted average of the particle ensemble is the posterior parameter vector for the current period. covariance Then it is calculated using the weighted second moment.

[0110] After obtaining the posterior parameters, it is necessary to reconstruct the compressed projection signal and fuse it with the photoacoustic prediction signal. Let... For sparse projection measurement vectors, the measurement matrix Determined by the binary phase pattern. A fast iterative threshold shrinkage solution is used:

[0111] ;

[0112] Solving for sparse reconstructed spectra Photoacoustic prediction signal Differentiable joint physical model with new parameters The result was obtained from the forward calculation. Based on the photoacoustic absorption coefficient... With acoustic gain The photoacoustic compensation term is derived, and the final corrected spectrum is:

[0113] ;

[0114] in For the weight function, This represents the background noise of the acoustic sensor. The uncertainty of the correction spectrum propagates through linear error:

[0115] ;

[0116] in To measure the noise covariance. If If the threshold is exceeded, the system immediately increases the compressed sampling density or narrowband reference scan density to provide more information for the next cycle, thus forming a closed loop.

[0117] In principle, the linear mapping matrix utilizes the smoothness properties of the latent space to extrapolate parameters in a short time, avoiding the weight collapse common in particle filtering. The process noise covariance is calculated based on physical limits, introducing environmental dependence to ensure prior diversity. The Monte Carlo method allows for a larger step size under gradient guidance without reducing the acceptance rate, thus reducing the number of particles required. Reconstruction and fusion employ a complementary weighting strategy, with sparse projection preserving high-frequency details and the photoacoustic channel enhancing low signal-to-noise ratio regions. Covariance-driven adaptive sampling automatically densifies observations during high uncertainty stages, saving power and accelerating convergence.

[0118] Example: The spectrometer of this invention was installed on a mobile platform and operated continuously for 1 hour, during which the temperature varied by 12°C and the mechanical shock peak was 6g. The system triggered the covariance threshold 5 times, automatically adjusting the compression ratio dynamically from 1 / 16 to 1 / 8 and then back to 1 / 12. Compared with the fixed compression ratio scheme, the dynamic scheme reduced the average absolute error of the calibrated spectrum by 28%, increased the Monte Carlo effective particle number by 42%, and increased the algorithm power consumption by less than 5%. Further verification in a standard laboratory black cavity showed that the wavelength uncertainty of the calibrated spectrum was better than 0.03nm, meeting the requirements for quantitative absorption measurement in the field.

[0119] By introducing historical condition linear mapping, process noise constraints, and gradient-driven Monte Carlo updates, this invention achieves high-dimensional adaptive estimation of parameters for portable non-imaging spectrometers. By combining sparse reconstruction, photoacoustic compensation, and covariance feedback sampling, the convergence time is significantly shortened and the accuracy of field calibration and system robustness are improved.

[0120] Preferably, the linear mapping matrix is ​​obtained by inputting the parameters of the previous period's differentiable joint physical model and the sequence of historical differentiable joint physical model parameters into the autoencoder to obtain the latent space vector, and then fitting the latent space vector using the least squares method.

[0121] In the rapid calibration process of portable non-imaging spectrometers, the parameters of the differentiable joint physics model exhibit short-term stable and long-term slowly varying characteristics with temperature, humidity, and mechanical shock. Directly using random walks or conventional Kalman first-order predictions will lead to a rapid decrease in the effective number of particles, thereby reducing Monte Carlo update efficiency. This invention uses an "autoencoder + linear mapping matrix" method to extrapolate and predict parameters, maintaining the model's ability to express nonlinear coupling relationships while ensuring the predictions maintain a mathematically linear structure, facilitating subsequent covariance analytical propagation.

[0122] First, the posterior parameter vector of the previous period and A matrix is ​​formed by concatenating historical posterior parameter sequences. , For parameter dimensions. Encoder It consists of a three-layer fully connected network, and the activation function is a linear rectified one without saturation, thus ensuring differentiability over a large dynamic range. The encoder outputs a latent vector. ,in The training objective of an autoencoder is to minimize the reconstruction error.

[0123] ;

[0124] In the formula For decoder, This represents the Frobenius norm. The encoder compresses the original high-dimensional physical meaning coupled with parameters into the latent space, preserving the main directions of change and filtering out measurement noise.

[0125] During the online phase, in response to the recent Latent vectors The least squares method is used to map the latent space using a single linear mapping matrix. ,satisfy The explicit solution is:

[0126] ;

[0127] in This represents the submatrix with the last row removed. This represents the submatrix with the first row removed. To avoid numerical instability issues in matrix inversion, this invention calculates... Add a small diagonal regularization term at the time. Linear mapping matrix. It can be viewed as a Koopman approximation of latent space evolution, which can capture the dominant drift direction of parameters within a small timescale.

[0128] The prediction step will use the latent vector from the previous period. Left multiplication Obtain the prior latent vector The prior parameters are then recovered by the decoder. Simultaneously, process noise is added. Obtain particle prediction values Process noise covariance Dark count rate of the detector With integration time Estimation, in which variance is set for each parameter component. To ensure that the prediction does not diverge excessively under the limiting noise constraint, the prediction then proceeds to the Monte Carlo update stage.

[0129] momentum Sampling from a unit covariance normal distribution, the potential energy is defined as the regularized optimal transmission distance. Using gradients With momentum The Leapfrog integral is performed to update the sampling state, followed by a Metropolis decision on acceptance or rejection. This process achieves efficient exploration of high-dimensional posterior distributions while maintaining approximate conservation of the Hamiltonian.

[0130] Obtain the updated posterior parameters Then, using the measurement matrix Sparse spectra are reconstructed using a fast iterative thresholding shrinkage algorithm, and based on acoustic gain. With acousto-optic absorption coefficient The photoacoustic prediction signal is normalized in amplitude and fused at the wavelength level with exponential weights to obtain the corrected spectrum. Uncertainty Calculated using the linear error propagation formula. When the covariance matrix... When the maximum diagonal element exceeds the threshold, the system reduces the compression ratio from high compression (e.g., 1 / 16) to medium compression (e.g., 1 / 8) and increases the narrowband reference scan density to ensure that the amount of measurement information in the next cycle meets the target convergence rate.

[0131] Example: , latent dimension Linearization prediction of sampling window parameters was performed. A Hamiltonian iteration with a fixed number of 10 steps was completed within 33 ms, with 128 particles. A 15-minute test in a stable room temperature scenario showed that the autoencoder-based linear mapping prediction scheme converged to 2% posterior variance after 7 cycles, while random walk prediction required 20 cycles. In an external temperature jump experiment, linear mapping prediction could re-lock parameters within 3 cycles after a 5°C temperature change, keeping the peak shift of the corrected spectrum within 5 pm, while the traditional method increased the shift to 18 pm.

[0132] Historical condition linear mapping transforms the high-dimensional nonlinear parameter evolution problem into a latent space linear propagation problem, effectively improving prediction accuracy. When combined with Monte Carlo updates, the entire process achieves real-time, high-fit calibration under portable hardware constraints, providing stable and reliable spectral correction capabilities for mobile measurement platforms.

[0133] Preferably, the process noise covariance is determined based on the detector dark count rate and integration time, and normal noise with consistent variance is superimposed on each parameter dimension in particle prediction.

[0134] Portable non-imaging spectrometers require generating a priori particle set for each sampling period during particle filtering. Without a reasonable process noise covariance, the particle cloud either diverges rapidly or falls into sample degradation. This invention constructs a process noise covariance matrix based on the detector dark count rate and integration time, and adds uniformly normal noise to each dimension of the differentiable joint physical model parameters during particle prediction, thus balancing physical interpretability and numerical stability.

[0135] The dark count rate describes the random electrons produced by the detector per unit time under dark conditions, with the integration time being the exposure duration of a single frame measurement. Dark electron events follow a Poisson distribution, with both the mean and variance being the expected number of the events. Let the dark count rate be... (Unit: count / s), integration time is (unit: seconds), then the expected number of dark electrons per frame is: After approximating the discrete Poisson process as a continuous Gaussian process, the power spectral density of the optoelectronic link due to dark noise can be equivalently expressed as:

[0136] ;

[0137] In the formula This represents the dark noise power; the denominator on the right shows that the longer the exposure time, the lower the noise power per unit time. To apply dimensionlessly consistent process noise in the parameter space, this invention will... Assigned to the parameter vector of the differentiable joint physical model For each dimension, construct the diagonal covariance matrix:

[0138]

[0139] in For parameter dimensions, for An identity matrix. This ensures that the particle diffusion intensity is the same in all dimensions, and that the intensity is determined by physical limits, without relying on empirical parameter tuning.

[0140] In the particle prediction phase, the first The prior parameters of each particle are obtained through:

[0141] ;

[0142] In the formula For encoder, It is a linear mapping matrix. For decoder, .because The covariance is the same in all dimensions, and the diffusion radius of the particles along each principal axis is consistent, which can avoid biased diffusion caused by the correlation between parameters, and at the same time keep the effective number of samples of particle filtering stable.

[0143] In principle, the larger the process noise covariance, the wider the particle diffusion, covering more potential true values; the smaller the covariance, the higher the particle concentration, which accelerates convergence. This invention directly anchors the covariance to the physical limit of dark counting, neither relying on on-site adjustments nor requiring sensitive automatic scaling with changes in integration time. For example, when the luminous flux decreases and the integration time needs to be extended, Follow As the particle cloud radius increases and decreases, it naturally contracts, preventing excessive divergence; in strong light fields, it shortens the exposure time. Increase the size of the particle cloud and widen it appropriately to maintain prior coverage.

[0144] Example: Using a dark count rate of 150 count / s and an integration time of 10ms, substituting into the formula yields... With a particle count of 128, the effective particle number remains steadily above 85; if... With the empirical value of 10 fixed, the effective number of particles dropped to 45, resulting in sample degradation. Further adjustments were made to the integration time to 5 ms. The dynamic update was set to 60, and the Monte Carlo acceptance rate remained at 0.8. Compared with the fixed covariance scheme, the auto-scaling scheme reduced the posterior variance by 25% and reduced the peak position shift of the corrected spectrum by 9 pm, verifying the positive effect of physical constraint covariance on the stability and calibration accuracy of particle filtering.

[0145] This invention utilizes the calculation formula of the detector dark count rate and the embedded process noise covariance within the integration time to directly control the prior particle diffusion intensity by physical limits, avoiding the problem of empirical thresholds being poorly adapted to different light flux scenarios. Simultaneously, the addition of uniformly varianced normal noise to each dimension facilitates hardware vectorization, enabling unified processing of high-dimensional parameters and ensuring rapid, adaptive convergence of the calibration link for portable non-imaging spectrometers deployed in multiple scenarios.

[0146] Preferably, the Monte Carlo method is executed with a constant step size and a constant number of iterations, and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model is used as the gradient of the potential energy term.

[0147] Portable non-imaging spectrometers need to perform high-dimensional inference of differentiable joint physical model parameters within extremely short sampling windows when operating in the field. If relying solely on random walk particle filtering, sample degradation occurs as the dimensionality increases; while traditional gradient descent, although converging quickly, struggles to guarantee global exploration of complex posterior distributions. This invention introduces Hamiltonian Monte Carlo updates into the outer loop of the particle filter, using the gradient of the differentiable joint physical model parameters with respect to the regularized optimal transmission distance as the gradient of the potential energy term, and performing leapfrog iterations with a fixed step size and a fixed number of iterations, enabling high-dimensional constrained dynamics simulations to be completed on-chip floating-point units.

[0148] In principle, Hamiltonian Monte Carlo uses parameter vectors... Introducing the momentum vector Constructing the Hamiltonian Among them, potential energy This equals the regularized optimal transmission distance, and the kinetic energy uses a unit covariance Gaussian distribution. Because... In an ideal continuous system, conservation of propagation along the Hamiltonian dynamic trajectory does not alter the target's posterior distribution, thus allowing for larger step sizes without reducing the acceptance rate. This invention uses a fixed leapfrog step size. and steps This avoids the overhead of repeatedly estimating the adaptive step size on a low-power microcontroller. The discrete update formula is:

[0149] ;

[0150] ;

[0151]

[0152] gradient The results are obtained in real time by the automatic differentiation framework, requiring no manual derivation. Update After the round, the proposal status is obtained. Determined via Metropolis:

[0153] ;

[0154] The decision to accept or reject depends on the time invertibility and symplectic structure of the Leapfrog integral. The deviation is only Therefore, it can maintain a high acceptance rate even with a fixed step size and a fixed number of steps.

[0155] This invention uses particle filter predictions as the initial position in Hamiltonian Monte Carlo simulations, with momentum sampled independently and randomly. The fixed step size and number of steps are microcontroller-friendly: gradient calculation, matrix-vector multiplication, and exponential operations in the computation pipeline can be performed using single-instruction multiple-data parallelism, and the number of iterations is known, facilitating compiler loop expansion. The potential energy gradient is taken from the regularized optimal transmission distance; compared to Euclidean distance, the contribution of wavelength displacement is explicitly reflected in the gradient, allowing the sampled trajectory to point more quickly to the true posterior high-probability region.

[0156] This invention was tested in a 128-dimensional parameter space. (Fixed) It is 0.003. With a step size of 10, 128 particles can be updated within a 33ms sampling window per frame, with an average acceptance rate of 0.82. If the adaptive step size Noura scheme is used, multiple backtracking trials are required internally, increasing the time per frame by 1.7 times. If the classic random walk is used, the number of non-convergent rounds increases by 3 times and the number of effective samples decreases to 40%.

[0157] Example: A colorimetric solution containing four absorption peaks was continuously measured for 600 frames. Using the fixed-step Hamiltonian Monte Carlo scheme of this invention, the average peak position shift was 5 pm, with a corrected spectral uncertainty of 2.4%; the random walk scheme resulted in a peak position shift of 18 pm and an uncertainty of 6.3%. In a scenario with 4g vibration of the moving platform, the fixed-step scheme only required 6 frames to reconverge, while the random walk required 20 frames, demonstrating that gradient-driven sampling is more sensitive to environmental disturbances.

[0158] This invention achieves consistent, fast, and convergent particle updates on low-power hardware by directly using the regularized optimal transmission distance gradient as the potential energy gradient, combined with Hamiltonian Monte Carlo with constant step size and constant iteration number, providing efficient and quantifiable uncertainty control capabilities for portable non-imaging spectrometers.

[0159] Preferably, when the maximum diagonal element of the covariance exceeds a preset threshold, the compression ratio of the compressed sampling strategy is adjusted from the first compression ratio to the second compression ratio, and the narrowband reference scan density is increased simultaneously.

[0160] The adaptive sampling mechanism of this invention uses the largest diagonal element of the posterior covariance matrix as the core indicator for quantifying uncertainty. When the posterior covariance matrix... In the After each sampling window is calculated, the system first extracts the diagonal vector. .like Greater than the preset threshold ,Right now If the current parameter estimation is deemed unreliable, it is necessary to reduce the uncertainty by increasing the amount of information. This invention simultaneously increases the amount of information through two independent pathways: compression ratio and narrowband reference scan density, taking into account the measurement quality in both regions with sufficient signal energy and weak signal regions.

[0161] Compression ratio is defined as the ratio of the number of rows in the measurement matrix to the dimension of the true spectrum, denoted as . A lower compression ratio results in more projections per frame, ensuring higher constraints on compressed sensing reconstruction; however, an excessively low compression ratio will lower the frame rate and increase power consumption. This invention configures the compression ratio in discrete increments, for example... When the adaptive event is triggered, the compression ratio changes from the current setting. Switch to a lower gear The formula can be written as:

[0162] ;

[0163] in This is a set of discrete compression ratios. When... At this time, the system maintains the current compression ratio and does not need to be reduced.

[0164] Narrowband reference scan density is defined as the proportion of the number of narrowband references inserted per frame to the total number of sampling points in the full band, denoted as . The narrowband reference is a narrow-line output of a tunable cavity. By varying the scan density, more collimation anchors can be inserted in different wavelength regions, improving wavelength registration. After an adaptive event is triggered, the scan density increases proportionally. promote:

[0165] ;

[0166] in This represents the maximum density allowed by the hardware. Once the scan density reaches its limit, only the compression ratio is adjusted, and it is no longer increased. .

[0167] To prevent frequent switching under noise spikes or transient disturbances, the system is equipped with a cooling counter. After the event is triggered, the cooldown counter is set to an integer. During the period when the cooling counter is greater than zero, even If the threshold is exceeded again, the adjustment is not repeated; instead, the counter is decremented. This avoids repeated adjustments during the transition phase that could cause oscillations in the sampling strategy.

[0168] The principle behind the above adjustment logic can be explained from the perspective of information gain. Taking compression ratio as an example, the number of rows in the measurement matrix is ​​determined by... Increase to Subsequently, the column space of the measurement matrix expands, reducing the noise amplification factor of the reconstruction operator and leading to an overall contraction of the posterior covariance matrix. For sparse sensing models, the prior is a norm-1 constraint; as the number of observation equations increases, the reconstruction results become more certain, thereby reducing... Diagonal elements. Similarly, increasing the narrowband reference scan density increases the number of wavelength anchor points, which helps suppress peak position drift errors.

[0169] Example 1: The system was run at room temperature steady state with an initial compression ratio of 1 / 16 and a narrowband reference scan density of 0.2. The first 10 frames of the test... Always lower The system maintains its original sampling configuration. A slight mechanical perturbation is then applied to the optical path, shifting the peak position by 12 pm. In the 11th frame, the diagonal element of the posterior covariance rises above the threshold, triggering adaptation, reducing the compression ratio to 1 / 12, and increasing the scan density to 0.3. Adjustments begin in the 14th frame. Recovery below the threshold.

[0170] Example 2: As the outdoor ambient temperature slowly increases between 20°C and 40°C, the detector's quantum efficiency decreases with temperature. The system triggers an adaptive adjustment at frame 40, reducing the compression ratio from 1 / 12 to 1 / 8. The temperature continues to rise, but the light flux remains sufficient; no further adjustments are made in the subsequent 100 frames, demonstrating that the adaptive strategy does not switch frequently.

[0171] The effectiveness of adaptive sampling was evaluated through comparative experiments. The baseline scheme with a fixed compression ratio of 1 / 16 and a fixed scan density of 0.2 exhibited a peak position standard deviation of 15 pm under environmental interference. Using the adaptive scheme, the peak position standard deviation decreased to 7 pm, and the average relative error of the reconstructed spectrum decreased from 4.5% to 2.8%. While each reduction in compression ratio lowers the frame rate, the faster covariance convergence results in a 25% reduction in the time required for the overall system to reach the same uncertainty within 100 frames compared to the fixed scheme.

[0172] Regarding power consumption, the adaptive scheme increases the number of measurement rows and reference scans, which raises the power consumption per frame, but the trigger frequency is limited by the threshold and cooling counter. Overall statistics show that in a 1-hour experiment with three temperature variations, the power consumption difference between the adaptive and fixed schemes was less than 8%, while the calibration accuracy was significantly improved, indicating that the adaptive overhead was completely under control.

[0173] like Figure 2 As shown, a portable non-imaging spectrometer calibration device is used to implement the portable non-imaging spectrometer calibration method described above. The device includes:

[0174] The acquisition module is used to acquire optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals, and environmental signals, and performs noise suppression processing on these signals to generate observation data. The acquisition module consists of six synchronous sampling channels. The optical frequency comb and narrowband reference are each equipped with an erbium-doped fiber laser output coupling terminal, a temperature-controlled piezoelectric tuning cavity, and a coaxial photodiode. These photodiodes are connected to a transimpedance amplifier before entering a 16-bit analog-to-digital converter (ADC). The compressed projection signal passes through a binary phase superlens and falls onto a two-dimensional charge-coupled array. The array output is directly written to the on-chip SRAM via an on-chip ADC. The photoacoustic signal is converted to millivolt levels by a thin-film piezoelectric element, amplified by low noise, and filtered by a bandpass filter before entering the ADC. The photoelectric signal uses a discrete avalanche photodiode with a programmable gain amplifier at the front end. The environmental signal is collected by a digital temperature and humidity sensor and a triaxial accelerometer. Clocks for all channels are allocated by a programmable crystal oscillator to ensure consistent sampling phase. The acquisition FPGA uses an embedded wavelet hard core to perform high-frequency threshold reduction and primary reconstruction; variational mode decomposition and norm 1 thresholding are performed on-chip by a tightly coupled digital signal processor, and the output observation data is written to shared DDR.

[0175] The error module is used to input observation data into the differentiable joint physical model and output predicted sparse projection signals and predicted photoacoustic signals, forming sparse residual vectors and photoacoustic residual vectors. It generates error data based on the regularized optimal transmission distance and its gradient relative to the parameters of the differentiable joint physical model. The error module runs in a co-processor architecture of an on-chip RISC-V core and a tensor coprocessor. Observation data is fed into a vector multiply-accumulate array via DirectMemoryAccess to achieve forward inference of the differentiable joint physical model. The core floating-point unit simultaneously calculates the sparse residuals and photoacoustic residuals, and uses exponential lookup tables and row / column scaling units to perform Sinkhorn-Knopp iterations to obtain the regularized optimal transmission distance. The model graph embeds a backward operator during automatic differentiation framework compilation, and the coprocessor backpropagates the gradient in the same stream. Finally, the distance and gradient are packaged together as error data and stored in on-chip static random access memory.

[0176] The update module, based on the differentiable joint physical model parameters and error data from the previous cycle, predicts the differentiable joint physical model parameters using a linear mapping matrix. Under process noise covariance constraints, it updates the differentiable joint physical model parameters and covariance using a Monte Carlo method. The updated differentiable joint physical model parameters are then used to reconstruct the compressed projection signal and fuse it with the predicted photoacoustic signal to obtain the corrected spectrum and its uncertainty. The compressed sampling strategy and narrowband reference scan density are adjusted according to the covariance before proceeding to the next cycle. The update module uses an on-chip 8MB high-speed static random access memory to store the parameter sequence for nearly 50 cycles. Both the encoder and decoder are mapped to a vector multiply-accumulate array; the linear mapping matrix is ​​updated in real-time using an on-chip least-squares inversion hard key. During the particle prediction stage, the predicted parameters are written into the hardware-laid... A register array is used, and covariance-derived white noise is superimposed on the vector multiply-add array. Hamiltonian Monte Carlo uses a tensor coprocessor to execute a fixed 10-step Leapfrog, with gradient data directly taken from the error module output. After parameter updates, the update module calls the sparse reconstruction acceleration kernel within the same vector path to complete compressed projection inversion, and mixes it with the photoacoustic prediction signal at the register level according to weights to obtain the corrected spectrum. The covariance monitoring logic compares the maximum diagonal element with a threshold in real time; if the limit is exceeded, it is then... The digital micromirror array is controlled to lower the compression ratio, while the tunable cavity stepper controller is simultaneously adjusted to increase the scan density. The entire update process is completed in a closed loop within a 33ms sampling interval, ensuring that the portable device maintains calibration accuracy and low power consumption during field operation.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0178] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method of calibrating a portable non-imaging spectrometer, the method comprising: include: The system acquires the following signals: optical frequency comb signal generated by micro-ring resonator and used as absolute wavelength anchor point; narrowband reference signal formed by inserting controllable narrow lines into tunable cavity across the entire wavelength band and used to constrain the instantaneous response of the detection link; compressed projection signal formed by writing binary phase pattern generated by pseudo-random sequence on the surface of sparse coded superlens; photoacoustic signal formed by converting the intensity of light under test by thin film photoacoustic element; photoelectric signal formed by the direct response of detector to light under test; and environmental signals including temperature, humidity, incident angle and mechanical vibration. Noise suppression processing is performed on each signal to obtain the observation data. The observation data is input into a differentiable joint physical model, which uses a set of differentiable functions to simultaneously characterize the effects of detector quantum efficiency, optical path transmittance, electronic gain, dark noise, acoustic gain, and acousto-optic absorption coefficient on the spectral measurement link. The model generates a photoelectric channel prediction signal and a prediction photoacoustic signal with wavelength as the independent variable, and forms a prediction sparse projection signal based on the measurement matrix corresponding to the binary phase pattern. Based on the compressed projection signal, the photoacoustic signal, the predicted sparse projection signal, and the predicted photoacoustic signal, a sparse residual vector and a photoacoustic residual vector are formed. The gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model is calculated by automatic differentiation. Error data is generated based on the regularized optimal transmission distance and the gradient. Based on the parameters and error data of the differentiable joint physical model in the previous period, the parameters of the differentiable joint physical model are predicted by using the linear mapping matrix obtained from the historical sequence of differentiable joint physical model parameters. The parameters and covariance of the differentiable joint physical model are updated by the Monte Carlo method under the constraint of process noise covariance. The compressed projection signal is reconstructed using the updated differentiable joint physical model parameters and fused with the predicted photoacoustic signal to obtain the corrected spectrum and the corrected spectral uncertainty. After adjusting the compressed sampling strategy and narrowband reference scan density according to the covariance, the next cycle begins.

2. The method of claim 1, wherein, Noise suppression processing includes performing discrete wavelet transform decomposition on various signals, reconstructing the signal after reducing the high-frequency subband coefficients, performing variational mode decomposition on the reconstructed signal, and applying a threshold reduction based on the first norm to the reconstructed signal.

3. The method of claim 1, wherein, The optimal transmission distance with regularization is calculated using the Sinkhorn-Knopp iterative algorithm, which is executed until convergence under the conditions of setting the regularization coefficient and the number of iterations.

4. The method according to claim 1, characterized in that, The linear mapping matrix is ​​obtained by inputting the parameters of the previous period's differentiable joint physical model and the sequence of historical differentiable joint physical model parameters into the autoencoder to obtain the latent space vector, and then fitting the latent space vector using the least squares method.

5. The method according to claim 1, characterized in that, The process noise covariance is determined based on the detector dark count rate and integration time, and in particle prediction, a normal noise with consistent variance is superimposed on each parameter dimension.

6. The method according to claim 1, characterized in that, The Monte Carlo method is executed with a constant step size and a constant number of iterations, and uses the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model as the gradient of the potential energy term.

7. The method according to claim 1, characterized in that, When the maximum diagonal element of the covariance exceeds a preset threshold, the compression ratio of the compressed sampling strategy is adjusted from the first compression ratio to the second compression ratio, and the narrowband reference scan density is increased simultaneously.

8. A portable non-imaging spectrometer calibration device, used to implement the portable non-imaging spectrometer calibration method according to any one of claims 1 to 7, characterized in that, The device includes: The acquisition module is used to acquire optical frequency comb signals, narrowband reference signals, compressed projection signals, photoacoustic signals, photoelectric signals and environmental signals, and to perform noise suppression processing on the signals to generate observation data; The error module is used to input observation data into the differentiable joint physical model, output predicted sparse projection signal and predicted photoacoustic signal, form sparse residual vector and photoacoustic residual vector, and generate error data according to the regularized optimal transmission distance and the gradient of the regularized optimal transmission distance relative to the parameters of the differentiable joint physical model. The update module is used to predict the parameters of the differentiable joint physical model based on the parameters and error data of the previous cycle through a linear mapping matrix. Under the constraint of process noise covariance, the Monte Carlo method is used to update the parameters and covariance of the differentiable joint physical model. The compressed projection signal is reconstructed using the updated parameters of the differentiable joint physical model and fused with the predicted photoacoustic signal to obtain the corrected spectrum and the corrected spectral uncertainty. After adjusting the compressed sampling strategy and narrowband reference scan density according to the covariance, the module enters the next cycle.

Images