Wavelength modulation spectroscopy database augmentation method based on physical models and field measurements

Abstract

The application discloses a wavelength modulation spectrum database capacity expansion method based on a physical model and field measurement, relates to the technical field of spectrum data processing, and comprises the following steps: S1, setting random spectrum parameters, and constructing an absorption spectrum database through simulation; S2, measuring light intensity without gas absorption, and constructing a laser baseline database; and S3, based on the absorption spectrum database and the laser baseline database, calculating a final deep learning training database through a wavelength modulation spectrum principle. The application not only maintains the advantages of the simulation database in parameter coverage range and numerical accuracy, but also integrates the characteristics of harmonic noise and distortion in a real measurement system, thereby providing a reliable data basis for effectively suppressing harmonic noise and distortion in a subsequent network.

Description

Technical Field

[0001] This invention relates to the field of spectral data processing technology, specifically to a method for expanding the capacity of a wavelength modulation spectral database based on a physical model and on-site measurements. Background Technology

[0002] Tunable semiconductor absorption spectroscopy (TMS) has been widely used in industrial production due to its advantages such as non-invasiveness, high sensitivity, selectivity, immediacy, and the ability to quantitatively measure various gases. However, in TMS measurement systems, 1 / f noise, characterized by its power spectral density being inversely proportional to frequency, typically dominates. To further improve detection sensitivity and effectively suppress 1 / f noise concentrated in the low-frequency region, wavelength modulation spectroscopy (WMS) has emerged. In WMS, by applying high-frequency modulation to the laser's injected current or temperature, the target signal related to gas absorption is boosted to the higher harmonics region of the modulation frequency, thereby achieving separation of the target signal from low-frequency 1 / f noise. Benefiting from its superior performance in detection limits and sensitivity, WMS has been widely applied in many fields, including chemical production, pharmaceutical preparation, atmospheric remote sensing, biological detection, combustion diagnostics, and spectral parameter calibration.

[0003] Although WMS has separated the main noise through modulation and achieved significant improvements in signal-to-noise ratio and measurement accuracy, the nonlinearity of electronic equipment (residual amplitude modulation of lasers, high-order harmonic distortion of electronic equipment such as detectors and analog-to-digital converter chips), the etalon effect of optical instruments (collimators, optical windows, gas cells), and the shape and amplitude distortion of harmonic signals caused by complex environmental factors (such as particulate scattering, background absorption, and temperature and pressure fluctuations) still limit the reduction of the measurement lower limit and the improvement of measurement stability of this method. Harmonic distortion generated by electronic equipment, due to its inherent characteristics and mains or electromagnetic interference, typically exhibits waveforms that change slowly over time, and its spectrum is concentrated around the modulation harmonics, severely overlapping with the harmonic signals absorbed by the target gas. This makes it impossible to effectively suppress such interference through simple long-term averaging, as with random electronic noise, and its time-varying characteristics pose a significant challenge to subsequent harmonic analysis and parameter inversion.

[0004] To address the problem of harmonic distortion, researchers have explored various methods, including optimizing the parameters of phase-locked filtering (PLB) and introducing more advanced denoising algorithms. For example, optimizing the bandwidth and order of the PLB based on harmonic extraction. However, due to the severe overlap between the harmonic distortion spectrum and the target harmonic, this approach has limited effectiveness. Furthermore, researchers have attempted to use more advanced filters, such as discrete wavelet transform, continuous wavelet transform, variational mode decomposition, and Savitzky-Golay filters, to further denoise the harmonics obtained from PLB. Because the spectrum of harmonic distortion severely overlaps with the spectrum of the absorbed signal, directly optimizing the parameters (filter order and cutoff frequency) of bandpass filters (such as Butterworth filters) makes it difficult to separate the two. In addition, while other advanced filters have achieved some success under specific conditions, these methods generally require parameter optimization for specific application scenarios (such as wavelet basis functions, scale range, number of modes, polynomial order, and window size). When harmonic distortion exhibits time-varying characteristics, pre-optimized parameters often become ineffective and need to be readjusted, significantly increasing the computational burden and system complexity. The aforementioned traditional methods are ineffective in suppressing harmonic distortion in WMS (Wideband Microsystems) because they have limited ability to handle complex nonlinear problems. These methods struggle to effectively distinguish between target harmonics generated by gas absorption and distortions caused by the instrument or environment, and are unable to learn the dynamic changes in distortion modes. Therefore, to further improve the measurement accuracy, robustness, and real-time performance of WMS in complex environments, it is urgent to explore more advanced and intelligent signal processing strategies. These strategies must effectively address the challenges of instrument nonlinear distortion and interference from complex environments, thereby enabling accurate inversion of gas parameters from contaminated harmonic signals.

[0005] In recent years, deep learning, with its powerful nonlinear mapping capabilities, automatic feature extraction capabilities, and ability to learn latent patterns from complex data, has provided new avenues for solving problems that traditional algorithms struggle with. It has demonstrated outstanding performance and gained widespread application in fields such as computer vision, natural language processing, and signal denoising. As a branch of machine learning, deep learning essentially utilizes large-scale labeled datasets to perform goal-oriented parameter optimization (such as backpropagation) on neural network models with a large number of parameters, thereby learning the complex mapping function between data input and desired output. Therefore, the effectiveness of deep learning in learning mapping relationships is mainly determined by the model architecture and its number of parameters, the size and quality (accuracy and breadth) of the training database, and the optimization algorithm for the model parameters. Currently, researchers have combined deep learning with WMS (Wireless Signal Processing) to achieve functions such as spectral denoising, inversion of measured gas parameters, and interference suppression energy. Although deep learning has shown great potential in WMS signal processing, it faces a core challenge in practical applications: model performance is highly dependent on a large-scale, high-precision training database that covers a wide range of measurement conditions and includes real measurement noise and harmonic distortion. The quality and scale of the training data directly determine the model's performance, generalization ability, and robustness to environmental disturbances. However, there are significant difficulties in constructing a WMS training database that meets these requirements.

[0006] In existing technologies, such as numerical simulations based on the WMS physical model, large-scale simulated harmonic signal databases covering a wide range of operating conditions can be generated by inputting different gas parameters (concentration, temperature, and pressure, etc.) and laser parameters (tuned light intensity and wavelength, etc.), combining the absorption spectrum calculation model, and superimposing random noise with statistical characteristics comparable to measured noise. However, the drawback is that the database needs to accurately reflect the mapping relationship between measured harmonics and real harmonics. This requires that the input data of the database be consistent with real signals that contain harmonic distortions caused by nonlinearities of electronic devices (residual amplitude modulation of lasers, nonlinearities of electronic devices such as detectors and ADCs), etalon effects of optical instruments (collimators, optical windows, gas cells), and complex environmental factors (such as particulate scattering, background absorption, and fluctuations in temperature and pressure). Because the causes of harmonic distortion are complex and change slowly over time, they are difficult to describe with models, making it difficult for simulation-built databases to truly represent real signals.

[0007] In existing technologies, it is possible to obtain real WMS signals through numerous experimental measurements using different parameters, such as temperature, concentration, and pressure, and then construct a deep learning database. However, a drawback is that for the database to broadly and accurately reflect the mapping relationship between measured harmonics and real harmonics, it requires a large number of widely applicable, accurate, and randomized operating conditions with measurement parameters (gas concentration, pressure, temperature, laser tuning range, tuning center, etc.) to prevent model overfitting. Such experimental conditions with broad and randomized measurement parameters are almost impossible to achieve. Furthermore, at low harmonic SNR, it is difficult to obtain standard theoretical harmonics and measurement parameters through harmonic measurements alone. This necessitates the use of other measurement methods more accurate than WMS for accurate database calibration. These two factors make it virtually impossible to obtain a large, widely applicable, accurate database with randomized measurement parameters.

[0008] Existing technologies, for example, use limited experimental data when constructing a database and increase its capacity through data expansion. This method directly borrows database expansion ideas from other fields, such as computer vision, scaling, translating, or adding interference to expand the database based on limited measurement data. However, using database expansion methods from other fields fails to fully integrate with the physical model of the WMS. The scaling, translating, or interference-adding operations essentially simulate harmonic variations caused by pressure fluctuations, laser drift, and harmonic distortion during the measurement process. Furthermore, direct translation and scaling do not match the actual pressure fluctuations and laser drift during measurement, ultimately leading to a certain deviation in the experimental database. Summary of the Invention

[0009] To address the problems of traditional database construction methods that fail to reflect harmonic distortion in real measurements using simulation data and fail to obtain an accurate database containing a wide range of random operating conditions using experimental data, this invention proposes a wavelength modulation spectral database expansion method based on physical models and field measurements.

[0010] The technical solution of this invention is: a method for expanding the capacity of a wavelength modulation spectral database based on a physical model and on-site measurements, comprising the following steps:

[0011] S1. Set random spectral parameters and construct an absorption spectral database through simulation;

[0012] S2. Measure the light intensity without gas absorption and construct a laser baseline database;

[0013] S3. Based on the absorption spectral database and laser baseline database, the final deep learning training database is calculated using the wavelength modulation spectral principle.

[0014] Furthermore, S1 includes the following sub-steps:

[0015] S11, set random spectrum parameters;

[0016] S12, calculate the absorption rate function corresponding to the spectrum parameters by simulation to obtain an absorption spectrum database.

[0017] Further, in S11, the spectrum parameters include integral absorbance, Doppler broadening, collision broadening, laser light intensity tuning, and laser wavelength tuning.

[0018] Further, the expression of laser wavelength tuning is:

[0019] ;

[0020] wherein, is the laser output wavelength in the experiment, is the scanning frequency, is the modulation frequency, is the phase angle of the scanning wavelength, is the phase angle of the modulation wavelength, is the scanning range, is the modulation range, is the tuning center, is a triangular wave function, is time.

[0021] Further, in S12, the expression of the absorption rate function is:

[0022] ;

[0023] wherein, is the integral absorbance, is the Doppler broadening, is the collision broadening, is the laser output wavelength in the experiment, is a voigt line function, is a Lorentz line function, is a Gauss line function, L is the gas absorption path, S(T) is the spectral line intensity, T is the temperature, P is the gas pressure, and C is the concentration of the gas to be measured.

[0024] Further, S2 includes the following sub-steps:

[0025] S21, measure the non-absorption light intensity baseline of the laser within a preset time;

[0026] S22, perform mean value normalization on the non-absorption light intensity baseline and construct a laser baseline database.

[0027] Furthermore, in S22, the expression for mean normalization of the baseline of non-absorbed light intensity is:

[0028] ;

[0029] in, , representing the normalized laser intensity The original laser intensity, For time.

[0030] The beneficial effects of this invention are: it maintains the advantages of simulation databases in terms of parameter coverage and numerical accuracy, while incorporating the characteristics of harmonic noise and distortion in real measurement systems, thus providing a reliable data foundation for subsequent networks to effectively suppress harmonic noise and distortion. Attached Figure Description

[0031] Figure 1 A flowchart illustrating a method for expanding a wavelength modulation spectral database based on a physical model and field measurements;

[0032] Figure 2 This is a schematic diagram of a method for expanding the wavelength modulation spectral database based on a physical model and field measurements.

[0033] Figure 3 (a) in the figure shows the linearity fitting result between the harmonic amplitude of the original data and the concentration of the gas to be measured;

[0034] Figure 3 (b) in the figure shows the linearity fitting result of the harmonic amplitude and the concentration of the gas to be measured using the traditional method, namely simulation to build a database training model;

[0035] Figure 3 (c) in the figure is the linearity fitting result of the harmonic amplitude and the concentration of the gas to be measured in the method proposed in this invention;

[0036] Figure 4 Allan variance plots for three methods. Detailed Implementation

[0037] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0038] like Figure 1 As shown, this invention provides a method for expanding the capacity of a wavelength modulation spectral database based on a physical model and on-site measurements, comprising the following steps:

[0039] S1. Set random spectral parameters and construct an absorption spectral database through simulation;

[0040] S2. Measure the light intensity without gas absorption and construct a laser baseline database;

[0041] S3, based on the absorption spectrum database and the laser baseline database, the final deep learning training database is calculated by the wavelength modulation spectrum principle.

[0042] As shown in Figure 2 the present application first measures the non-absorbing laser baseline on site to construct the corresponding laser baseline database; then sets random spectrum parameters corresponding to the experiment in the spectrum simulation to calculate the absorption rate linear and wavelength function database; finally, combines the above two databases and the WMS principle to calculate the laser intensity database of the through gas to be measured containing noise and the theoretical absorption rate database without noise, and uses the wavelet transform filter and the phase-locked filter method to distinguish the corresponding harmonics; the noise-containing harmonics calculated by the noise-containing transmitted light intensity are the input of the database, and the pure harmonics calculated by the noise-free absorption rate are the output of the database.

[0043] In the process of deep learning database construction, the present application proposes a wavelength modulation spectrum (WMS) database capacity expansion method based on the combination of physical model and on-site measurement (PIBA). In the field of deep learning combined with WMS, in order to ensure the effect of spectrum noise reduction of the model, the database needs to widely and accurately reflect the mapping relationship between the measured harmonics and the real harmonics, that is, it needs to have a large number of, extensive, accurate, and random working conditions of measurement parameters (gas concentration, pressure, temperature, laser tuning range, tuning center, etc.) to prevent overfitting of the model. In the experiment, it is almost impossible to measure extensive and random working conditions of measurement parameters. In addition, when the SNR of the harmonic is low, it is difficult to obtain the standard theoretical harmonic and the measurement parameter through the measurement of the harmonic. This leads to the necessity of using other more accurate measurement methods than WMS to accurately calibrate the database. These two points lead to the fact that the experimental data cannot realize a large number of extensive, accurate, and random measurement parameter databases. In addition, if the database needs to accurately reflect the mapping relationship between the measured harmonics and the real harmonics, the input data of the database needs to be consistent with the real signal containing harmonic distortion caused by the nonlinearity of electronic equipment (residual amplitude modulation of the laser, nonlinearity of the detector and ADC, etc.), the etalon effect of optical instruments (collimator, optical window, gas cell), and complex environmental factors (such as particulate matter scattering, background absorption, temperature and pressure fluctuations). Due to the complexity of the reasons for harmonic distortion, it changes slowly over time and is difficult to describe with a model, so it is difficult to realize the representation of the real signal by simulating the database. In summary of the above two points, neither simulation nor experiment alone can construct a database that meets the extensive and accurate mapping relationship between the measured harmonics and the real harmonics.

[0044] Therefore, the application proposes a method of database expansion of baseline signal fused with WMS physical model and in-situ measurement: on the basis of non-absorption laser baseline with harmonic distortion reserved, combined with simulation model which can generate a large number of, extensive, accurate and random measurement parameters, finally a database which can completely and accurately depict the mapping from measurement harmonic to real harmonic is obtained. PIBA mainly includes three modules of random absorption rate generation, wavelength tuning considering laser drift generation and long-time measurement of laser baseline.

[0045] In the embodiment of the application, S1 comprises the following sub-steps:

[0046] S11, setting random spectral parameters;

[0047] S12, obtaining an absorption spectrum database by simulating and calculating the absorption rate function corresponding to the spectral parameters.

[0048] Firstly, in the random absorption line generation module, a spectral database covering a wide range of working conditions and with random distribution of measurement parameters (concentration, temperature and pressure, etc.) needs to be generated to ensure the robustness of the model to environmental parameters (gas concentration, pressure and temperature). Specifically, when constructing the spectral absorption line database, for the working conditions of this experiment (0.4 atm, 263 K, 0~5 N2O), the range of spectral parameters is set as: integral absorbance A=0~10 -3 cm -1 , Doppler broadening =0.005~0.02cm -1 , collision broadening =0.03~0.05cm -1 . Finally, the corresponding absorption rate function is calculated by absorption spectrum theory.

[0049] In the embodiment of the application, in S11, the spectral parameters include integral absorbance, Doppler broadening, collision broadening, laser intensity tuning and laser wavelength tuning.

[0050] In the embodiment of the application, the expression of laser wavelength tuning is:

[0051] ;

[0052] wherein, is the output wavelength of the laser in the experiment, is the scanning frequency, is the modulation frequency, is the phase angle of the scanning wavelength, is the phase angle of the modulation wavelength, is the scanning range, is the modulation range, is the tuning center, is a triangular wave function, is time.

[0053] In the embodiment of the present application, the expression of the absorption rate function in S12 is:

[0054]

[0055] wherein, is the integral absorbance, is the Doppler broadening, is the collision broadening, is the laser output wavelength in the experiment, is the voigt line shape function, is the Lorentz line shape function, is the Gauss line shape function, L is the gas absorption optical path, S(T) is the spectral line intensity, T is the temperature, P is the gas pressure, and C is the concentration of the gas to be measured.

[0056] In the embodiment of the present application, S2 comprises the following sub-steps:

[0057] S21, measuring the non-absorption light intensity baseline of the laser within a preset time;

[0058] S22, performing mean value normalization on the non-absorption light intensity baseline, and constructing a laser baseline database.

[0059] In order to improve the anti-interference ability of the model to the laser output wavelength drift (such as the wavelength modulation scanning amplitude and the tuning center), the wavelength drift in the real measurement needs to be considered when constructing the simulation. Therefore, in the simulation, the randomized laser wavelength tuning parameters similar to the real measurement are introduced: the scanning range =9~11 voigt , the modulation range =2.0~2.4 voigt , and the center wavelength offset (the difference between the tuning center and the spectral line center ) =-0.03~0.03cm -1 . Thus, the wavelength tuning database capable of describing the laser wavelength drift effect is constructed.

[0060] In the embodiment of the present application, the randomized laser wavelength tuning parameters in S21 comprise the scanning range, the modulation range, and the center wavelength offset.

[0061] The phase angle ranges of the scanning and modulation wavelengths are 1.0~1.1 and 1.2~1.3 ​A corresponding random range was also set when constructing the database.

[0062] In this embodiment of the invention, in S22, the expression for mean normalization of the baseline of non-absorbed light intensity is as follows:

[0063] ;

[0064] in, , representing the normalized laser intensity The original laser intensity, For time.

[0065] To accurately characterize the harmonic distortion characteristics of the actual measurement system, long-term measurements of gas-free absorption laser intensity were used when constructing the database. In the preliminary experiments, for Long-term monitoring (>24h) was conducted. Baseline signals were acquired at 1-minute intervals, accumulating more than 1,500 types of baseline samples, with a total sample size exceeding 5E4.

[0066] This invention significantly improves the linearity between harmonic amplitude and concentration: In the N2O concentration gradient-harmonic amplitude calibration experiment, measurements were performed at five concentration gradients: 1.2%, 0.62%, 0.32%, 0.22%, and 0.125% (corresponding to peak absorbance ranges from 9.6E-4 to 1.0E-4). Each set of experiments lasted over one hour, and more than 1E5 sets of data were collected. The model trained using PIBA showed a significant improvement. Regarding random error, the fluctuation of harmonic amplitude was further reduced to 1.39E-5, a reduction of 61.71% compared to the original signal. In terms of linearity, the model... 2 The value was increased from 0.9174 to 0.9912. This optimization is directly attributed to the model's accurate identification and suppression of harmonic baselines.

[0067] This invention significantly improves the detection limit of the measurement system: The accuracy of PIBA measurement was verified in a 185-minute continuous measurement experiment of 0.67% N₂O. In the experiment, the average harmonic amplitude obtained by direct measurement was -10.75. By measuring with the gas cell evacuated and subtracting the baseline (the vertical baseline corresponding to the harmonic amplitude was -5.01), the harmonic amplitude was obtained as -5.74. This deviates from the simulation model's calculated value (-5.56) by only 3.24%. The fluctuation range of the original harmonic amplitude (half-width, half-height, half-width) of the probability distribution was 8.72, corresponding to a measurement concentration uncertainty of 0.94%. Figure 3 As shown in (a), the noise reduction model trained solely on simulation data reduced the amplitude fluctuation of the output from 8.72E-5 to 3.78E-5, a reduction of 56.65%, compared to the original harmonics.Figure 3 As shown in (b), however, due to the lack of description of the harmonic baseline in the database, the harmonic mean obtained by this model is -8.39, which deviates from the true value by 33.73%, seriously affecting the accuracy of the measurement. In contrast, the model trained based on PIBA performs better in both accuracy and stability, as shown in Figure (c). In terms of accuracy, the mean result of this model is -5.5E-5, with a deviation from the true value of only 1.08%. This is a reduction of 66.6% compared to the direct measurement result and a reduction of 96.8% compared to the model trained on simulation data. In terms of stability, this model reduces the harmonic amplitude fluctuation caused by random errors from 8.72 to 2.93, a reduction of 66.40%. This is 9.75% higher than the model trained on simulation data.

[0068] This invention significantly improves the detection limit of the measurement system. To evaluate the long-term measurement stability of the PIBA method, Allan variance analysis was performed on the original phase-locked filtering method, the deep learning model trained on a pure simulation database, and the model trained on a PIBA fusion database after 1000 consecutive measurements. Figure 4 As shown, the three methods achieved the minimum Allan variance at means of 158, 264, and 197, with values ​​of 4.77E-6, 2.00E-6, and 1.08E-6, respectively. Compared to the original phase-locked filtering method, the model based on the pure simulation database showed improved stability, with a corresponding reduction of 58.07% in the detection limit. However, the model trained on the PIBA fusion database (combining experimental baseline and physical model) exhibited a more significant performance improvement, with its detection limit reduced by 77.36% compared to the original method. Specifically, the lowest detectable N2O concentration decreased from 429 ppm to 97 ppm, corresponding to a reduction in the peak absorbance detection limit from 3.43E-5 to 7.76E-6.

[0069] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for expanding the capacity of a wavelength modulation spectral database based on a physical model and field measurements, characterized in that, Includes the following steps: S1. Set random spectral parameters and construct an absorption spectral database through simulation; S2. Measure the light intensity without gas absorption and construct a laser baseline database; S3. Based on the absorption spectral database and laser baseline database, the final deep learning training database is calculated using the wavelength modulation spectral principle.

2. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 1, characterized in that, S1 includes the following sub-steps: S11. Set random spectral parameters; S12. Obtain the absorption spectrum database by simulating and calculating the absorption rate function corresponding to the spectral parameters.

3. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 2, characterized in that, In S11, the spectral parameters include integrated absorbance, Doppler broadening, collision broadening, laser intensity tuning, and laser wavelength tuning.

4. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 3, characterized in that, The expression for the laser wavelength tuning is: ； in, The wavelength of the laser output in the experiment. For scanning frequency, For modulation frequency, The phase angle of the scanning wavelength. The phase angle of the modulation wavelength, For the scanning range, For modulation range, As the tuning center, It is a trigonometric wave function. For time.

5. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 2, characterized in that, In S12, the absorption rate function The expression is: ； in, The integral absorbance is... To broaden the Doppler, To expand for collisions, The wavelength of the laser output in the experiment. For VOOGT linear functions, For Lorentz linear functions, S is the Gaussian line shape function, L is the gas absorption optical path length, S(T) is the spectral line intensity, T is the temperature, P is the gas pressure, and C is the concentration of the gas to be measured.

6. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 1, characterized in that, S2 includes the following sub-steps: S21. Measure the baseline of the laser's non-absorbed light intensity within a preset time. S22. Normalize the mean of the non-absorbed light intensity baseline and construct a laser baseline database.

7. The method for expanding the wavelength modulation spectral database based on a physical model and on-site measurements according to claim 6, characterized in that, In step S22, the expression for mean normalization of the baseline of non-absorbed light intensity is as follows: ； in, To normalize the laser intensity, The original laser intensity, For time.

Images