Method for processing greenhouse gas fourier infrared spectrum data by principal component dynamic selection

Abstract

The application discloses a greenhouse gas Fourier infrared spectrum data processing method based on principal component dynamic selection, which comprises the following steps: after pre-processing mid-infrared Fourier time-series greenhouse gas measurement spectrum, data sets are formed by summarizing, and spectral matrices are formed by block by block; principal component decomposition is performed on each spectral matrix to obtain principal components of time variables and transmittance, and spectral matrices are reconstructed by using different numbers of principal components; by calculating the Euclidean distance and cosine distance of the reconstructed spectral matrix and the original spectral matrix, and comparing with a threshold value, the most suitable reconstructed spectral matrix is selected to reconstruct each measurement spectrum; by using a nonlinear least square regression model, the reconstructed spectrum is inversed to obtain the concentration change of the greenhouse gas component over time. The method improves the quality of the spectral data while maintaining the time resolution, meets the demand of high-precision inversion of each component, and provides an effective method for adaptive data processing of the greenhouse gas measurement spectrum.

Description

Technical Field

[0001] This invention relates to the field of mid-infrared time-series spectral data processing technology, specifically a method for processing greenhouse gas Fourier transform infrared spectral data with dynamic principal component selection. Background Technology

[0002] Since the Industrial Revolution, the types and quantities of greenhouse gases emitted by humankind have continuously increased, exacerbating serious consequences such as global warming. Therefore, accurately measuring changes in greenhouse gas concentrations is crucial, providing a strong basis for developing solutions to address climate change. Fourier transform infrared spectroscopy (FTIR) offers the advantage of simultaneous measurement of multiple components, allowing for the simultaneous measurement of CO2, N2O, and CO concentrations. However, ambient air samples exhibit numerous absorption peaks in their FTIR spectra, significant overlap, and complex band structures. Instrument and environmental noise further affect spectral quality. Therefore, processing FTIR spectral data remains a hot research topic to ensure accurate measurement of greenhouse gas concentrations.

[0003] Numerous studies have shown that appropriate data processing methods can effectively improve the quality of spectral data, facilitating accurate inversion of gas components and obtaining high-quality analytical data. Current Fourier spectral processing methods mainly include blind estimation methods based on discrete cosine regularization, stepwise spectral reconstruction methods based on Wiener filtering, and signal averaging methods. However, these methods each have their limitations. For example, blind estimation and filtering methods require substantial prior knowledge, resulting in complex data processing. Their effectiveness is closely related to parameter selection, and they cannot significantly improve the data quality of all measured spectra. Signal averaging methods can only reduce Gaussian noise in spectral data and decrease the temporal resolution of the measurement data. Therefore, an adaptive spectral processing method that does not reduce the temporal resolution of the measurement data is better suited to the needs of mid-infrared time-series spectral data processing.

[0004] Principal component analysis (PCA) is a fundamental method in chemometrics. It extracts the most important information from a set of Fourier transform mid-infrared time-series greenhouse gas measurement spectra, reducing the dimensionality of spectral data, utilizing it more effectively, improving the signal-to-noise ratio, and shortening processing time. The number of principal components determines the effectiveness of PCA applications. Typically, the amount of information explained dictates the choice of the number of principal components. However, the variance contribution from characteristic variations in Fourier transform mid-infrared time-series measurements is small, and the variance explained rate is not the optimal basis for selecting the number of principal components. Therefore, a reasonable threshold is needed to determine the number of principal components to improve the quality of spectral data in various measurement scenarios. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for processing greenhouse gas Fourier transform infrared spectral data with dynamic principal component selection. This method is based on principal component analysis and is used for processing mid-infrared Fourier time-series greenhouse gas measurement spectral data. It enables adaptive processing of mid-infrared Fourier time-series greenhouse gas measurement spectra, reduces the impact of spectral band aliasing and measurement noise on spectral quality, and meets the requirement for simultaneous high-precision inversion of each gas component.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for processing Fourier transform infrared spectral data of greenhouse gases with dynamic principal component selection is applicable to the processing of relatively stable measurement spectral data of gaseous pollutants such as O3, C2H4, CS2, CH2Cl2, C2H3C, SO2, and OCS. The method specifically includes the following steps:

[0008] Step 1: Measure the mid-infrared Fourier time-series spectral data of ambient air samples using a Fourier transform infrared spectrometer, and after preprocessing such as baseline correction, summarize the data to form a measurement dataset.

[0009] Step 2: Divide the measurement dataset into blocks to form A M1 A M2 A M3 Isospectral data matrices were generated, and each spectral data matrix was standardized. The block-based matrices retained the spectral absorption characteristics of multiple greenhouse gases, while bands that did not contain the spectral characteristics of CO2, N2O, and CO were filtered out, thereby improving the speed of principal component decomposition; A M1 A M2 A M3 Both are m×n spectral data matrices, where m is the number of measurements and n is the wavenumber range;

[0010] Step 3: Process the standardized spectral data matrix A M1 Principal component decomposition is performed to obtain the loading matrix K. M1 Load matrix K M1 It is an m×m matrix;

[0011] Step 4: Select the load matrix K sequentially. M1 The spectral data are reconstructed using the 1st principal component, 2nd principal components, ..., m-1 principal components to obtain the reconstructed spectral matrix A. R1 A R2 A R3 A Rm-1 ;

[0012] Step 5: Calculate the spectral data matrix A M1 With the reconstructed spectral matrix A R1 A R2 AR3 A Rm-1 Euclidean distance d Euclid_A and cosine distance d carbo_A ;

[0013] Step 6: Calculate the Euclidean distance d Euclid_A and cosine distance d carbo_A The reconstructed spectral matrix is ​​compared with the set threshold. Based on the comparison results, the reconstructed spectral matrix with the Euclidean distance and cosine distance closest to the threshold is selected to reconstruct each measured spectrum in order to obtain high-quality spectral data. The set threshold is obtained through numerical simulation calculation.

[0014] Step 7: Repeat steps 3, 4, 5, and 6 to process the spectral data matrix A one by one. M2 Spectral data matrix A M3 Then, complete the processing of the measurement dataset;

[0015] Step 8: Reconstruct the spectrum by using a nonlinear least squares regression model to obtain the concentration of greenhouse gas components over time.

[0016] Furthermore, the preset threshold in step six is ​​calculated using the following steps:

[0017] Step a: Based on the spectral line parameters in the database, obtain the numerical simulation spectral matrix B using numerical simulation and Monte Carlo statistical methods. M and Gaussian noise spectral matrix B with RMS noise level Ns MN The Monte Carlo statistical method is a stochastic simulation method that evaluates the properties and statistical distribution of a spectrum through repeated random trials; numerical simulation of the spectral matrix B. M For an m×n matrix, the Gaussian noise spectral matrix B MN It is an m×n matrix;

[0018] Step b: For the Gaussian noise spectral matrix B MN After standardization, the loading matrix K is obtained using principal component decomposition. MN Load matrix K MN It is an m×m matrix;

[0019] Step c: Select the load matrix K sequentially. MN Spectral reconstruction was performed on different numbers of principal components to obtain the reconstructed spectral matrix K. MR1 K MR2 K MR3 ... K MRm-1 ;

[0020] Step d: Calculate the reconstructed spectral matrix K respectively. MR1 K MR2K MR3 ... K MRm-1 and spectral data matrix B M Euclidean distance d Euclid_MR and cosine distance d carbo_MR To determine the Euclidean distance d Euclid_MR Cosine distance d carbo_MR The threshold.

[0021] Compared with existing methods, the advantages of the present invention are:

[0022] This invention classifies Fourier transform mid-infrared time-series greenhouse gas measurement spectra by setting cosine distance and Euclidean distance thresholds, and dynamically selects principal components for spectral reconstruction. It can effectively retain the concentration change information of each gas in the time-series measurement spectrum, realize adaptive processing of Fourier transform mid-infrared time-series greenhouse gas measurement spectral data, and has high practical value. Attached Figure Description

[0023] Figure 1 This diagram illustrates the processing steps of the mid-infrared Fourier time-series greenhouse gas measurement spectral data processing method based on principal component analysis according to the present invention.

[0024] Figure 2 This includes single numerical simulation spectral data and numerical simulation spectra with superimposed Gaussian noise.

[0025] Figure 3 The results show the inversion concentration comparison of time-series measured spectra.

[0026] Figure 4 The linear fitting results are for the CO2 concentration obtained by spectral inversion measured at 1 min and the CO2 concentration obtained by spectral inversion reconstructed at 1 min. Detailed Implementation

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and examples, but the drawings and examples are not intended to limit the technical solutions of the present invention.

[0028] like Figure 1 As shown, the principal component dynamic selection method for greenhouse gas Fourier transform infrared spectral data processing of this invention was applied to time-series measurement spectra to obtain time-series concentration change information, and the results were compared with the concentration inversion results obtained by the average spectral method. The specific steps are as follows:

[0029] Step 1) Based on the spectral line parameters of the HITRAN database, the numerical simulation spectral matrix B was obtained through numerical simulation and Monte Carlo statistical methods. M (256×5) and Gaussian noise spectral matrix B with Ns (0.001RMS) noise level. MN (256×5), numerical simulation spectrum and Gaussian noise spectrum are as follows Figure 2 As shown. The Monte Carlo statistical method is a stochastic simulation method that evaluates the properties and statistical distribution of a spectrum through repeated random trials;

[0030] Step 2) For the Gaussian noise spectral matrix B MN After standardization, principal component decomposition is performed to obtain the loading matrix K. MN (5×5), load matrix K is selected sequentially. MN The spectral matrix K is reconstructed from the 1-4 principal components. MR1 K MR2 K MR3 and K MR4 ;

[0031] Step 3) Calculate and obtain the Gaussian noise spectral matrix B MN and spectral matrix K MR1 K MR2 K MR3 K MR4 The Euclidean distances are 0.0019, 0.0015, 0.0012, and 0.00073, respectively, and the cosine distances are 9.5360 × 10⁻⁶. -5 7.9497×10 -5 3.1244×10 -5 1.1830×10 -5 The calculated Euclidean distance and cosine distance are used as classification thresholds. The calculation of Euclidean distance and cosine distance is to ensure the quality of the reconstructed spectrum.

[0032] Step 4) Measure the mid-infrared Fourier time-series spectral data of ambient air samples using a Fourier transform infrared spectrometer, and after preprocessing such as baseline correction, summarize to form a measurement dataset M (16384×810).

[0033] Step 5) Effectively divide the measurement dataset into blocks to form individual spectral data matrices A. M1 (334×5), A M2 (334×5), A M3 (334×5), etc., and standardize each spectral data matrix;

[0034] Step 6) Apply the standardized spectral data matrix A M1 Principal component decomposition was performed, and spectral reconstruction was carried out by sequentially selecting 1, 2, 3, and 4 principal components from the loading matrix to obtain the reconstructed spectral matrix A. R1 A R2 A R3 A R4 ;

[0035] Step 7) Calculate the reconstructed spectral matrix A respectively.R1 A R2 A R3 A R4 and spectral data matrix A M1 Euclidean distance d Euclid_A and cosine distance d carbo_A The Euclidean distances between the reconstructed spectral matrices and the spectral data matrices of different principal components from 2:00 to 8:00 were 0.03400, 0.00163, 0.00051, and 0.00002, respectively, and the cosine distances were 1.34250 × 10⁻⁶. -6 3.21977×10 -7 8.31745×10 -8 6.29376×10 -9 By comparing the calculated Euclidean distance, cosine distance, and classification threshold, it can be seen that with two principal components, the calculated values ​​fall within the set threshold range. Therefore, the reconstructed spectral matrices corresponding to the two principal components are used to reconstruct each measured spectrum from 2 to 8 hours. The Euclidean distances between the reconstructed spectral matrices and the spectral data matrices for different principal components from 0 to 2 hours and from 8 to 11 hours are 0.07607, 0.00252, 0.00087, and 0.00001, respectively, while the cosine distances are 7.324 × 10⁻⁶. -5 3.21977×10 -6 and 9.73277×10 -7 6.25133×10 -8 Similarly, with three principal components, the calculated Euclidean and cosine distances are within the set threshold range. Therefore, it is necessary to use the reconstructed spectral matrices corresponding to the three principal components to reconstruct each measured spectrum within the time intervals of 0–2 and 8–11.

[0036] Step 8) Reconstruct the spectrum using a nonlinear least squares regression model to obtain the concentrations of greenhouse gas components over time. For example... Figure 4 As shown, there is a significant correlation between human-caused pollution emissions and measured CO2 concentrations during the morning rush hour. The changes in CO2 concentration retrieved from the 1-minute, 5-minute average spectra, and 1-minute reconstructed spectra all clearly reflect the degree of pollution emissions, and the accuracy comparison of their retrieved concentrations is shown in Table 1. As shown in Figure 5, the correlation coefficient between the CO2 concentration retrieved from the 1-minute average spectra and the CO2 concentration retrieved from the 1-minute reconstructed spectra can reach 0.894.

[0037] In summary, this data processing method can improve the quality of the analyzed data to a certain extent, reduce the impact of instrument noise, and retain the peak information in the original measurement without reducing the time resolution.

[0038] Table 1

[0039]

[0040] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing Fourier transform infrared spectral data of greenhouse gases with dynamic principal component selection, characterized in that, Specifically, the following steps are included: Step 1: Measure the mid-infrared Fourier time-series spectral data of ambient air samples using a Fourier transform infrared spectrometer, perform baseline correction preprocessing, and then compile the data to form a measurement dataset. Step 2: Divide the measurement dataset into blocks to form A M1 A M2 A M3 Spectral data matrices were generated, and each matrix was standardized. The block-based matrices retained the spectral absorption characteristics of various greenhouse gases, while bands that did not contain the spectral characteristics of CO2, N2O, and CO were filtered out, thus improving the speed of principal component decomposition. M1 A M2 A M3 Both are m×n spectral data matrices, where m is the number of measurements and n is the wavenumber range; Step 3: Process the standardized spectral data matrix A M1 Principal component decomposition is performed to obtain the loading matrix K. M1 Load matrix K M1 It is an m×m matrix; Step 4: Select the load matrix K sequentially. M1 The spectral matrix is ​​reconstructed using the 1st principal component, 2nd principal components, ..., m-1 principal components, thus obtaining the reconstructed spectral matrix A. R1 A R2 A R3 A Rm-1 ; Step 5: Calculate the spectral data matrix A M1 With the reconstructed spectral matrix A R1 A R2 A R3 A Rm-1 Euclidean distance d Euclid_A and cosine distance d carbo_A ; Step 6: Calculate the Euclidean distance d Euclid_A and cosine distance d carbo_A The reconstructed spectral matrix is ​​compared with the set threshold. Based on the comparison results, the reconstructed spectral matrix with the Euclidean distance and cosine distance closest to the threshold is selected to reconstruct each measured spectrum in order to obtain high-quality spectral data. The set threshold is obtained through numerical simulation calculation. Step 7: Repeat steps 3, 4, 5, and 6 to process the spectral data matrix A one by one. M2 Spectral data matrix A M3 This completes the processing of the measurement dataset; Step 8: Reconstruct the spectrum by using a nonlinear least squares regression model to obtain the concentration of greenhouse gas components over time.

2. The method for processing greenhouse gas Fourier transform infrared spectral data with dynamic principal component selection according to claim 1, characterized in that, The threshold value set in step six is ​​calculated using the following steps: Step a: Based on the spectral line parameters in the database, obtain the numerical simulation spectral matrix B using numerical simulation and Monte Carlo statistical methods. M and Gaussian noise spectral matrix B with RMS noise level Ns MN The Monte Carlo statistical method is a stochastic simulation method that evaluates the properties and statistical distribution of a spectrum through repeated random trials; numerical simulation of the spectral matrix B. M For an m×n matrix, the Gaussian noise spectral matrix B MN It is an m×n matrix; Step b: For the Gaussian noise spectral matrix B MN After standardization, the loading matrix K is obtained using principal component decomposition. MN Load matrix K MN It is an m×m matrix; Step c: Select the load matrix K sequentially. MN Spectral reconstruction was performed on different numbers of principal components to obtain the reconstructed spectral matrix K. MR1 K MR2 K MR3 ... K MRm-1 ; Step d: Calculate the reconstructed spectral matrix K respectively. MR1 K MR2 K MR3 ... K MRm-1 and spectral data matrix B M Euclidean distance d Euclid_MR and cosine distance d carbo_MR To determine the Euclidean distance d Euclid_MR Cosine distance d carbo_MR The threshold.

Images