Atmospheric pollutant full-coverage data fusion method based on discrete cosine transform

Abstract

The invention provides a discrete cosine transform-based atmospheric pollutant full-coverage data fusion method, which comprises the following steps of: 1, carrying out data preparation and preprocessing, namely adjusting model data and satellite observation data to the same precision by using an area weighted aggregation method, ensuring the consistency of the model data and the satellite observation data in geographic space, and carrying out data fusion on the model data and the satellite observation data; subsequent data fusion processing is facilitated; step 2, performing data fusion processing: performing filling and smoothing operation on three-dimensional space-time proportion parameters of the satellite observation data and the model data based on a space-time discrete cosine transform (DCT) smoothing principle; and step 3, performing correlation analysis on the atmospheric pollutant seamless full-coverage data and the ground station observation data obtained after fusion, and performing advantage and disadvantage judgment on a correlation analysis result according to a set judgment threshold. According to the invention, the model data can be effectively calibrated and the deficiency of satellite observation can be supplemented.

Description

Technical Field

[0001] The invention belongs to the technical field of environmental monitoring, and in particular relates to a method for fusion of full coverage data of atmospheric pollutants based on discrete cosine transform. Background Art

[0002] Satellite technology has made significant progress in observing the concentration of atmospheric pollutants, providing an important tool for global environmental monitoring. For example, Europe's Sentinel-5P satellite can capture the distribution of key pollutants such as nitrogen dioxide, ozone, and methane at high resolution around the world through its Tropomi instrument; the OMI instrument on the US Aura satellite also provides valuable data for atmospheric science research. In addition, China's Gaofen-5 satellite (GF-5) is also equipped with advanced atmospheric detection instruments that can monitor a variety of atmospheric pollutants and contribute to domestic and global air quality research. Together, these satellites form a global atmospheric monitoring network that improves our understanding and response capabilities to changes in the atmospheric environment. These satellites can not only monitor the temporal and spatial changes of pollutants, but also track their sources and migration paths, greatly improving our understanding of atmospheric chemical processes. However, despite the increasing maturity of satellite observation technology, there are still some challenges. One of the most prominent problems is the impact of meteorological conditions, such as cloud cover that can lead to gaps in data collection, especially in cloudy or rainy areas, which limits the continuity and integrity of the data. In addition, nighttime observations are limited by insufficient light, which can also lead to missing data for some time periods. Therefore, establishing an all-weather, full-coverage dataset is of great significance for in-depth research on the atmospheric environment, analyzing pollutant trends, and predicting future pollution conditions.

[0003] Chemical transport models (CTM) play a vital role in the current observation and prediction of atmospheric pollutant concentrations. These models can simulate the concentration distribution of atmospheric pollutants and maintain a high degree of consistency with satellite observations, thereby verifying their reliability and accuracy. A significant advantage of CTM is that the temporal and spatial resolution can be flexibly adjusted, allowing researchers to customize model parameters according to specific needs to obtain more refined simulation results. For example, at the city scale, high resolution can be used to capture the impact of local pollution sources; while at the global scale, lower resolution can be used to evaluate large-scale pollutant transport patterns. Common chemical transport models include but are not limited to CAMx (Community Multiscale Air Quality Modeling System), CMAQ (Community Multi-scale Air Quality Modeling System), WRF-Chem (Weather Research and Forecasting model coupled with Chemistry, a model of online coupling of meteorological and chemical modules), GEOS-Chem (Goddard Earth Observing System Model with Chemistry), etc. These models take into account meteorological conditions, emission inventories, and complex chemical reaction mechanisms, making the simulation results closer to the actual situation. However, chemical transport models also have certain limitations, one of the main problems being large simulation errors. Since the model depends on the quality of input data, such as the accuracy of emission inventories and the accuracy of meteorological forecasts, any deviation may lead to an amplified error in the final simulation results. In addition, complex chemical reaction mechanisms and physical processes may also introduce additional uncertainties. Therefore, it is crucial to correct the simulation data. By comparing and correcting with measured data (such as satellite observations, ground monitoring station data, etc.), the model error can be effectively reduced and the prediction accuracy can be improved. This correction process is of great significance for accurately assessing the state of the atmospheric environment, analyzing pollutant trends, and predicting future pollution conditions, and helps to better formulate and implement environmental protection measures.

[0004] With the worsening of atmospheric environmental pollution, the necessity and urgency of solving environmental problems are increasingly recognized by countries around the world. The above-mentioned satellite observation technology and chemical transport model simulation methods each have their own defects, but each has its own advantages. The inventors of this application have found that the high-quality fusion of the data of the two can not only achieve full coverage of satellite data, but also complete the correction of model results, and realize rapid, continuous and efficient monitoring of the spatiotemporal variation characteristics of atmospheric pollution components, thereby meeting the needs of my country's atmospheric environment monitoring and environmental pollution research. Summary of the invention

[0005] The purpose of the present invention is to provide a method for fusing atmospheric pollutant full coverage data based on discrete cosine transform, which can fuse satellite data with chemical transport model simulation results to obtain daily full coverage data of atmospheric pollutants. The method comprises the following steps: Step 1: Data preparation and preprocessing, including: using the chemical transport model to simulate atmospheric pollutant data for the same period as satellite observations to obtain model data, and using the area-weighted aggregation method to adjust the model data and satellite observation data to the same accuracy to ensure their consistency in geographic space for subsequent data fusion processing; Step 2: Data fusion processing: Based on the smoothing principle of space-time discrete cosine transform, fill and smooth the three-dimensional space-time scale parameters of satellite observation data and model data. The methods used in this step include: preprocessing missing data using multidimensional nearest neighbor interpolation, automatically determining smoothing parameters using generalized cross-validation scores, and using three-dimensional discrete cosine transform to convert data between the space-time domain and the frequency domain. In the conversion, smoothing parameters are used to enhance smoothing and reduce noise in the frequency domain from low frequency to high frequency. Step three: perform a correlation analysis on the fused seamless full coverage data of atmospheric pollutants and the ground station observation data, and judge the quality of the correlation analysis results based on the set evaluation threshold.

[0006] Compared with the prior art, the present invention has the following beneficial effects: The atmospheric pollutant full coverage data fusion method based on spatiotemporal discrete cosine transform retains the advantages of the model in data integrity and spatial distribution consistency and satellite data accuracy, and takes into account the spatial and temporal correlation of atmospheric pollutants. The fused data realizes the calibration of chemical transport model data and the filling of satellite data, and has the advantages of strong interpretability and good scalability in terms of method. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0008] Figure 1 A carbon monoxide column concentration map of a local area in China observed by the Tropomi satellite provided in an embodiment of the present invention; Figure 2 A graph of carbon monoxide column concentration in a local area of ​​China simulated by the GEOS-Chem model provided in an embodiment of the present invention; Figure 3 A carbon monoxide column concentration map of a local area in China using the fusion data of the method of the present invention provided in an embodiment of the present invention; Figure 4 A flowchart of a method for fusion of atmospheric pollutant data based on discrete cosine transform provided in an embodiment of the present invention; Figure 5 A correlation diagram of the fused data using the method of the present invention and the TCCON site comparison provided in the embodiment of the present invention. DETAILED DESCRIPTION

[0009] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0010] GEOS- Chem is a three-dimensional model of global atmospheric chemistry driven by meteorological inputs retrieved by the Goddard Earth Observing System (GEOS) of NASA's Global Modeling and Assimilation Office. This example uses version 14.1.1 (http: / / acmg.seas.harvard.edu / geos / ), driven by the Modern Research and Application Retrospective Analysis (MERRA-2) reanalysis data. The model was applied to simulate CO (Carbon Monoxide) in the Chinese region for a total of five years from the beginning of 2019 to the end of 2023 on a 0.25°× 0.3125° horizontal grid. Tag-CO is a calculation scheme for geochemical models that only simulates the concentration and distribution of CO. Compared with the full chemical simulation scheme, the selected scheme can save more computing time. The relevant simulation experiments used in the example are all tag carbon monoxide experiments. Using different carbon monoxide emission inventories in different regions, the CO concentration is calculated based on surface emissions and chemical effects. Canada's anthropogenic emissions are retrieved from the Air Pollutant Emission Inventory (APEI) v2016. The North American regional emissions are calculated using the National Emissions Inventory (NEI) v2015-03. The African Diffuse and Inefficient Combustion Emissions Inventory includes anthropogenic emissions from the burning of biofuels and fossil fuels in the African region. The MIXv1.1 inventory is used for Asia. Emissions from aircraft and ships from all over the world are included in the Community Emissions Data System (CEDS) inventory. The Global Fire Emissions Database (GFED2) provides data on carbon monoxide emissions from biomass burning. The MEGAN (Emissions of gas and Aerosols from Nature) model is used to calculate the volatile organic compound (VOC) biogenic emission inventory. The time step for convective and parallel transport in the simulation is 10 minutes. For the chemical processes involved in the GEOS-Chem model, the simulation time step is 20 minutes.

[0011] TROPOMI (TROPOspheric Monitoring Instrument) is a high-resolution spectrometer installed on the Sentinel-5 Precursor satellite, which is mainly used to monitor various pollutants in the atmosphere. TROPOMI data are usually divided into different levels, among which "Level 2 data" is observation data after preliminary processing, which has extracted specific atmospheric parameters from the original observation data. In this embodiment, TROPOMI Level 2 carbon monoxide data is used, which provides the column concentration of carbon monoxide in the atmosphere, that is, the total mass of carbon monoxide in the atmosphere per unit area. TROPOMI has a spatial resolution of 7 × 3.5 kilometers, and can cover the world once a day in terms of time. It also uses data from the beginning of 2019 to the end of 2023 for a total of five years.

[0012] By using the area-weighted average method, the data of TROPOMI and GEOS-Chem are aligned to a spatial grid of 0.05°×0.05°, and aligned day by day in time. The TROPOMI valid data is used to compare the data of GEOS-Chem at the corresponding time and space to obtain the initial time and space ratio. There are vacancies in the ratio. After the smoothing algorithm in the present invention is processed, a smooth and complete time and space ratio can be obtained. The time and space ratio is applied to the GEOS-Chem data to reproduce the complete TROPOMI satellite data.

[0013] The following describes a method for fusion of atmospheric pollutant full coverage data based on discrete cosine transform according to an embodiment of the present invention. This method fuses satellite observation data with atmospheric pollutant data simulated by a chemical transport model, aiming to improve the accuracy of model simulation and fill in the missing parts of satellite observation data. The method includes the following steps: Step 1: Data preparation and preprocessing, including: chemical transport model simulation and satellite observation of atmospheric pollutant data for the same period to obtain model data, and use area-weighted aggregation method to adjust the model data and satellite observation data to the same accuracy to ensure the consistency of the two in geographic space for subsequent data fusion processing.

[0014] Step 2: Data fusion processing: Based on the smoothing principle of space-time discrete cosine transform (DCT), fill and smooth the three-dimensional space-time scale parameters of satellite observation data and model data. The methods used in this step include: preprocessing the missing data of the above three-dimensional space-time scale parameters using multidimensional nearest neighbor interpolation, automatically determining the smoothing parameters using generalized cross-validation scores, using three-dimensional discrete cosine changes to convert the three-dimensional space-time scale parameters between the space-time domain and the frequency domain, and using the smoothing parameters to enhance, smooth and reduce noise in the frequency domain of the three-dimensional space-time scale parameters from low frequency to high frequency during the conversion.

[0015] Step 3: Perform a correlation analysis on the seamless full coverage data of atmospheric pollutants obtained after fusion and the ground station observation data, and judge the results of the correlation analysis according to the set judgment threshold. Among them, the ground station observation data refers to the carbon monoxide column concentration observed on the ground by the Total Column Carbon Observing Network (TCCON), and the correlation analysis refers to the degree of fit between the ground station observation data and the seamless full coverage data of atmospheric pollutants obtained after fusion at the same time and space. The judgment threshold uses the determination coefficient (R², Coefficient of Determination).

[0016] In step 1, the data selection for data fusion uses the chemical transport model (CTM) to simulate atmospheric pollutants and the satellite observation data of atmospheric pollutants. The chemical transport model is based on solid physical and chemical theory, can accurately simulate chemical reaction processes in the atmosphere, and supports multiple resolution settings. The grid size can be adjusted according to research needs, thereby improving the simulation accuracy at the local or regional scale. Although satellites are affected by meteorological conditions such as clouds and produce missing values, the valid values ​​they provide can monitor atmospheric pollutants relatively accurately and provide high-precision preliminary values ​​for fused data.

[0017] In step one, it is necessary to initialize the spatiotemporal ratio of the satellite and the model, that is, to adjust the model data and the satellite observation data to the same accuracy through the area-weighted aggregation method, and then align them in time and space, so as to transform the fusion problem into the filling and smoothing problem of the three-dimensional spatiotemporal ratio parameters of the model data and satellite observation data obtained in this way, as shown in the following formula.

[0018] (1) in, represents the results of the chemical transport model simulation, represents the satellite observation results, It represents the ratio between the first two elements in terms of time, latitude and longitude.

[0019] The data fusion processing in step 2 includes: using the space-time nearest neighbor filling method to preliminarily fill in the three-dimensional space-time ratio parameters of the model data and satellite observation data, where the space-time nearest neighbor method is a method for processing missing data, which uses the proximity relationship in time and space to fill in missing values. The weights of the vacancies and effective values ​​are retained, and after discrete cosine transformation, 80% of the high frequencies are cut off for initialization of rough smoothing. At the same time, the source data part of the data after initialization of rough smoothing is restored through the weights of the vacancies and effective values.

[0020] Figure 4 A flow chart of a method for fusion of full coverage data of atmospheric pollutants based on discrete cosine transform provided in an embodiment of the present invention is shown.

[0021] The data fusion processing in step 2 also includes: according to the weights of each grid point of the three-dimensional space-time scale parameter and the preliminary filling data obtained by preliminary filling, the three-dimensional space-time scale is iteratively smoothed using formula (2), and the initial value of the iteration is Take 0 and select the smoothing parameter s through generalized cross-validation (GCV), as shown in formula (5).

[0022] (2) (3) (4) (5) (6) in, represents the smoothing result of the k+1th iteration; DCT and IDCT represent the discrete cosine transform and inverse discrete cosine transform of the matrix. When k=0, the initialization is a zero matrix; y represents the three-dimensional space-time scale parameter initially filled; W is a weight matrix, the vacancies of the original three-dimensional space-time scale parameter are 0, and the non-vacancies are 1; is the smoothing matrix for each iteration, obtained by formula (3), where is the three-dimensional identity matrix; is a three-dimensional diagonal matrix, which is obtained by formula (4), where is the size of each dimension of the three-dimensional matrix y, represents the i-th diagonal element of the j-th dimension of a diagonal matrix; s is the iterative smoothing parameter, obtained by the generalized cross-validation function of formulas (5) and (6), The function represents the square root of the weight value of each point; represents the Euclidean normal form; is the total number of elements in y, is the number of missing values ​​of y, H is the hat matrix of y, which is used to project the vector of observations onto the best fitting line or multidimensional hyperplane. Denotes the trace of the matrix H. When n is large enough, its relationship with s can simplify formula (6).

[0023] Generate a 3D array of the same size based on 3D space-time coefficients , the generation rule is as shown in formula (4), which is called the dynamic weight array. The dynamic weight array is generated according to the discrete cosine transform and the position of each element of the three-dimensional space-time coefficient. In the frequency domain, the low-frequency elements have lower weights in the dynamic weight array, and on the contrary, the high-frequency elements have higher weights in the dynamic weight array. It can be seen from formula (3) that when the s value is larger, the high-frequency part of the three-dimensional space-time coefficient after the frequency domain conversion will be retained less, the smoothing will be more obvious, but a lot of details will be lost; on the contrary, the smaller the s value is, the more high frequencies will be retained, the result will be closer to the source data, but the smoothing effect will be poor. The generalized cross-validation function, that is, formula (5), is used to punish excessive smoothing and under-smoothing conditions to obtain a suitable smoothing coefficient s.

[0024] The smoothing parameters are applied to the data of the rough smoothing of the temporal and spatial coefficient initialization. The smoothed results account for 75% and the unsmoothed results account for 25%. This ratio is iterated until the relative change of the two most recent smoothed results is small enough to determine whether it has converged.

[0025] After the three-dimensional space-time coefficients are filled and smoothed, they can be applied to the results of the chemical transport model simulation, that is, the three-dimensional space-time coefficients are multiplied by the model data to obtain the fusion data of the satellite and chemical transport model simulation.

[0026] Figure 1 The unfilled CO concentration data observed by the TROPOMI satellite are shown. It can be seen that there are large gaps, which is not conducive to the study of atmospheric pollution components. Figure 2 The CO concentration data simulated by the GEOS-Chem model shows a distribution consistent with that of satellite data, but is not as accurate as satellite data. Figure 3 The CO concentration data is obtained by using the full coverage data fusion method of atmospheric pollutants after discrete cosine transformation, which effectively calibrates the model data and supplements the deficiencies of satellite observations.

[0027] In this embodiment, the data fusion method of full coverage of atmospheric pollutants by discrete cosine transform has been used to fuse the data of China from 2019 to 2023. The data of the Total Column Carbon Observing Network (TCCON) at the Hefei station (31.91N, 117.17E) and the Xianghe station (39.8 N, 116.96E) at 13:00±1h were selected as the validation set, and the data of the filling result data corresponding to the center 1° of the station were selected for correlation comparison.

[0028] like Figure 5 As shown in the figure, the correlation R2 of the Topomi satellite observation results at the Hefei and Xianghe stations reached 0.778 and 0.802, while the correlation R2 of the data fused by the atmospheric pollutant full coverage data fusion method based on spatiotemporal discrete cosine transform of the present invention also reached 0.774 and 0.803. The correlation of the fused data is close to or even better than the satellite observation results, indicating that the research results can fully meet the analysis requirements of the distribution and evolution of atmospheric pollution components.

[0029] In summary, this paper proposes a method for fusion of atmospheric pollutant data with full coverage based on discrete cosine transform, which aims to fuse satellite observation data with the simulation results of chemical transport model to generate daily full coverage atmospheric pollutant concentration data. The method first preprocesses the two data sources and unifies them to the same resolution level to ensure temporal and spatial consistency. Then, a three-dimensional spatiotemporal coefficient is established to characterize the functional relationship between the model and satellite data, and the coefficient is initialized using the effective observation data of the satellite.

[0030] In order to fill the missing parts in the spatiotemporal coefficients, the present invention uses discrete cosine transform (DCT) to retain low-frequency components and remove most high-frequency components in the frequency domain, and then restores the coefficient matrix through inverse discrete cosine transform (IDCT). In this process, a dynamic weight array weighted according to frequency characteristics is generated, and the degree of data smoothing and detail retention is balanced by adjusting the smoothing parameter s, and the optimal smoothing coefficient is determined using a generalized cross-validation function.

[0031] The fusion method performs multiple iterative smoothing of the spatiotemporal coefficients and combines the initial and smoothed data until the convergence criterion is reached. After the filling and smoothing operations are completed, the spatiotemporal coefficients are applied to the output of the chemical transport model to obtain a fused data set. This method not only maintains the integrity of the model data and the consistency of the spatial distribution, but also makes full use of the accuracy of satellite data, while considering the correlation of pollutants in the spatiotemporal dimension. In addition, this method has strong interpretability and good scalability, and can effectively calibrate the model data and supplement the deficiencies of satellite observations.

[0032] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fusion of atmospheric pollutant full coverage data based on discrete cosine transform, characterized in that: The following steps are involved: Step 1: Data preparation and preprocessing, including: using the chemical transport model to simulate atmospheric pollutant data for the same period as satellite observations to obtain model data, and using the area-weighted aggregation method to adjust the model data and satellite observation data to the same accuracy to ensure their consistency in geographic space for subsequent data fusion processing; Step 2: Data fusion processing: Based on the smoothing principle of space-time discrete cosine transform, fill and smooth the three-dimensional space-time scale parameters of satellite observation data and model data. The methods used in this step include: preprocessing missing data using multidimensional nearest neighbor interpolation, automatically determining smoothing parameters using generalized cross-validation scores, and using three-dimensional discrete cosine transform to convert data between the space-time domain and the frequency domain. In the conversion, smoothing parameters are used to enhance smoothing and reduce noise in the frequency domain from low frequency to high frequency. Step three: perform a correlation analysis on the fused seamless full coverage data of atmospheric pollutants and the ground station observation data, and judge the quality of the correlation analysis results based on the set evaluation threshold.

2. The method for fusion of atmospheric pollutant data based on discrete cosine transform according to claim 1, characterized in that: The model data and satellite observation data are adjusted to the same accuracy through the area weighted aggregation method, and then aligned in time and space, so that the fusion problem is transformed into the filling and smoothing problem of the three-dimensional space-time scale parameters of the model data and satellite observation data obtained in this way, as shown in the following formula: （1） in, represents the results of the chemical transport model simulation, represents the satellite observation results, It represents the ratio between the first two elements in terms of time, latitude and longitude.

3. The method for fusion of atmospheric pollutant data based on discrete cosine transform according to claim 1, characterized in that: The data fusion processing in step 2 includes: using the space-time neighbor filling method to preliminarily fill in the three-dimensional space-time scale parameters of the model data and satellite observation data, and retaining the weights of the vacancies and effective values. After discrete cosine transform, 80% of the high frequency is cut off to initialize rough smoothing, and at the same time, the source data part of the rough smoothed data is restored through the weights of the vacancies and effective values.

4. The method for fusion of atmospheric pollutant data based on discrete cosine transform according to claim 3, characterized in that: The data fusion processing of step 2 also includes: according to the weights of each grid point of the three-dimensional space-time scale parameter and the preliminary filling data obtained by preliminary filling, the three-dimensional space-time scale is iteratively smoothed using formula (2): （2） （3） （4） （5） （6） in, represents the smoothing result of the k+1th iteration; DCT and IDCT represent the discrete cosine transform and inverse discrete cosine transform of the matrix. When k=0, the initialization is a zero matrix; y represents the three-dimensional space-time scale parameter initially filled; W is a weight matrix, the vacancies of the original three-dimensional space-time scale parameter are 0, and the non-vacancies are 1; is the smoothing matrix of each iteration, obtained by formula (3), where is the three-dimensional identity matrix; is a three-dimensional diagonal matrix, which is obtained by formula (4), where is the size of each dimension of the three-dimensional matrix y, represents the i-th diagonal element of the j-th dimension of a diagonal matrix; s is the iterative smoothing parameter, obtained by the generalized cross-validation function of formulas (5) and (6), The function represents the square root of the weight value of each point; represents the Euclidean normal form; is the total number of elements in y, is the number of missing values ​​of y, H is the hat matrix of y, which is used to project the vector of observations onto the best fitting line or multidimensional hyperplane. represents the trace of the matrix H.

Images