A method for retrieving atmospheric temperature and humidity profiles based on machine learning application of FY-4A / GIIRS data

Abstract

The application is a method for retrieving atmospheric temperature and humidity profiles based on machine learning and FY-4A / GIIRS data, belonging to the technical field of satellite meteorological observation. The application uses the channel data of the hyperspectral infrared atmospheric vertical sounder GIIRS of the FY-4A satellite as sample feature data set, uses the atmospheric temperature and humidity data of the ERA5 reanalysis data set and the GIIRS data set to construct labels, and constructs a training data set; based on the deep separable convolution, the dense connection mechanism and the attention mechanism, the U-NET model is improved, and a three-dimensional atmospheric temperature and humidity profile model is constructed; the training data set is used to optimize the model, and the optimized model is used for real-time retrieval, and the small area results are spliced to obtain the three-dimensional atmospheric temperature and humidity profile of the whole research area. The application realizes the retrieval of the three-dimensional atmospheric temperature and humidity profile of the research area, better utilizes the channel data and deeply mines the information, and improves the accuracy of the retrieved atmospheric temperature and humidity profile.

Description

Technical Field

[0001] This invention belongs to the field of satellite meteorological observation technology, specifically involving a method for retrieving atmospheric temperature and humidity profiles based on machine learning using FY-4A / GIIRS data. Background Technology

[0002] Atmospheric temperature and humidity are essential parameters for describing atmospheric motion and thermodynamic state, and are indispensable for meteorological forecasting research. Atmospheric temperature and humidity profiles can be used not only to initialize and evaluate numerical weather prediction models, but also to estimate weather stability and provide real-time forecasts of severe convective weather. Furthermore, atmospheric temperature and humidity parameters directly affect the interaction between solar shortwave radiation and longwave radiation from the Earth-atmosphere system, thus influencing the global radiation energy balance. Good atmospheric temperature and humidity profile information plays a crucial role in advancing numerical weather prediction, meteorological support, and even climate change research.

[0003] With the development of satellite technology and infrared sensors, the advantages of spaceborne infrared hyperspectral imaging of atmospheric temperature and humidity profiles have become increasingly apparent. Firstly, being mounted on a satellite, it makes obtaining upper-level data easier. Secondly, traditional direct observations or radio occultation observations only provide values ​​at a single point, resulting in poor horizontal extension. In contrast, spaceborne infrared hyperspectral imaging directly scans and images the nadir point using sensors, achieving direct horizontal extension in space and featuring high spectral and spatial resolution. On December 11, 2016, my country successfully launched its second-generation geostationary meteorological satellite, FY-4A, marking the first time a geostationary meteorological satellite carried an interferometric atmospheric hyperspectral imager (GIIRS). GIIRS provides the infrared radiation energy observed in the Earth's atmosphere. Due to the selective absorption of infrared thermal radiation over large areas, observations of the CO2 absorption band are typically used to retrieve atmospheric temperature, while observations of the H2O absorption band are used to retrieve atmospheric humidity, thus achieving the goal of retrieving atmospheric temperature and humidity through infrared radiation energy.

[0004] ERA5 data is the fifth generation of global atmospheric reanalysis data launched by the European Centre for Medium-Range Weather Forecasts (ECMWF). The foundation of ERA5 reanalysis data is data assimilation. It uses next-generation ensemble four-dimensional variational and EDA (Electronic Design Automation) assimilation techniques to assimilate meteorological data from various sources worldwide, including remote sensing data, land and ocean surface data, and upper atmosphere data. ERA5 assimilates a vast amount of data; over the forty years from 1979 to 2019, the number of assimilated data increased from approximately 750,000 per day to 24 million per day. The advancements in assimilation technology and the sheer volume of assimilated data result in ERA5 having a very high operational level, and its data accuracy is generally considered quite high.

[0005] Traditional datasets, primarily based on single-point observations, can only provide data for a specific location in space, thus lacking the ability to reproduce spatially continuous atmospheric temperature and humidity profiles. With the development of spaceborne infrared instrument technology, infrared hyperspectral data has gradually gained the ability to probe entire spatial regions, attracting increasing attention from scholars, and techniques for retrieving these profiles have become a key research focus. Since the 1970s, methods for retrieving atmospheric temperature and humidity profiles from infrared hyperspectral data have been broadly categorized into three types: statistical regression, physical inversion, and machine learning. The method of directly establishing a statistical regression model between radiance values ​​and the atmospheric temperature and humidity profile to be retrieved is called statistical regression. The advantages of statistical regression inversion algorithms are low computational resource consumption and stable results. However, the linear and relatively simple nonlinear models used in this method generally have limited inversion capabilities for complex nonlinear cases in actual data, and they are heavily dependent on sample size, making them unsuitable for regions with insufficient sample data. Physical inversion overcomes the shortcomings of statistical regression, which does not consider the nature of radiative transfer and has unclear physical meaning. It is a calculation method based on the radiation process. Although it has a theoretical foundation that statistical regression algorithms do not have, it has extremely strict requirements on the weight function. It is precisely because it needs to strictly follow the physical process that it has a huge amount of computation and a slow calculation speed. Machine learning uses neural networks to replace complex nonlinear fitting relationships. It does not require complex physical process analysis. Moreover, thanks to the continuous optimization of computer performance, the calculation process is much simpler than the physical inversion process. Malmgren-Hansen et al. first proposed using convolutional networks to invert atmospheric profiles using IASI (Infrared Atmospheric Sounding Interferometer) data, and concluded that deep learning can be applied to atmospheric profile inversion (Reference 1: Malmgren-Hansen D, Laparra V, Nielsen AA, et al. Statistical retrieval of atmospheric profiles with deep convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2019,158:231-240). Xue Qiumeng constructed one-dimensional CNN and three-dimensional CNN to invert atmospheric temperature and humidity profiles using Level 1 observational data of GIIRS. The test results show that the temperature inversion accuracy of one-dimensional CNN is higher, while the humidity inversion accuracy of three-dimensional CNN is relatively higher (Reference 2: Xue Qiumeng. Research on All-Sky Atmospheric Temperature and Humidity Profile Inversion Algorithm Based on FY-4A / GIIRS [D]. Nanjing University of Information Science and Technology, 2023).However, the model used in this scheme was not designed for spaceborne infrared hyperspectral data, and it did not consider how to better utilize the data from each channel or how to deeply mine information, so the accuracy of inverting the three-dimensional atmospheric temperature and humidity profile needs to be improved. Currently, how to apply deep learning to spatially correlated spaceborne infrared hyperspectral data for atmospheric temperature and humidity profile inversion and obtain reliable accuracy is a key area of ​​research that needs to be addressed. Summary of the Invention

[0006] This invention addresses the problems of inaccurate atmospheric temperature and humidity profile inversion and lack of spatial correlation in current methods. It provides a method for inverting atmospheric temperature and humidity profiles based on machine learning and using FY-4A / GIIRS data. By using GIIRS data provided by FY-4A and ERA5 reanalysis data to construct a data training set, a three-dimensional and accurate deep learning model is trained to perform atmospheric temperature and humidity profile inversion.

[0007] This invention provides a method for retrieving atmospheric temperature and humidity profiles based on machine learning using FY-4A / GIIRS data, comprising the following steps:

[0008] Step 1: Download the GIIRS channel data and ERA5 reanalysis data provided by the FY-4A meteorological satellite within the specified time range of the study area, preprocess them, and then construct the training dataset;

[0009] The process of preprocessing GIIRS channel data to generate a sample feature dataset includes: performing apodization on the GIIRS channel data, then converting it into brightness temperature data using the Planck inverse function, and using PCA principal component analysis to reduce the dimensionality of the 1650 channels in the GIIRS brightness temperature data, resulting in a 300-channel, 32×32 pixel image for each sample feature; downloading ERA5 reanalysis data as label data, and using bilinear interpolation to map the label data to each sample space grid, with the label data including temperature and humidity profiles.

[0010] Step 2: Construct atmospheric temperature profile inversion models and atmospheric humidity profile inversion models using the improved U-Net model;

[0011] The improvements to the U-Net model include three aspects: (1) an attention mechanism is introduced in the skip connections of the four layers corresponding to the decoding and encoding parts. An attention mechanism is introduced when concatenating the feature map output by the encoding sub-module of the same layer with the feature map of the input decoding sub-module on the channel side; (2) a dense connection mechanism is introduced in the first four layers of the encoding part. In each layer of the sub-module, the input feature map is connected to the second feature map obtained by the first convolution on the channel side, and then the second convolution is performed to obtain the feature map of that layer; (3) the convolution in each layer of the encoding part is replaced with depthwise separable convolution.

[0012] An atmospheric temperature profile inversion model is constructed using the U-Net model improved in the three aspects. The input of the model is a 32×32 pixel image with 300 channels, and the output is a temperature profile containing 37 pressure layers with a spatial range of 32×32 pixels.

[0013] An atmospheric humidity profile inversion model is constructed using the U-Net model improved in terms of (1) and (2). The input of the model is a 32×32 pixel image with 300 channels, and the output is a humidity profile containing 37 pressure layers with a spatial range of 32×32 pixels.

[0014] Step 3: Train the constructed atmospheric temperature profile inversion model and atmospheric humidity profile inversion model using the training dataset; during real-time prediction, preprocess the acquired GIIRS channel data of the study area to obtain a 32×32 pixel image set with 300 channels, and then input it into the trained atmospheric temperature profile inversion model and atmospheric humidity profile inversion model respectively to output the atmospheric temperature and humidity profiles of the corresponding spatial areas.

[0015] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0016] (1) The method of this invention uses spaceborne infrared hyperspectral data and leverages the nonlinear mapping capability of deep learning. Using ERA5 reanalysis data, currently the most widely used dataset in the world, as labels, it constructs a model reflecting the intrinsic relationship between spaceborne infrared hyperspectral data and atmospheric temperature and humidity profiles. This allows for the 3D inversion of atmospheric temperature and humidity profiles of the desired region based solely on the given spaceborne infrared hyperspectral data. Experimental verification shows that the model constructed using this method achieves high accuracy in inversion.

[0017] (2) The method of the present invention improves the U-Net model by using a self-attention mechanism module, a dense connection module and a deep separable convolutional module. It considers how to better utilize channel data and deep mining information, strengthens the feature extraction of channel and grid data, and improves the model prediction accuracy. This enables the model to better learn the relationship between spaceborne infrared hyperspectral data and atmospheric temperature and humidity, and improves the accuracy of inversion. When constructing the inversion model, considering that the input GIIRS channel data has thousands of channels, the improved temperature profile inversion network uses depthwise separable convolution instead of ordinary convolution in the same layer convolution operation, effectively reducing the number of parameters and computational cost while maintaining high accuracy and improving generalization level. By using a dense connection mechanism, the two convolution processes and three layers of data in the same layer encoding submodule are processed backward in a dense connection manner. The dense connection mechanism is more aggressive than residual connection, as it can directly accept the data of all previous layers instead of just the fusion result of the previous layer or the first two layers, which is more conducive to information transfer and feature reuse. A self-attention mechanism is introduced to enhance the identification of key regions in the feature map. Considering that the humidity in the atmosphere changes by orders of magnitude with increasing altitude, the improved UNET network used for atmospheric humidity profile inversion does not add depthwise separable convolution, but still uses ordinary convolutional layers to ensure the accuracy of humidity profile inversion.

[0018] (3) In the method of the present invention, PCA principal component analysis is also used to screen the 1650 channels provided by GIIRS channel data. Compared with the currently popular empirical method of screening, the data screened by the present invention is beneficial to significantly improve the accuracy of the atmospheric temperature and humidity profile inversion model.

[0019] (4) When constructing the training dataset, the method of the present invention interpolates the GIIRS channel data and ERA5 data in time and space, making full use of the high quality of ERA5 data and the large number of channels of GIIRS channel data, and realizing the three-dimensional inversion of atmospheric temperature and humidity profile. Attached Figure Description

[0020] Figure 1 This is an overall flowchart of the method for retrieving atmospheric temperature and humidity profiles according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of bilinear interpolation;

[0022] Figure 3 This is a schematic diagram of a self-attention module;

[0023] Figure 4 This is a schematic diagram of depthwise separable convolution;

[0024] Figure 5 This is a schematic diagram of a dense connection mechanism;

[0025] Figure 6 This is a schematic diagram of the network structure of the improved U-Net model in an embodiment of the present invention;

[0026] Figure 7 This is a vertical error profile diagram of two temperature profile inversion models analyzed at the altitude layer according to an embodiment of the present invention;

[0027] Figure 8 This is a schematic diagram showing the inversion effect of two temperature profile inversion models in the horizontal direction in an embodiment of the present invention;

[0028] Figure 9 This is a vertical error profile diagram of two humidity profile inversion models analyzed at the altitude layer according to an embodiment of the present invention;

[0029] Figure 10 This is a schematic diagram illustrating the horizontal inversion effect of two humidity profile inversion models in an embodiment of the present invention. Detailed Implementation

[0030] To clearly and completely present the advantages, technical features, and objectives of this invention, the invention will be described in detail and completely below with reference to the accompanying drawings and embodiments. It should be noted that the listed embodiments are part of this invention, but not all of it. Other embodiments obtained by those skilled in the art without making innovative breakthroughs are all within the scope of protection of this invention.

[0031] like Figure 1 As shown in the figure, the method for retrieving atmospheric temperature and humidity profiles based on FY-4A / GIIRS data using machine learning in this embodiment of the invention mainly includes the following four steps.

[0032] Step 1: Collect GIIRS channel data and ERA5 reanalysis data products provided by FY-4A satellite, match the temporal and spatial resolutions of the two data, and construct a training dataset after preliminary processing.

[0033] The detailed steps for constructing the training dataset in this embodiment of the invention are as follows:

[0034] Step 101: Download GIIRS channel data provided by the FY-4A meteorological satellite within a specified time range as samples and extract sample feature data. The GIIRS channel data completes a full scan of the study area every 2 hours, with a nadir resolution of 16 km. Download hourly ERA5 reanalysis data containing 37 atmospheric temperature and humidity profiles as training label data. The spatial resolution of the ERA5 reanalysis data is 0.25° × 0.25°, and the 37 layers refer to 37 pressure layers within the altitude range of 1000 hPa to 1 hPa. In this embodiment of the invention, the study area is China and its surrounding regions.

[0035] Step 102: Process the GIIRS channel data. Apodization is performed on the GIIRS channel data that has not undergone apodization to improve data accuracy and quality. The apodized radiation data is then converted into brightness temperature data using the Planck inverse function for easier data processing and scientific analysis. The channel data is then stitched together to obtain small-area data suitable for deep learning. Stitching these small-area data together yields large-area data for the entire China region and surrounding areas.

[0036] The following formula is used to perform apodization on the GIIRS channel data:

[0037] r n,new =0.23r n-1 +0.54r n +0.23r n+1

[0038] In the formula r n r represents the emissivity of the nth channel. n-1 r represents the emissivity of the (n-1)th channel. n+1 It is the emissivity of the (n+1)th channel; r n,new This represents the emissivity of the nth channel obtained after apodization. The emissivity of the first and last channels is not processed. After the apodization process, processed radiation data can be obtained. Compared with radiation data, brightness temperature, as a physical quantity, is easier to use for data processing and scientific analysis. Therefore, this invention uses the Planck inverse function to convert GIIRS radiation data into brightness temperature data, as shown in the following formula:

[0039]

[0040] In the formula, t represents the blackbody temperature (K), and v represents the wavenumber (cm). -1 ), r is the blackbody radiation (mW / m 2 –sr–cm 2 c1 = 1.191042 × 10 5 (mW / m 2 –sr–cm -4 c2 = 1.4387752 (K cm).

[0041] Since the GIIRS detector uses a 32×4 pixel array for scanning, each file is actually represented by 32×4 pixels in space and scanned sequentially. In this embodiment of the invention, combining the results of eight adjacent scans yields a 32×32 pixel image. For the study area of ​​this embodiment, the FY-4A meteorological satellite obtains the scan results of the study area by scanning seven latitude lines from north to south (from 60 degrees north latitude to 0 degrees north latitude). Each latitude line can be combined to form five 32×32 pixel images. Considering the large difference in latitude between north and south and data quality issues, this embodiment of the invention mainly considers land areas. Therefore, the acquisition results on the last latitude line are removed during the calculation, resulting in 30 32×32 pixel images obtained from each scan result of the FY-4A meteorological satellite.

[0042] Step 103: The GIIRS brightness temperature data processed in step 102 contains 1650 channels of images, including 689 long-wave and 961 medium-wave. Feature dimensionality reduction is performed on the 1650 channels. The PCA principal component analysis method of machine learning is used to reduce the dimensionality of the 1650 channels. The final sample feature dataset is obtained by dimensionality reduction through the transformation basis composed of feature vectors.

[0043] The main idea of ​​PCA (Principal Component Analysis) is as follows:

[0044] Suppose the original matrix X is as follows:

[0045]

[0046] In the formula x i (1≤i≤p) represents the i-th column vector. The standardized matrix X0 is obtained by standardizing the original matrix to eliminate the dimension and difference between the data.

[0047]

[0048] In the formula This represents the mean of the i-th column. The covariance matrix R can then be calculated as follows:

[0049]

[0050] The superscript T indicates transpose.

[0051] The covariance matrix has the following relationship with its eigenvalues ​​and eigenvectors:

[0052] Rw i =λiw i

[0053] Where λ i w represents the i-th eigenvalue. iLet X represent the i-th eigenvector. Therefore, by applying eigenvalue decomposition and singular value decomposition to R, we can obtain the corresponding eigenvalues ​​and eigenvectors. Since eigenvalue decomposition is only applicable to square matrices, while singular value decomposition is applicable to all matrices, singular value decomposition is generally more convenient for dimensionality reduction of data. For matrix X... m×n Then there exists an orthogonal matrix U. m×m and orthogonal matrix V n×n Make:

[0054] X = U × D × V T

[0055] In the formula, U is the left singular value matrix, V is the right singular value matrix, and D = diag(σ1,σ2,…σr), where σ1,σ2,…σr are the singular values ​​of the matrix. r These are singular values. U and V satisfy the following properties:

[0056] XX T =UDD T U T

[0057] X T X = VD T DV T

[0058] We can obtain that the column vector of U is XX. T The eigenvectors of V are X, and the column vectors of V are X. T The eigenvectors of X.

[0059] When applying PCA for dimensionality reduction, it is only necessary to calculate the right singular matrix V, select appropriate column vectors from V to form the transformation basis, and multiply the transformation basis with the original matrix to obtain the dimensionality-reduced matrix. In this embodiment of the invention, after testing, selecting 300 as the dimensionality after PCA principal component analysis yielded the best training results.

[0060] After the above data processing, the raw GIIRS channel data can yield 5×6, or 30, 32×32 small region images every 2 hours. Considering the quantity of the dataset, in this embodiment of the invention, each 32×32 small region image is taken as a sample. After PCA principal component analysis, 300 channel images are selected, and each sample feature is a 300×32×32 image group.

[0061] Step 104: Match the ERA5 reanalysis data spatially and temporally with the sample dataset obtained in Step 103, and use bilinear interpolation to interpolate the ERA5 data onto the grid of the feature dataset.

[0062] like Figure 2 As shown, the four red dots represent data points, namely Q. 11(x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 (x2, y2), where the green point P(x, y) is the point we want to interpolate. As shown in the graph, applying two linear interpolations yields the values ​​of R1 and R2:

[0063]

[0064]

[0065] The value of point P can be obtained by applying a single linear interpolation in the y-direction based on the values ​​of R1 and R2.

[0066]

[0067] Combining the three formulas above, we can obtain:

[0068]

[0069] When generating the training dataset, bilinear interpolation was used to match label data to each sample in the ERA5 reanalysis data. Label data refers to temperature and humidity profiles containing 37 pressure levels. First, time matching was performed, selecting the label data for the hour closest to the sample data's time. For example, if a sample data point occurred at 00:45 on January 1, 2020, then the label data for 01:00 on January 1, 2020 would be selected. Next, spatial matching was performed, applying bilinear interpolation to interpolate the label data to the spatial location of the sample data, resulting in a one-to-one correspondence between sample and label data pairs. Each processed label data point is formatted as 37×32×32, where 37 represents 37 different pressure levels. Each sample has a 32×32 pixel temperature and humidity profile, with each profile containing the temperature or humidity of 37 pressure levels.

[0070] Step 2: Construct an improved U-Net model for atmospheric temperature and humidity profile inversion.

[0071] A typical U-Net model mainly consists of an encoder on the left, a decoder on the right, and skip connections on both sides. The encoder contains five sub-modules. Except for the last sub-module, the remaining sub-modules first extract feature maps from the input through two convolutional layers, then downsample them through a pooling layer before inputting them into the next sub-module. The last sub-module performs a convolution operation on the input feature map, then upsamples it before inputting it into the decoder. Corresponding to the first four sub-modules of the encoder, the decoder consists of four sub-modules. Each decoder sub-module uses skip connections to concatenate the input feature map with the feature map extracted by the corresponding sub-module of the encoder across channels, then performs a convolution operation on the concatenated feature map, and finally upsamples it before inputting it into the next sub-module. The feature map output by the last sub-module of the decoder is the same size as the image input to the first encoder sub-module.

[0072] The innovative improvements to deep learning models in this invention are as follows: First, addressing the large number of channels in GIIRS channel data, the model is improved by prioritizing infrared channels relevant to temperature and humidity information while minimizing noise. The ordinary skip connections between encoding and decoding are replaced with attention-based connections to enhance focus on relevant information. Second, to fit the differences between meteorological inversion problems and conventional deep learning methods—meteorological information differs from ordinary pixel-based three-channel data, possessing complex and crucial characteristics—a dense connection mechanism is introduced to enhance the model's reuse of feature information, reducing the impact of significant channel information loss during downsampling. Third, considering the resource consumption of the data and other practical application issues, depthwise separable convolutions are used to balance computational speed and model accuracy. This invention improves and matches classic semantic segmentation deep learning networks with practical inversion tasks, thus achieving a key step in successful application to the field of meteorological observation.

[0073] Specifically, in this embodiment of the invention, an improved U-Net model is constructed as a deep learning network model for retrieving atmospheric temperature and humidity profiles. The model improvements mainly include: replacing ordinary convolution with depthwise separable convolution, introducing a dense connection mechanism, and introducing a self-attention mechanism.

[0074] like Figure 3An attention mechanism is introduced, implemented through attention gating. First, two feature maps g and x are transformed to have the same number of channels through a 1×1 convolution. Then, the channels of the two feature maps are added and processed by the ReLU activation function, followed by another 1×1 convolution and a Sigmoid activation function to assign importance scores to specific parts. Finally, this is multiplied by the feature map x to obtain the final output. As mentioned above, this invention inverts atmospheric temperature and humidity information based on hundreds of visible light channels of GIIRS. If the designed model cannot reuse temperature and humidity-related information or reuses other redundant noise information, the inversion accuracy will be reduced. Therefore, the attention mechanism is introduced to highlight the salient regions (i.e., relevant information) of the feature maps and suppress task-irrelevant regions (i.e., noise), forcing the model to have distinctiveness in feature maps of different channels or regions. Figure 6 As shown, in the improved U-Net model of this invention, attention mechanisms are introduced on the skip connections at the corresponding depths of the four layers in the encoding and decoding parts. The feature maps output by the encoding submodule and the feature maps input to the decoding submodule are concatenated on the channel using the attention mechanism, which helps to ensure that the model has the ability to respond to foreground content.

[0075] like Figure 4 Depth-separable convolution is divided into two parts: channel-wise convolution and pointwise convolution. In channel-wise convolution, each kernel is responsible for one channel, and the number of channels is the same as the number of kernels. Pointwise convolution allows for changing the number of channels; it iterates through each pixel, and the kernels all have the same depth. This two-part convolution of depth-separable convolution can significantly reduce the number of convolution parameters. If used properly, it can greatly improve the efficiency of the model without sacrificing performance. Because spaceborne infrared hyperspectral data has a large number of channels, the efficiency during actual inversion needs to be considered. Figure 6 As shown, when constructing the temperature profile inversion model, the method of the present invention replaces the convolution performed in each sub-module of the encoding part with depthwise separable convolution to reduce the number of computational parameters and computational cost, while taking into account both efficiency and performance.

[0076] like Figure 5The dense connection module replaces sequential connections with dense connections. Although it borrows the idea of ​​residual connections from ResNet, dense connections differ from residual connections. First, residual connections only connect to one or a few preceding layers, while dense connections connect to all preceding layers. Second, residual connections simply perform element-wise addition between different layers, while dense connections directly connect channels of feature maps from different layers. These differences give dense connections two major advantages: First, they facilitate backpropagation of gradients during training, reducing gradient vanishing. This is because during backpropagation, the signal from one layer is received by all subsequent layers, preventing signal decay. Therefore, dense connections do not exacerbate gradient problems by increasing the number and depth of layers. Second, dense connections enable extensive feature reuse. Each layer receives signals from all preceding layers, thus preserving rich feature information. This effectively compensates for the significant loss of channel information with each downsampling of GIIRS channel data. Therefore, as... Figure 6 As shown, the improved model designed by the present invention introduces dense connection operations in the first four sub-modules of the encoding part. That is, in each sub-module, the feature map input to the layer is connected to the second feature map obtained by the first depthwise separable convolution (or ordinary convolution) through a channel, and then the second depthwise separable convolution (or ordinary convolution) is performed to output the feature map of the layer.

[0077] like Figure 6 As shown, the overall functionality of the improved U-Net neural network of this invention can be summarized as consisting of a feature extraction part, an upsampling part, in-layer connections, and an output part. The feature extraction part reduces the size of the feature map and extracts features; the input image undergoes downsampling and convolution. The upsampling part restores the image size and increases the receptive field of the filter. After feature extraction, the data is sequentially connected in-layer, passed through two convolutional layers, upsampled, and then fed into the next layer, undergoing the same operation until it returns to the first layer. In-layer connections directly connect the feature-extracted data and the upsampled data within the same layer, allowing for the reuse of feature maps and obtaining more accurate information. The output part restores the changed image size to its original size.

[0078] It is important to note that the method of this invention establishes two inversion models for temperature profiles and humidity profiles, respectively. Considering that the temperature value varies relatively little across the entire altitude range, while the humidity value changes by orders of magnitude with increasing altitude, the improved U-Net model is used for the temperature profile inversion model, while the humidity profile inversion model does not introduce depthwise separable convolution but still uses ordinary convolution, introducing dense connection and self-attention mechanisms.

[0079] like Figure 6 As shown, this is an improved U-Net neural network designed for temperature profile inversion according to an embodiment of the present invention. The input sample is a 300×32×32 feature map, where 300 represents the number of channels and 32×32 represents the feature map size. After passing through four sub-modules of the encoding part, feature maps of 64×32×32, 128×16×16, 256×8×8, and 512×4×4 are obtained sequentially. Then, after a depthwise separable convolution operation in the last encoding sub-module, upsampling is performed, and a 512×4×4 feature map is output to the first layer sub-module of the decoding part. In the decoding part, the feature map is concatenated with the corresponding layer extracted from the encoding part and then subjected to a normal convolution operation until a 37×32×32 temperature profile inversion result is output.

[0080] Step 3: Train the improved deep learning network model using the training dataset to obtain the atmospheric temperature profile inversion model and the atmospheric humidity profile inversion model.

[0081] In this embodiment of the invention, pre-constructed GIIRS feature data is used as input and ERA5 label data is used as output. Specifically, 32×32 pixel feature data from 300 channels within the same region and time range are input, and 32×32 atmospheric temperature profiles and 32×32 atmospheric humidity profiles with 37 layers are obtained through temperature profile inversion models and humidity profile inversion models, respectively. Since the two tasks of temperature and humidity inversion are different and the models are slightly different, the atmospheric temperature inversion model and humidity inversion model are trained independently. The mean absolute error (MAE) and root mean square error (RMSE) functions are used as the model evaluation functions, and the parameters of the deep learning network are trained using a variable step-size learning rate. The optimizer during training uses the Adam gradient descent method, setting an initial learning rate and parameters to represent the degree of learning rate decay. After iterative iteration, the optimal parameter model is obtained. The deep learning network model with the optimal parameters is saved as the atmospheric temperature and humidity profile inversion model.

[0082] The formulas for calculating MAE and RMSE are as follows:

[0083]

[0084]

[0085] Where n represents the number of samples in the test set, y i This represents the observation value of the i-th sample in the test set. This represents the prediction inversion result for the i-th sample.

[0086] Step four involves preprocessing the complete GIIRS data of China and surrounding areas collected at a certain time. This includes performing apodization on the GIIRS channel data, converting it into brightness temperature data using the Planck inverse function, performing image stitching, and converting the acquired GIIRS data into a set of 30 images with 300 channels and 32×32 pixels each through transformation basis dimensionality reduction. Then, the data is predicted using the trained atmospheric temperature profile inversion model and atmospheric humidity profile inversion model, respectively, outputting 30 32×32 pixel temperature profile and humidity profile images. These images are then stitched together to obtain the atmospheric temperature and humidity profiles for the entire study area.

[0087] The method of the present invention was experimentally verified. The experimental data consisted of 6890 processed GIIRS channel data points from January 1st to 31st, 2020 (31 days) as the training set, with 2100 processed data points selected at 7-day intervals as the test set for experimental verification.

[0088] All experimental data were preprocessed using the method described in step 1 of this invention. Then, the atmospheric temperature profile inversion model and atmospheric humidity profile inversion model constructed in this invention were trained. To further illustrate the inversion effect of the improved U-Net network-built model, based on the same experimental data and the same data preprocessing as this invention, the inversion effect of using the unmodified U-Net network to construct the atmospheric temperature profile inversion model and atmospheric humidity profile inversion model with the preprocessed data is also provided, and the inversion effect is compared with the model constructed in this invention. The model constructed by the improved U-Net network in this invention is labeled IM-UNET, and the comparison model constructed by the original U-Net network is labeled UNET.

[0089] like Figure 7 As shown, the vertical axis represents height, the horizontal axis of the left graph is the mean square error (MAE) of the inversion results, and the horizontal axis of the right graph is the root mean square error (RMSE) of the inversion results. The dashed line corresponds to the temperature inversion model constructed by the method of this invention, and the dotted line corresponds to the temperature inversion model constructed by the original U-Net network. From Figure 7 As can be seen, the root mean square error (RMSE) of the inversion results of the model of this invention is lower than that of the comparative model across all altitudes. Except for the range of 870hPa-1000hPa, where the RMSE is higher than 1.5K, the RMSE of other altitude layers is lower than 1.5K, and ideally reaches an accuracy of around 1K. Regarding the average error, the model of this invention exhibits negative bias below 200hPa and above 460hPa, and positive bias in the middle range; while the comparative model, except for a small area in the upper reaches and the range above 800hPa, exhibits the opposite bias to the model of this invention. Comparing the magnitudes of the biases, except for the ranges below 170hPa and 270hPa-440hPa, the biases of the model of this invention are relatively close, with the bias of the model of this invention being smaller than that of the comparative model.

[0090] like Figure 8 As shown, the horizontal temperature fields of the two models and ERA5 at 1000 hPa are illustrated. (a) represents the ERA5 temperature field, (b) represents the temperature field retrieved by the comparative model, and (c) represents the temperature field retrieved by the model of this invention. It can be seen that the temperature field retrieved by the comparative model is somewhat overestimated in the upper left region of the image (62°E–80°E, 45°N–55°N); there are gaps near 132°E and 53°N, with the retrieved temperature below the lower limit of 250K; in the range of 100°E–110°E, 28°N–34°N, the retrieved temperature field is severely overestimated on the right side, with no transition at its right and upper right boundaries. Apart from this range, the overall temperature overestimates the temperature within the Tibetan Plateau region. In contrast, the temperature field retrieved by the model of this invention has a smoother transition at the edges of the stitched image, and its overall inversion effect is closer to the ERA5 temperature field.

[0091] like Figure 9 As shown, the vertical axis represents height, the horizontal axis in the left graph represents the average error of the inversion results, and the horizontal axis in the right graph represents the root mean square error of the inversion results. The dashed line corresponds to the humidity inversion model constructed by the method of this invention, and the dotted line corresponds to the humidity inversion model constructed by the original U-Net network. From Figure 9 As can be seen, except for the range of 230 hPa to 150 hPa where the model of this invention exhibits a negative bias and the comparative model exhibits a positive bias, the positive and negative bias trends of the two models are basically consistent at other altitudes, and the bias of the former shrinks faster in the troposphere. Except for the range of 210 hPa to 501 hPa where the model of this invention exhibits a slightly higher bias, the bias of the model of this invention is better than that of the comparative model in all other ranges. Across the entire altitude range, the root mean square error of the model of this invention is smaller than that of the comparative model, and the maximum error of the model of this invention does not exceed 1.0 g / kg.

[0092] like Figure 10 As shown, the horizontal humidity fields of the two models and ERA5 data at an altitude of 1000 hP are illustrated. (a) represents the ERA5 humidity field, (b) represents the humidity field retrieved by the comparative model, and (c) represents the humidity field retrieved by the model of this invention. It can be seen that the humidity field retrieved by the comparative model is somewhat underestimated in the region of 120°E–130°E and 9°N–11°N, and somewhat overestimated near the East China Sea. Around 15°N, there is a significant difference along the entire latitude line, with obvious stitching at the junctions and a lack of smooth transitions. The humidity fields at 850 hPa and 600 hPa exhibit the same problem. In contrast, the temperature field retrieved using the model of this invention shows less noticeable stitching, i.e., smoother boundary transitions, and its overall retrieved effect is closer to the ERA5 temperature field.

[0093] The above-disclosed embodiments illustrate how those skilled in the art can implement or use the present invention. Modifications and imitations of the embodiments of the present invention will be readily apparent to such persons, and the general principles described herein can still be implemented in other embodiments. Therefore, the principles and techniques disclosed in this invention are not limited to the embodiments shown, but should be accorded the maximum scope consistent with their characteristics.

Claims

1. A method for retrieving atmospheric temperature and humidity profiles based on machine learning using FY-4A / GIIRS data, characterized in that, Includes the following steps: Step 1: Download the GIIRS channel data and ERA5 reanalysis data provided by the FY-4A meteorological satellite for the specified time range of the study area, and construct the training dataset after preprocessing. The process of preprocessing GIIRS channel data to generate a sample feature dataset includes: performing apodization on the GIIRS channel data, then converting it into brightness temperature data using the Planck inverse function, and using PCA principal component analysis to reduce the dimensionality of the 1650 channels of the GIIRS brightness temperature data to obtain a 300-channel, 32×32 pixel image for each sample; and generating label data for each sample from ERA5 reanalysis data, which includes temperature and humidity profiles. Step 2: Construct atmospheric temperature profile inversion models and atmospheric humidity profile inversion models using the improved U-Net model; The improvements to the U-Net model include three aspects: (1) an attention mechanism is introduced in the skip connections of the four layers corresponding to the decoding and encoding parts. An attention mechanism is introduced when concatenating the feature map output by the encoding sub-module of the same layer with the feature map of the input decoding sub-module on the channel side; (2) a dense connection mechanism is introduced in the first four layers of the encoding part. In each layer of the sub-module, the input feature map is connected to the second feature map obtained by the first convolution on the channel side, and then the second convolution is performed to obtain the feature map of that layer; (3) the convolution in each layer of the encoding part is replaced with depthwise separable convolution. An atmospheric temperature profile inversion model is constructed using the U-Net model improved in the three aspects. The input of the model is a 32×32 pixel image with 300 channels, and the output is a 32×32 pixel temperature profile containing 37 pressure layers in the corresponding spatial range. An atmospheric humidity profile inversion model is constructed using the U-Net model improved in terms of (1) and (2). The input of the model is a 32×32 pixel image with 300 channels, and the output is a humidity profile containing 37 pressure layers with 32×32 pixels in the corresponding spatial range. Step 3: Train the constructed atmospheric temperature profile inversion model and atmospheric humidity profile inversion model using the training dataset; during real-time prediction, preprocess the acquired GIIRS channel data of the study area to obtain a 32×32 pixel image set with 300 channels, and then input it into the trained atmospheric temperature profile inversion model and atmospheric humidity profile inversion model respectively to output the atmospheric temperature and humidity profiles of the corresponding spatial areas.

2. The method according to claim 1, characterized in that, In step 1, the GIIRS channel data is apodized using the following formula: r n,new ＝0.23r n-1 +0.54r n +0.23r n+1 Where, r n r represents the emissivity of the nth channel. n-1 r represents the emissivity of the (n-1)th channel. n+1 It is the emissivity of the (n+1)th channel, r n,new This represents the emissivity of the nth channel after apodization. The emissivity of the first and last channels is not processed.

3. The method according to claim 1, characterized in that, In step 1, the GIIRS detector on the FY-4A meteorological satellite scans using a 32×4 pixel array. The results of eight adjacent scans are stitched together to obtain a 1650-channel 32×32-pixel image. The study area is composed of multiple 32×32-pixel images. The dimensionality of the 1650-channel image is reduced using PCA principal component analysis, and the dimensionality after reduction is selected as 300.

4. The method according to claim 1, characterized in that, In step 1, atmospheric temperature and humidity profiles of 37 pressure layers within the altitude range of 1000 hPa to 1 hPa are obtained from the ERA5 reanalysis data. Then, label data is matched for each sample. First, time matching is performed, selecting the ERA5 reanalysis data of the nearest whole point in time to the sample. Then, spatial matching is performed, using bilinear interpolation to interpolate the atmospheric temperature and humidity to the spatial location of the sample. For each sample, the obtained label data is a temperature profile and humidity profile containing 37 pressure layers, both represented in a 37×32×32 data format. That is, each sample corresponds to a 32×32 pixel temperature profile and humidity profile, and each temperature or humidity profile has the temperature or humidity of 37 pressure height points.

5. The method according to claim 1, characterized in that, In step 3, when training the atmospheric temperature profile inversion model and the atmospheric humidity profile inversion model, the mean absolute error and root mean square error are used as the evaluation criteria for training, and the network parameters are trained using a variable step size learning rate.

Images