GNSS and Transform-based refined remote sensing atmospheric water vapor inversion method

Abstract

The invention discloses a refined remote sensing atmospheric water vapor inversion method based on a GNSS and a Transform. The method comprises the following steps: acquiring OLCI remote sensing data, ERA5 meteorological data and GNSS-PWV data of a target area; performing quality control on the OLCI remote sensing data, and screening effective observation data; performing space-time registration on the OLCI remote sensing data, the ERA5 meteorological data and the GNSS-PWV data, and constructing a multi-dimensional feature vector; inputting the multi-dimensional feature vector into a Transform model for training, and learning a coupling relationship between features through a self-attention mechanism; and outputting an atmospheric precipitable water quantity inversion result based on the trained Transform model. According to the invention, the precision and spatial resolution of atmospheric water vapor detection are improved.

Description

Technical Field

[0001] The present invention belongs to the fields of remote sensing meteorological data processing, satellite navigation meteorology and artificial intelligence software technology, and in particular relates to a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer. Background Art

[0002] Precipitable water vapor (PWV) is a crucial meteorological element for quantitatively describing the spatiotemporal distribution of water vapor and assessing convective potential and precipitation evolution. With the rapid development of refined short-term forecasts, global climate change monitoring, and water cycle research, businesses urgently need PWV products with high spatiotemporal resolution, real-time updates, and stable accuracy for diverse scenarios such as numerical weather forecast data assimilation, rainstorm monitoring and early warning, and drought assessment.

[0003] Currently, water vapor detection is achieved through a variety of methods, including radiosonde detection, ground-based GNSS inversion, and remote sensing observations. Radiosondes offer high accuracy, but their temporal and spatial resolution is very low, making them inadequate for real-time water vapor monitoring. Ground-based GNSS inversion of the spatial distribution of water vapor requires interpolation, and its accuracy is limited by the density of GNSS stations. Water vapor products provided by remote sensing satellites have an error of approximately 10%, which cannot meet the needs of high-precision water vapor detection. These conclusions indicate that a single detection technology cannot meet the current demand for high-precision, high-temporal and high-resolution water vapor monitoring. To obtain two-dimensional water vapor data with good real-time performance, high spatial resolution, and high accuracy, the integration of meteorological fields from numerical models such as OLCI multispectral radiance and ERA5, along with GNSS-PWV calibration, to construct high-precision water vapor inversion models has become a research hotspot in remote sensing meteorology, both domestically and internationally.

[0004] The Ocean and Land Colour Instrument (OLCI) on the Sentinel-3 satellite provides 21 visible-to-near-infrared bands covering 400–1020 nm at 300-meter resolution. The combination of bands 18 (strong water vapor absorption at 885 nm) and 19 (900 nm reference window) is the most sensitive to atmospheric water vapor and is considered the core remote sensing data source for large-scale water vapor retrieval following MERIS. Research has confirmed a significant nonlinear relationship between the radiance ratio of bands 18 and 19 in the OLCI and water vapor. This relationship can be modeled using a fitting formula to achieve the conversion of remote sensing data into water vapor. The global ground-based GNSS atmospheric sounding network can provide the PWV values ​​necessary for satellite-based retrieval algorithms. In recent years, with the rapid development of artificial intelligence, methods for inverting water vapor using neural networks that combine remote sensing radiance data with GNSS-PWV have rapidly developed. This method uses the precipitable water volume (PWV) inverted from the GNSS station network spread across the ground as the high-precision "true value" or supervision signal, and the spectral radiation brightness, geographic location parameters, and assimilated meteorological fields (temperature, pressure, etc.) obtained by satellite remote sensing as input features. Through end-to-end learning of the nonlinear mapping between multi-source observations and PWV by deep neural networks, pixel-level or grid-level water vapor inversion is achieved.

[0005] Currently, most approaches utilize convolutional neural networks, multilayer perceptrons, long-memory neural networks, and other approaches. However, these approaches still exhibit the following key shortcomings in terms of algorithmic capabilities and software engineering: First, they are limited in their ability to model long-range spatial dependencies. Due to their fixed local receptive fields, convolutional neural networks can only capture textures in neighboring pixels; recurrent networks are prone to gradient vanishing in long sequences. Consequently, they lack a comprehensive understanding of regional-scale water vapor distribution. Second, their ability to capture multi-source input features is limited. Currently used neural networks lack a multi-head attention mechanism, making it difficult to capture complex nonlinear relationships between input and output, resulting in low model accuracy. Third, their ability to generalize across regions is insufficient. Current research is largely limited to small-scale, short-term studies, often focusing on a single, narrow region. Because data size significantly impacts neural network training, research on water vapor inversion using massive amounts of data is needed. Therefore, a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformers is urgently needed. Summary of the Invention

[0006] To solve the above technical problems, the present invention proposes a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer, which can characterize the complex spectral absorption-meteorological coupling relationship, is superior to the above neural network, and can increase the spatial resolution of water vapor products to 300m, so as to make up for the problem that GNSS water vapor inversion is limited by station density and the low accuracy of remote sensing water vapor products, and generate refined water vapor products with high real-time performance and good accuracy.

[0007] To achieve the above objectives, the present invention provides a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer, comprising:

[0008] Acquire OLCI remote sensing data, ERA5 meteorological data, and GNSS-PWV data for the target area;

[0009] Performing quality control on the OLCI remote sensing data and screening valid observation data;

[0010] Performing spatiotemporal registration on the OLCI remote sensing data, ERA5 meteorological data, and GNSS-PWV data to construct a multidimensional feature vector;

[0011] Input the multidimensional feature vector into the Transformer model for training, and learn the coupling relationship between features through the self-attention mechanism;

[0012] The inversion results of atmospheric precipitable water are output based on the trained Transformer model.

[0013] Optional quality control procedures include:

[0014] Parse the binary code of OLCI data quality file LQSF;

[0015] Distinguish clear sky, partially cloudy and fully obscured pixels based on the coding results;

[0016] Only observations under completely clear sky conditions are retained.

[0017] Optionally, the spatiotemporal registration process includes:

[0018] Set the registration thresholds to ≤30 minutes for time difference and ≤1 km for spatial distance;

[0019] The ERA5 meteorological parameters were resampled to the OLCI grid using the temporal nearest neighbor and spatial nearest neighbor methods;

[0020] Match the GNSS-PWV data to the corresponding OLCI pixel locations.

[0021] Optionally, a multidimensional feature vector includes:

[0022] Radiance of the OLCI water vapor sensitive band and window band;

[0023] Air pressure and atmospheric weighted mean temperature provided by ERA5;

[0024] The longitude, latitude and elevation information of the observation point.

[0025] Optionally, the training process of the Transformer model includes:

[0026] Perform linear projection on the input features to obtain the embedding vector;

[0027] Perform multi-head self-attention calculations through multi-layer encoders;

[0028] A feedforward network is used to perform nonlinear transformation on the attention calculation results.

[0029] Optionally, the training process also includes:

[0030] Calculate the Huber loss between the predicted value and the true value of GNSS-PWV;

[0031] Add spatial smooth regularization term to constrain the continuity of output results;

[0032] The data error term and the regularization term are weighted and combined into a comprehensive loss function.

[0033] Optionally, the training process uses a gradient accumulation strategy including:

[0034] Divide the training batch into multiple sub-batches;

[0035] Accumulate the gradient calculation results of each sub-batch;

[0036] When the cumulative number reaches the set value, the parameter is updated.

[0037] Optionally, the model training termination condition is: there is no significant decrease in the RMSE of the validation set for 5 consecutive training cycles.

[0038] Technical effects of the present invention:

[0039] (1) Self-attention global modeling significantly improves inversion accuracy

[0040] Traditional LSTM gradients easily dissipate in long sequences, and convolution relies on a fixed field of view. However, the Transformer's multi-head self-attention computes the correlations between all elements in a single forward pass, naturally preserving complex coupling relationships across long distances, channels, and even modalities. When applied to high-dimensional, nonlinear problems like water vapor inversion, it can directly capture the holistic connection between remote sensing spectra, observation geometry, and meteorological fields. Multi-head self-attention is introduced to model global dependencies across pixels across the entire scene. Compared to traditional local convolution or linear regression, it can simultaneously capture the multi-dimensional, nonlinear coupling relationships between meteorological data, remote sensing radiance data, and observation geometry.

[0041] (2) Fusion of multi-source meteorological priors improves cross-regional generalization capabilities

[0042] This method incorporates reanalysis variables such as ERA5 air pressure and atmospheric weighted mean temperature as auxiliary channels and employs an adaptive normalization strategy within the network. The inclusion of these meteorological elements enables the model to dynamically adjust to climatic conditions in different climate zones. Field measurements demonstrate that the water vapor accuracy of the model output surpasses that of remote sensing data, improving the accuracy and spatial resolution of atmospheric water vapor detection and helping to meet the needs of refined water vapor detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0044] Figure 1 This is a flow chart of a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the specific implementation process of a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] like Figure 1-Figure 2As shown, this embodiment provides a refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer, including:

[0049] Prepare raw S1, GNSS, ERA5, and remote sensing OLCI observation data. Obtain OLCI radiance and quality files (LQSF) for any water vapor-sensitive band and window band (λ≈885nm, λ≈900nm) over land and ocean. Obtain ERA5 air pressure (P) and atmospheric weighted mean temperature (Tm) for the same time period. Simultaneously obtain GNSS tropospheric delays and convert them to PWVs (Purple Wavelengths) to serve as a benchmark for the water vapor output model's performance.

[0050] S2. Remote sensing OLCI data quality control. The quality file (LQSF) provided by the source is decoded to produce a 32-bit binary code. The remote sensing file at that moment is classified as "clear sky," "partially cloudy," or "completely obscured" based on the number of "0s" and "1s" in the specified bit. This method only uses observations under completely clear sky conditions. Then, the 3 sigma rule of thumb is used to remove outliers with large deviations.

[0051] S3. Multi-source data registration and standardization. Complete spatiotemporal registration of the three-source data using the criteria of imaging time difference ≤ 30 minutes and spatial distance ≤ 1 km. Perform logarithmization on the spectral data to ensure uniform input dimensions. Use the remote sensing data preprocessing module to filter out data points obscured by clouds and of poor quality. Use the meteorological matching module to resample ERA5 variables such as pressure and atmospheric weighted mean temperature to the OLCI grid using the nearest neighbor in time and nearest neighbor in space. Use the GNSS calculation component to complete the PWV calculation and generate the PWV raster.

[0052] S4. Feature vector construction. Generate a seven-dimensional vector for each pixel:

[0053] x=[I 885 , I 900 ,P,T m , Lat, Lon, Alt];

[0054] Among them, I 885 , I 900 is the radiation brightness of the two bands, P is the pressure, T m is the weighted average temperature of the atmosphere, and Lat, Lon, and Alt are the longitude, latitude, and elevation of each pixel.

[0055] S5. Transformer modeling.

[0056] S5-1. Feature Mapping. Perform linear projection on the input seven-dimensional vector x to obtain a d-dimensional embedding vector:

[0057] z=W ex+b e ,W e ∈R d×7 ,b e ∈R d ;

[0058] S5-2, Transformer encoding. Expand each training sample into a token sequence of length 1:

[0059] Z = [z];

[0060] After multi-head self-attention and feedforward network:

[0061]

[0062] Among them, MSA is the multi-head self-attention inside the Transformer, FFN is the feedforward network, which means each token has two independent layers of MLP, and l is the number of encoder layers. The average value of the sequence is:

[0063] h=mean(Z (L) , axis=1)∈R d ;

[0064] Finally, the above results are sent back to the internal encoder to obtain grid water vapor data:

[0065]

[0066] Among them, b r is the bias term of the linear regression output layer, which is used to compensate for the overall translation of the linear combination of weights. is the regression weight vector, and h is the d-dimensional feature after Transformer encoding.

[0067] S5-3. Loss function and optimization. This method will predict the value The error is measured with the true supervised value y, and all model parameters are iteratively updated using gradient descent. To balance robustness and scalability, this method adopts a combined strategy of conforming loss + AdamW + gradient accumulation to ensure millimeter-level precipitable water volume (PWV) accuracy and accelerate convergence.

[0068] The Huber loss is calculated between the predicted value and the supervised true value of the same batch of pixels, which is defined as:

[0069]

[0070] The threshold δ can be set anywhere in the range of 0.5–2 mm, with a preferred value of 1 mm. This loss maintains quadratic smoothness in small error ranges and grows linearly in large error ranges, achieving both robustness and differentiability.

[0071] For known random noise, add the gradient square regularization to the predicted data on the grid:

[0072]

[0073] This regularization term is passed through the weight coefficient λ (take 10 -3 -10 -2 ) is linearly combined with the data error term to obtain the comprehensive loss:

[0074] L=L data +λL smooth ;

[0075] To use a larger equivalent batch size if the graphics memory allows, the present invention divides the actual batch B into K sub-batches. For each sub-batch, the loss is calculated according to the above method and the gradient is accumulated. When the accumulation reaches K times or the end, another parameter update is performed.

[0076] S5-4. Parameter update. Use an adaptive moment estimation optimization algorithm with a weight decay term. Let the current gradient be g, the first-order and second-order moments be m and v, the learning rate be η, and the decay coefficients be β1 and β2. The update formula is:

[0077]

[0078] Among them, λ w is the weight decay rate.

[0079] S6. Output the results. Stop training when the validation set RMSE does not decrease significantly within 5 consecutive epochs.

[0080] Specifically, starting from the collection of raw observation data, first, it is necessary to download the OLCI first-level data products and data quality products for the target area and target time range from the satellite data center; then, it is necessary to synchronize the quasi-GNSS station observation data and the IGS orbit clock product, and solve the zenith tropospheric delay ZTD at a 30s sampling rate; finally, use the GMET software to obtain the ERA5 meteorological reanalysis file at the GNSS station and extract the surface air pressure and weighted average temperature.

[0081] According to the air pressure and atmospheric weighted mean temperature in ERA5, the zenith wet delay component is extracted in ZTD. The formula is as follows:

[0082]

[0083] ZWD=ZTD-ZHD;

[0084]

[0085] PWV = Π × ZWD;

[0086] Where P is the ground pressure at the measuring station; h is the elevation of the measuring station; is the latitude of the measuring station; k′2 and k′3 are atmospheric refraction constants; ρ w is the density of liquid water; T m is the weighted mean temperature of the atmosphere.

[0087] Afterwards, spatiotemporal registration of the multi-source data is performed. The registration thresholds in this invention are a time difference of 30 minutes or less and a spatial distance difference of 1 kilometer or less. Pixels that do not meet the threshold are marked as missing, and GNSS-PWV is then matched to each grid point based on this principle. At this point, the seven variables—the two-band radiance of the remote sensing OLCI, the weighted average atmospheric temperature, the longitude, latitude, and elevation of the station, and the air pressure—are formed into a seven-dimensional vector. This serves as the model input, and the GNSS-PWV is the expected output, completing the acquisition and preprocessing of the raw data.

[0088] The obtained seven-dimensional vector is feature mapped and then fed into the Transformer model. Within the model, each independent image is divided into separate patches based on hardware conditions, and each patch generates a token sequence element. A multi-head self-attention and feedforward network is stacked on the token sequence, and the residual-normalization structure of the lth layer is:

[0089] H (l) =MSA(Z (l-1) )+Z (l-1) ;

[0090] Z (l) =FFN(H (l) )+H (l) ;

[0091] The number of attention heads used in MSA and the width of the FFN hidden layer can be set based on the amount of data being calculated and the computing power of the computer. The first step, MSA, utilizes a multi-head mechanism to capture global dependencies between dimensions within the sequence, overcoming the limitation of traditional convolutional methods that can only capture local information. The second step, FFN, performs nonlinear projection and remapping on each token, enhancing the model's representational capabilities. After outputting the vector h for each patch, a linear regression head is used to calculate the PWV value for each patch using the following formula:

[0092]

[0093] The patch-PWV results are fused in overlapping areas using Hann weights and back-projected to pixel resolution to obtain a two-dimensional PWV field of a complete image.

[0094] After the model outputs PWV, its data error and spatial smooth regularization are combined:

[0095] L=L Huber +λL smooth ,λ=10 -3 ;

[0096] Using gradient accumulation techniques, we set an appropriate batch size based on the amount of data being calculated or hardware requirements to accommodate the computer's video memory. When gradient overflow occurs, the scaling factor is automatically reduced and model parameters are updated. Training is terminated when the validation set RMSE shows no significant decrease for five consecutive epochs. The water vapor model is now complete. Water vapor data for the target region and time can be obtained by inputting the required seven-dimensional vector.

[0097] In summary, this paper addresses the challenges of existing remote sensing OLCI water vapor inversion techniques, which struggle to accurately describe the high-dimensional nonlinear coupling between spectra, meteorology, and geometry, and exhibit significant variations in accuracy under different weather and regional conditions. By doing so, we propose a one-step, multi-source fusion PWV inversion method based on Transformer self-attention. This method directly constructs an end-to-end mapping from "radiance in the water vapor-sensitive band + ERA5 meteorological priors → GNSS-PWV ground truth," leveraging multi-head self-attention during training to simultaneously learn long-range spatial correlation and nonlinear absorption mechanisms. This method effectively avoids the significant systematic errors associated with conventional wet projection or local convolutional models due to inappropriate assumptions, resulting in high accuracy in the RMSE of the output PWV relative to the reference PWV across diverse climate conditions and excellent cross-regional generalization. Furthermore, the overall process, consisting of four steps: registration, embedding, inference, and publication, combines gradient accumulation with mixed precision to accelerate model convergence, making it particularly suitable for scenarios such as refined meteorological monitoring, numerical forecasting, and disaster warning.

[0098] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer, characterized by: include: Acquire OLCI remote sensing data, ERA5 meteorological data, and GNSS-PWV data for the target area; Performing quality control on the OLCI remote sensing data and screening valid observation data; Performing spatiotemporal registration on the OLCI remote sensing data, ERA5 meteorological data, and GNSS-PWV data to construct a multidimensional feature vector; Input the multidimensional feature vector into the Transformer model for training, and learn the coupling relationship between features through the self-attention mechanism; The inversion results of atmospheric precipitable water are output based on the trained Transformer model.

2. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: The quality control process includes: Parse the binary code of OLCI data quality file LQSF; Distinguish clear sky, partially cloudy and fully obscured pixels based on the coding results; Only observations under completely clear sky conditions are retained.

3. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1 is characterized in that: The process of spatiotemporal registration includes: Set the registration thresholds to ≤30 minutes for time difference and ≤1 km for spatial distance; The ERA5 meteorological parameters were resampled to the OLCI grid using the temporal nearest neighbor and spatial nearest neighbor methods; Match the GNSS-PWV data to the corresponding OLCI pixel locations.

4. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: Multidimensional feature vectors include: Radiance of the OLCI water vapor sensitive band and window band; Air pressure and atmospheric weighted mean temperature provided by ERA5; The longitude, latitude and elevation information of the observation point.

5. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: The training process of the Transformer model includes: Perform linear projection on the input features to obtain the embedding vector; Perform multi-head self-attention calculations through multi-layer encoders; A feedforward network is used to perform nonlinear transformation on the attention calculation results.

6. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: The training process also includes: Calculate the Huber loss between the predicted value and the true value of GNSS-PWV; Add spatial smooth regularization term to constrain the continuity of output results; The data error term and the regularization term are weighted and combined into a comprehensive loss function.

7. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: The training process uses the gradient accumulation strategy including: Divide the training batch into multiple sub-batches; Accumulate the gradient calculation results of each sub-batch; When the cumulative number reaches the set value, the parameter is updated.

8. The refined remote sensing atmospheric water vapor inversion method based on GNSS and Transformer according to claim 1, characterized in that: The termination condition for model training is that there is no significant decrease in the RMSE of the validation set after 5 consecutive training cycles.

Images