Hurricane cycle assimilation prediction method and system based on spatiotemporal attention reinforcement learning

Abstract

The application discloses a hurricane circulation assimilation prediction method and system based on space-time attention reinforcement learning, and relates to the technical field of tropical cyclone prediction. The steps are as follows: formalizing the assimilation-prediction circulation process into a Markov decision process, constructing a hurricane assimilation-prediction simulation environment based on WRFDA-WRF to provide a physically consistent interactive environment for reinforcement learning; fusing three-dimensional convolution, spatial attention and a gated recurrent unit to construct a deep space-time encoder to extract the space-time features of the assimilation analysis field, and obtaining a high-dimensional state embedding with physical constraints and statistical representation capabilities; inputting the high-dimensional state embedding into an enhanced proximal policy optimization network based on an Actor-Critic framework to adaptively adjust the mixed background error covariance matrix weight in the circulation assimilation-prediction, and realizing the mixed weight regulation. The application realizes the precision of the tropical cyclone path and intensity prediction by dynamically adjusting the WRFDA background error covariance weight.

Description

Technical Field

[0001] This invention relates to the field of tropical cyclone forecasting technology, and in particular to a hurricane cycle assimilation forecasting method and system based on spatiotemporal attention reinforcement learning. Background Technology

[0002] Tropical cyclones, especially hurricanes, are among the most destructive natural disasters globally, and improving the accuracy of their track and intensity forecasts is crucial for disaster prevention and mitigation. In Earth science research, mesoscale numerical weather prediction systems, represented by Weather Research and Forecasting (WRF) models, have become core tools for simulating and predicting hurricanes, capable of finely characterizing atmospheric dynamics and physical processes. However, despite continuous advancements in computational capabilities and data utilization techniques, significant uncertainties remain in tropical cyclone forecasts, particularly in reproducing key processes such as intensity evolution, core structure, and rapid intensification. Improving the quality of the initial field is key to enhancing WRF forecast performance, and data assimilation (DA), by fusing short-term numerical forecasts with multi-source remote sensing observations, provides an effective way to construct high-precision initial fields. In DA systems, the background error covariance matrix (B) determines the quality of the analytical field; therefore, optimizing the background error covariance matrix to construct a high-quality, dynamically-physical consistent initial field has become a critical issue for improving tropical cyclone track and intensity forecasts.

[0003] Regarding the optimization of B, data-driven deep learning (DL) methods, with their advantages in high-dimensional nonlinear feature extraction, have been widely applied to typhoon track and intensity prediction, significantly improving prediction accuracy within an end-to-end regression framework. However, deep learning methods formalize the forecasting problem as a supervised regression task, failing to explicitly introduce the physical constraints provided by data assimilation (DA) and the B matrix. This results in a lack of dynamic consistency and interpretability in the results, especially in rapidly intensifying or weakening hurricane scenarios, where the regression smoothing effect often underestimates the magnitude of extreme changes. While deep learning fully utilizes the spatiotemporal characteristics of data, the lack of physical constraints makes it difficult to solve core problems, necessitating intelligent dynamic optimization methods that integrate the advantages of both. Reinforcement learning (RL) provides a feasible path for this. Unlike the supervised regression paradigm of deep learning methods, RL formalizes the problem as a Markov decision process (MDP), using a closed-loop interactive online learning strategy of "state-action-reward" to replace empirical parameter tuning with physical or business objectives as rewards. Deep reinforcement learning (DRL), by combining feature extraction from deep learning with decision optimization from reinforcement learning, has demonstrated its ability to handle non-stationarity and high-dimensional spatiotemporal complexity in multi-agent systems. However, its potential for optimizing the mixed weights of the B matrix in tropical cyclone data assimilation remains unexplored and requires further research, becoming a pressing issue for those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a hurricane cyclic assimilation forecasting method and system based on spatiotemporal attention reinforcement learning to solve the problems mentioned in the background art. It aims to improve the performance of tropical cyclone track and intensity forecasting by dynamically adjusting the WRFDA background error covariance weight.

[0005] To achieve the above objectives, the present invention provides the following solution: On one hand, it provides a hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning, the specific steps of which include the following: The assimilation-forecasting loop process is formalized as a Markov decision process, and a hurricane assimilation-forecasting simulation environment based on WRFDA-WRF is constructed to provide a physically consistent interactive environment for reinforcement learning. By integrating 3D convolution, spatial attention module and gated recurrent unit to construct deep spatiotemporal encoder to extract spatiotemporal features of assimilation analysis field, a high-dimensional state embedding with both physical constraints and statistical representation capabilities is obtained. The high-dimensional state is embedded into the enhanced near-end policy optimization network built on the Actor-Critic framework. The weights of the mixed background error covariance matrix are adaptively adjusted in the cyclic assimilation-forecasting process to achieve mixed weight regulation, which is used for tropical cyclone data assimilation and forecasting.

[0006] Preferably, the hurricane assimilation-forecasting simulation environment based on WRFDA-WRF integrates WRF and WRFDA to form a closed-loop analysis-forecasting-reanalysis system, and provides the enhanced near-end strategy optimization network with real meteorological report signals as environmental feedback.

[0007] Preferably, in the deep spatiotemporal encoder, the assimilation analysis field state tensor is first processed by the three-dimensional convolution to output multi-scale spatiotemporal features; then, the spatial attention module is introduced to capture the multi-scale spatial structure of the multi-scale spatiotemporal features to obtain enhanced features; finally, the enhanced features are input into the gated recurrent unit to capture the temporal dependencies across the assimilation-prediction cycle, thereby obtaining the high-dimensional state embedding.

[0008] Preferably, adaptive exploration noise, reward shaping, and priority experience replay mechanisms are integrated into the enhanced near-end policy optimization network to achieve stable adaptive control of the data assimilation-forecasting process.

[0009] Preferably, the modeling elements of the hurricane assimilation-forecasting simulation environment based on WRFDA-WRF include state, action, reward, transfer function, and discount factor; the state is composed of the analysis field output by WRFDA; the action is defined as a three-dimensional real-valued vector output in each assimilation-forecasting cycle; the reward is used to quantify the comprehensive contribution of the action performed by the agent in the current assimilation-forecasting cycle to the quality of the analysis field and the forecasting performance; the transfer function is used to define the dynamic evolution process of the assimilation-forecasting environment; and the discount factor is used to reflect the long-term continuous characteristics of the assimilation-forecasting cycle.

[0010] Preferably, the dynamic evolution process of the transfer function definition is as follows: at the current assimilation-prediction cycle t, based on the current state s t Receive action a from the intelligent agent t The actions are mapped to weight vectors through softmax normalization. Subsequently, a mixed background error covariance matrix is ​​formed based on the weighted combination.

[0011] Preferably, the three-dimensional convolution adopts a three-layer stacked convolution structure, with each layer employing batch normalization and ReLU activation function; the spatial attention module adds nonlinear representation through two consecutive 1×1×1 convolutions and dual activation functions to generate a spatial weight map, and performs element-wise weighting on the spatial weight map to obtain the enhanced features.

[0012] On the other hand, a hurricane cyclic assimilation prediction system based on spatiotemporal attention reinforcement learning is provided, including a simulation environment construction module, a deep spatiotemporal feature extraction module, and a policy optimization module; wherein, The simulation environment construction module is used to formalize the assimilation-prediction loop process into a Markov decision process, construct a hurricane assimilation-prediction simulation environment based on WRFDA-WRF, and provide a physically consistent interactive environment for reinforcement learning. The deep spatiotemporal feature extraction module is used to fuse three-dimensional convolution, spatial attention module and gated recurrent unit to construct deep spatiotemporal encoder to extract spatiotemporal features of assimilation analysis field and obtain high-dimensional state embedding with both physical constraints and statistical representation capabilities. The strategy optimization module is used to embed the high-dimensional state into the enhanced near-end strategy optimization network built on the Actor-Critic framework, and adaptively adjust the weights of the mixed background error covariance matrix in the cyclic assimilation-forecasting process to achieve mixed weight control for tropical cyclone data assimilation and forecasting.

[0013] According to the present invention, a hurricane cyclic assimilation prediction method and system based on spatiotemporal attention reinforcement learning is provided, and the present invention discloses the following technical effects: (1) A spatiotemporal attention reinforcement learning hybrid framework for tropical cyclone data assimilation-forecasting loop is proposed. Under the premise of keeping the WRF / WRFDA dynamic-thermal framework unchanged, the time-dependent adaptive optimization of the weights of the hybrid B matrix is ​​explicitly modeled as a Markov decision process: a deep spatiotemporal encoder composed of three-dimensional convolution, spatial attention and GRU extracts state features, and an enhanced proximal policy optimization network adaptively adjusts the weights of the multi-source background error covariance in the cyclic assimilation-forecasting, thereby achieving physically consistent, flow-dependent and interpretable hybrid weight regulation.

[0014] (2) This invention takes deep reinforcement learning as its core and aims to improve the performance of tropical cyclone path and intensity forecast by dynamically adjusting the mixed weights of multi-source covariance statistics in the WRFDA background error parameter file. The analysis field generated by the method of this invention is more consistent and stable in terms of structure, dynamics and physical properties of the model, and significantly improves the hurricane path forecasting skills on this basis, while keeping the hurricane center intensity forecast from degrading. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is the overall technical roadmap of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To achieve intelligent adaptive optimization of background error covariance in hurricane data assimilation, this invention proposes STAR-EnVar (Spatio-Temporal Attention-based Reinforcement Learning Hybrid EnVar), a spatio-temporal attention-based reinforcement learning hybrid framework for the tropical cyclone data assimilation-forecasting loop, such as... Figure 1 As shown. This framework formalizes the assimilation-prediction loop process as a Markov decision process (MDP), using a reinforcement learning agent to replace human experience-based decision-making, thereby achieving dynamic control over background error information. The purpose of this invention is to provide a hurricane loop assimilation prediction method based on spatiotemporal attention reinforcement learning, such as... Figure 2 As shown, the specific steps include the following: Step 1: Formalize the assimilation-prediction loop process into a Markov decision process, and construct a hurricane assimilation-prediction simulation environment based on WRFDA-WRF to provide a physically consistent interactive environment for reinforcement learning; Step 2: Integrate 3D convolution, spatial attention module and gated recurrent unit to construct deep spatiotemporal encoder to extract spatiotemporal features of assimilation analysis field and obtain high-dimensional state embedding with both physical constraints and statistical representation capabilities; Step 3: Embed the high-dimensional state into the enhanced near-end policy optimization network built on the Actor-Critic framework, and adaptively adjust the weights of the mixed background error covariance matrix in the cyclic assimilation-forecasting process to achieve mixed weight control, which is used for tropical cyclone data assimilation and forecasting.

[0019] The three steps described above together constitute a closed-loop optimization system from physical constraints to policy decisions, enabling STAR-EnVar to achieve adaptive control and continuous optimization of the hurricane data assimilation and forecasting process while ensuring dynamic-physical consistency with numerical models. The technical solutions for each part will be explained in detail below.

[0020] Hurricane Assimilation-Forecasting Simulation Environment Based on WRFDA-WRF To achieve reinforcement learning policy training under realistic dynamic-physical constraints, this invention formalizes the WRFDA-WRF system as a Markov Decision Process (MDP), denoted as DAEnv. This environment integrates observation assimilation, background error covariance adjustment, and numerical integration within a unified assimilation-prediction loop, providing a physically consistent and interactive learning platform for the policy network. This supports adaptive hybrid optimization of the background error covariance. The physical and algorithmic meaning of this statement is that in the DAEnv reinforcement learning simulation environment, a complete step is not an abstract state transition, but a real "assimilation-prediction" physical loop. Specifically, the environment receives three elements: the policy optimization network output hybrid weights (w) + the observations (y^o) within the current assimilation window + the background field (x^b) obtained from numerical model integration. WRFDA performs variational assimilation under the new background error covariance structure to obtain the analysis field (x^a), which is provided to WRF for short-term prediction (model integration) to obtain the background field needed for the next time step. In other words, DAEnv does not just encapsulate "assimilation", but treats covariance adjustment → data assimilation → numerical integration as a unified, closed-loop Markov state transition operator, thereby achieving physically consistent reinforcement learning environment dynamics.

[0021] The logical role of "formalizing the assimilation-forecasting loop as a Markov Decision Process (MDP) and constructing a hurricane assimilation-forecasting simulation environment based on WRFDA-WRF" in the overall methodology is to formalize the originally continuously evolving physical assimilation-forecasting process into an environmental dynamics model that can be invoked by reinforcement learning algorithms, thereby establishing a unified and physically consistent semantic foundation for subsequent learning modules (state representation network, policy network, value network, and reward function design). Its logical connection with subsequent steps is reflected in the following: First, through Markov Decision Process (MDP) modeling, the assimilation-forecasting system is characterized as an interactive physical environment, clarifying the transition mechanism from (s_t, a_t) to environment (DAEnv=WRFDA+WRF) to s_(t+1). Building upon this foundation, a subsequent deep spatiotemporal encoder extracts spatiotemporal features from the state sequence of the analysis field, constructing a compact and physically consistent state representation. The policy optimization network uses this feature representation as input to learn the optimal covariance adjustment policy, and guided by the reward function (characterized by forecasting techniques such as path, intensity, and precipitation), it continuously optimizes parameters through value estimation and policy updates. Ultimately, the entire reinforcement learning process is no longer about solving an abstract control problem, but rather a closed-loop learning process directly embedded in the real "assimilation-forecast error evolution dynamics," ensuring that the policy optimization objective and the improvement of numerical weather prediction performance are physically aligned.

[0022] Based on this, the state, action, reward, transition function, and discount factor of DAEnv are defined as follows: 1. Status The state S received by the agent t The analysis field is composed of data output from WRFDA. Twenty-two representative variables were selected from over two hundred model elements, covering key physical quantities such as dynamic fields, thermal structures, moist convection, and boundary layers. Among these, the height field variable supports vertical structure and wind shear diagnosis, while the temperature and pressure fields jointly characterize the thermal distribution, thus meeting the physical information requirements for track and intensity forecasts. To enhance time-series information, the state retains the most recent t... s (for t) step The analysis field at each assimilation time step (abbreviation) is used to apply linear time weights to distinguish time series, and the "time × vertical layer (σ-level)" is jointly expanded into a depth axis, forming a tensor (C,H,W,Z)=(22,180,180,41×t). s ), where C is the number of variable channels, H and W are the horizontal grid size, and Z is the expanded vertical depth.

[0023] 2. Actions Action a t Defined as a three-dimensional real-valued vector output by the Agent in each assimilation-prediction loop t. This vector, after being generated by the Actor network, is perturbed by adaptive Gaussian noise to obtain a Noise Action, enhancing exploration capabilities and serving as input to the environment. This executed action is then mapped to a weight vector via Softmax normalization on the environment side for mixing. The three types of sample statistical sources (see transfer function for details).

[0024] 3. Rewards reward function a is used to quantify the actions performed by the agent in the current assimilation-forecasting loop. t This invention contributes comprehensively to the quality of the analysis field and forecast performance. To incorporate operational evaluation criteria for hurricane track and intensity forecasts into the reinforcement learning process, a multi-objective reward function is constructed, comprehensively considering three key meteorological indicators: minimum central pressure error, maximum wind speed error, and track error. This design ensures the physical consistency of the reward signal while providing smooth and differentiable gradient information in the continuous action space, promoting stable optimization of the policy network.

[0025] The reward function employs differentiated evaluation logics for the assimilation and forecasting phases: In the assimilation phase, an immediate evaluation is performed based on the deviation between the current analysis field and historical observations to reflect the degree to which the analysis results conform to the observational constraints; in the forecasting phase, a time-leading weighted decay is introduced to reflect the increasing uncertainty as the forecast lead time increases. All sub-reward components are scaled by a fixed factor. Amplification is performed to enhance signal amplitude and improve training sensitivity; the agent in state S t Next, execute action a t Subsequently, the environment calculates the following reward based on the model output and its corresponding physical observations (hereinafter distinguished by the subscript "obs"): ; in, These represent the three components of the reward: the lowest central pressure, the maximum wind speed, and the path.

[0026] 1) Lowest central air pressure bonus ; The minimum central pressure error is defined as To reflect the graded characteristics of the intensity estimation error, the reward adopts a piecewise linear decay combined with negative rewards and penalties: ; A positive reward is given when the pressure error is small; when the deviation exceeds 10 hPa, it enters the penalty zone to suppress the deviation in the estimation of hurricane intensity.

[0027] 2) Maximum wind speed bonus ; The maximum wind speed error considers both the consistency of wind speed classification and the accuracy of numerical deviation. Let the wind speed difference be... Numerical deviation The intensity level is determined by the grading function L(V) based on the international Saffir-Simpson hurricane classification standard (divided by maximum sustained wind speed, in knots). Considering both grading consistency (60%) and numerical accuracy (40%), the maximum wind speed bonus function is defined as follows: ; in: ; When the level is correct, rewards are refined according to the numerical deviation; when the level is incorrect, stronger penalties are imposed to ensure the physical rationality of the intensity forecast and the consistency of the classification.

[0028] 3) Path rewards ; The path components are calculated based on the spherical distance error between the predicted and observed locations of the hurricane center, using the Haversine formula to convert latitude and longitude differences into surface distance d. t(Unit: km) to quantify the spatial bias of path forecasts and generate reward signals.

[0029] The assimilation phase within 50 km is considered a high-quality path match, i.e.: ; In the forecast phase, a tolerance threshold that increases with forecast lead time is considered. ; ; This design strengthens path matching in short-term forecasts and appropriately relaxes the penalty range in long-term forecasts to balance stability and physical rationality.

[0030] 4. Transfer function Transfer function The dynamic evolution process of the assimilation-prediction environment is defined. At the current assimilation-prediction cycle t, the environment is based on its current state s. t Receive action a from the intelligent agent t The actions are mapped to weight vectors through softmax normalization. This ensures that the mixed weights are non-negative and sum to 1. Subsequently, a mixed background error covariance matrix is ​​formed based on the weighted combination: ; The background error covariance is estimated from three different statistical samples: annual scale and climatological statistics for the typhoon season (August-October). The background error parameter file (be.dat) is kept constant for offline pre-computed calculations; while the streaming dependencies... The rolling 24-hour window is dynamically updated only when the specified window is completed (every 4 assimilations), and remains unchanged within adjacent update intervals (t* represents the time index of the most recent update).

[0031] although The value remains unchanged between the two update times, but the policy weight w t The ongoing changes will still cause Adaptive adjustments are generated in each loop to ensure the time-varying and flow-dependent characteristics of the background error covariance. This is done to mitigate systematic time-phase drift in short-term forecasts. A phase tolerance window of [21,27]h is introduced during construction, and forecast samples of +21, +24, and +27h are jointly used for covariance estimation to improve statistical robustness and mitigate sampling bias caused by forecast phase mismatch. This mechanism is equivalent to introducing spatiotemporal drift tolerance in flow-dependent covariance estimation, thereby improving the physical rationality and robustness of error sample time matching.

[0032] The environment is updating and mixing Then, WRFDA assimilation analysis and WRF model integration are performed to generate the analysis field for the next time step, i.e., the next environmental state s. t+1 And calculate the corresponding instant reward r t .

[0033] 5. Discount Factor In this embodiment, the discount factor γ is set to 0.99 to reflect the long-term continuous characteristics of the assimilation-forecast cycle. Since the improvement of the analysis field and the enhancement of forecast performance in several future assimilation cycles have a continuous impact on the overall policy optimization, a higher discount factor can ensure the effective transfer of long-term rewards.

[0034] II. Depth spatiotemporal encoder to extract spatiotemporal features of the assimilation analysis field.

[0035] To extract spatiotemporal features of the assimilation analysis field with physical consistency and high-dimensional representation capabilities, a deep spatiotemporal encoder integrating 3D convolution, spatial attention, and gated recurrent units was designed. In the deep spatiotemporal encoder, the state tensor of the assimilation analysis field is first processed by 3D convolution to output multi-scale spatiotemporal features; then, a spatial attention module is introduced to capture the multi-scale spatial structure of the multi-scale spatiotemporal features, resulting in enhanced features; finally, the enhanced features are input into the gated recurrent unit to capture the temporal dependencies across the assimilation-prediction cycle, thereby obtaining the high-dimensional state embedding.

[0036] 1. 3D Convolutional Encoder First, the direct output of WRFDA in DAEnv is the assimilation analysis field x^a, which first enters the state selection module ( Figure 1 In step ①), physical priors and dimensionality reduction rules are introduced at this stage, and the processed 180×180×(41*t_s) state tensor is used as the input of the subsequent three-dimensional convolutional encoder.

[0037] 3D convolutional encoders are used to process the assimilation analysis field state tensor. The model employs a three-layer stacked convolutional structure (kernel size 3×3×3, stride 2, padding 1), with channel numbers of 32, 64, and 128 respectively. Each layer uses batch normalization and the ReLU activation function to enhance feature stability and non-linear expressive power. The computation process is as follows: ; in, This represents a 3D convolution operation. These are the convolution kernel and bias parameters of the l-th layer, respectively. This indicates a batch normalization operation. It is a non-linear activation function.

[0038] This design expands the receptive field through layer-by-layer downsampling while maintaining structural continuity in the vertical direction, enabling the model to capture key atmospheric dynamic structures and their vertical thermodynamic coupling features at different spatial scales. The structure exhibits good scale transferability, adaptively extracting multi-scale spatiotemporal features from the main circulation of tropical cyclones to organized convection in the core, achieving a unified representation from local disturbances to large-scale circulation models.

[0039] 2. Spatial Attention Module To enhance the model's response to regions with significant dynamic and thermal disturbances, this invention improves the output of the three-dimensional convolutional encoder. Above this, a lightweight spatial attention mechanism is introduced. This module uses two consecutive layers... Convolution and dual activation functions (ReLU and Sigmoid) add non-linear representations, generating pixel-level spatial weight maps. : ; The first layer of convolution and ReLU activation is used to introduce nonlinear responses and feature sparsity, thereby highlighting significant meteorological disturbance regions; the second layer of convolution and Sigmoid activation maps the output to the [0,1] interval to limit the weight magnitude and achieve interpretable spatial weighting. Input features They are of the same size in the spatial dimension, which facilitates element-by-element comparison and fusion. The final attention features are obtained through element-by-element weighting: ; in, This represents element-wise product. While maintaining global feature consistency, this design adaptively enhances the spatial representation of regions dominated by dynamic and thermal disturbances, thereby improving the model's ability to identify the structure of key physical elements such as flow, temperature, and humidity fields.

[0040] 3. Gated Loop Unit Features after spatial attention enhancement Further input to the temporal feature fusion module captures temporal dependencies across the assimilation-prediction cycle. The module first uses 3D adaptive average pooling to compress the spatial dimensions, and then transforms the tensor into a compact feature vector using the Flatten operation: ; in, The time step features, after nonlinear projection and regularization, are used as input to the gated recurrent unit (GRU) to model the temporal dynamic dependencies across the assimilation-forecast cycle. ; in, The implicit state embedding at time t comprehensively represents the dynamic evolution trend and multi-scale background information. Through an adaptive temporal memory mechanism, this module captures the implicit temporal smoothness and flow dependency features between the assimilation-prediction cycle under physical consistency constraints, providing a high-dimensional state representation with dynamic interpretability for the policy network.

[0041] III. An Enhanced Proximal Policy Optimization Network Based on the Actor-Critic Framework Based on the Actor-Critic framework, an Enhanced Proximal Policy Optimization (EPPO) algorithm is used to achieve intelligent weighting of mixed background error covariance. To improve policy convergence efficiency and training stability, the EPPO framework further introduces adaptive exploration noise, reward shaping, and priority experience replay (PER) mechanisms to achieve stable adaptive control of the data assimilation-prediction process.

[0042] 1. Actor-Critic Strategy Structure To achieve stable and efficient policy optimization in a continuous action space, reinforcement learning agents employ a typical Actor-Critic architecture. This structure consists of a policy network (Actor) and a value network (Critic), both of which share the high-dimensional state embeddings output by a deep spatiotemporal encoder. And through collaborative optimization, efficient strategy improvement can be achieved.

[0043] (Policy Network) receives high-dimensional state embeddings generated by a deep spatiotemporal encoder. Output three-dimensional real vector action To enhance exploration diversity and avoid early policy convergence traps, an adaptive Gaussian noise perturbation is introduced. The execution action (noise action) is defined as: ; in, It represents the standard deviation of adaptive exploration noise, and its value decays exponentially with the number of training steps, thereby gradually reducing the amplitude of motion perturbation and achieving a smooth transition from the highly random exploration phase to the stable exploitation phase.

[0044] (Value network) based on the same input state Estimate state value It is used to calculate the advantage function and the policy gradient direction, thereby constraining the magnitude of policy improvement during the update phase.

[0045] Through collaborative optimization with policy-value separation, the Actor-Critic architecture effectively reduces gradient variance, improves training stability and convergence speed, and provides a stable estimation basis for the subsequent Enhanced Proximal Policy Optimization (EPPO) algorithm.

[0046] 2. Reward-based shaping mechanism To mitigate the challenges of stable convergence in hurricane assimilation-forecasting loop tasks, where raw reward signals often exhibit delayed feedback, scale inconsistencies, and non-stationarity, this invention introduces a physics-guided reward shaping mechanism on top of the immediate environmental reward *r*. This mechanism stabilizes the training process and maintains dynamic-physical consistency. The total reward after shaping is defined as follows: ; in The weights are used to control the importance of the progress enhancement term and the consistency term, respectively. This design not only improves the learnability of the reward signal numerically, but also physically constrains the rational evolution of the background error weights. This serves as a progress reward, measuring the relative improvement of the current assimilation-forecast cycle. By comparing the current immediate reward with the historical average and best results from the most recent few iterations, a local "relative improvement" signal is constructed to encourage continuous optimization rather than short-term fluctuations. This is equivalent to introducing a positive incentive in the form of temporal improvement, thereby providing smooth gradient feedback in non-stationary dynamical systems.

[0047] The consistency reward reflects the physical constraints of the data assimilation process, suppressing abrupt changes in action sequence and ensuring the statistical smoothness and physical feasibility of the mixed background error covariance matrix, so that the reinforcement learning update process does not violate the rules. The dynamic balance of the dynamic balance. It contains three regularization components: (1) smoothness constraint, which punishes excessive jumps between consecutive actions; (2) feasibility constraint, which encourages action weights to be close to 1 (in line with the physical normalization after softmax); (3) diversity constraint, which avoids excessive concentration of weights on a single path. This is an adaptive bonus term that dynamically adjusts the reward magnitude based on recent reward variance and monotonic improvement trends. A slight penalty is applied when the policy is unstable (large variance), while a positive bonus is provided when the reward shows monotonic improvement. This mechanism is equivalent to a variance-normalized reward rescaling based on variance estimation, which can prevent numerical oscillations in the early stages of training and reinforce continuous improvement in later stages.

[0048] Overall, the reward shaping mechanism theoretically maintains optimal equivalence to the original task reward, while significantly improving stability, physical consistency, and interpretability in the hurricane assimilation-forecasting task in practice. Through the dual constraints of progress and consistency, the agent achieves stable learning in the noisy, delayed feedback WRFDA-WRF system, maintaining smooth policy updates across different cyclone phases (development, enhancement, and decay). This module provides a robust and physically interpretable reward signal foundation for subsequent proximal policy optimization (PPO).

[0049] 3. Priority Experience Replay In the data assimilation-forecasting loop task, the agent's reward signals exhibit significant temporal correlation and non-stationarity. Especially during different evolution stages of a hurricane (e.g., generation, enhancement, landfall, and decay), the statistical characteristics of observation and model errors change drastically over time, making the importance of experience samples at each time step for policy updates significantly different. If traditional uniform replay is used, high-value samples in key stages will be diluted, easily leading to inefficient policy updates or even forgetting. Therefore, this invention introduces a Priority Experience Replay (PER) mechanism in the EPPO training loop to achieve adaptive focusing on different key assimilation-forecasting stages. The experience cache stores interaction samples in 10-step cycles: ; in, Embed the state output of the GRU. For actions involving noise, V represents the reward value estimate, used to reward the model's performance after shaping. To measure the importance of samples to policy improvement, sampling priority is defined based on temporal difference error (TD error), and high-priority samples are resampled with higher probability in subsequent EPPO update stages, thereby improving learning efficiency and stability. The priority experience replay mechanism effectively enhances the model's learning ability in non-stationary critical stages of the hurricane assimilation-forecasting task without altering the core structure of the PPO.

[0050] 4. Enhanced Proximity Strategy Optimization Loop Through the synergistic effect of the aforementioned Actor-Critic policy structure, reward shaping mechanism, and Priority Experience Replay (PER), the STAR-EnVar framework employs Enhanced Proximal Policy Optimization (EPPO) during the policy update phase to achieve stable updates of the mixed background error policy. The core of EPPO lies in constraining the policy update step size by pruning the policy ratio, preventing excessively large gradient updates from causing performance oscillations or degradation, thereby maintaining a balance between exploration and exploitation.

[0051] In this embodiment, the performance of the STAR-EnVar data assimilation scheme was evaluated through single-point idealization experiments and cyclic assimilation-prediction experiments. First, a single-point observation perturbation experiment was conducted to examine the incremental structure and physical rationality of the B matrix, including the horizontal increment distribution, vertical response structure, and dynamic equilibrium characteristics. Subsequently, cyclic assimilation and prediction experiments based on actual observations were performed, and the overall improvement of the STAR-EnVar scheme on the analyzed and predicted fields, as well as its stability across cases, were statistically analyzed. The experimental results clearly demonstrate the advantages of the STAR-EnVar method in terms of physical consistency, observational fitting ability, and cross-case robustness.

[0052] The STAR-EnVar of this invention, a hurricane cyclic assimilation and forecasting system based on spatiotemporal attention reinforcement learning, consists of three synergistic modules: a simulation environment construction module, a deep spatiotemporal feature extraction module, and a policy optimization module. First, the WRFDA-WRF-based hurricane assimilation-forecasting simulation environment (DAEnv) provides a physically consistent interactive environment for reinforcement learning. Second, the deep spatiotemporal feature extraction module extracts spatiotemporal features of the assimilation analysis field through 3D convolution, spatial attention, and gated cyclic structures, forming a high-dimensional state embedding that can be used for policy optimization. Third, the policy optimization module is based on the Enhanced Proximal Policy Optimization (EPPO) algorithm and integrates adaptive exploration noise, reward shaping, and priority experience replay (PER) mechanisms to achieve intelligent weighting and stable updating of the mixed background error covariance. Together, these three modules constitute a closed-loop optimization system from physical constraints to policy decisions, enabling STAR-EnVar to achieve adaptive control and continuous optimization of the hurricane data assimilation-forecasting process while maintaining dynamic-physical consistency with numerical models.

[0053] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning, characterized in that, The specific steps include the following: The assimilation-forecasting loop process is formalized as a Markov decision process, and a hurricane assimilation-forecasting simulation environment based on WRFDA-WRF is constructed to provide a physically consistent interactive environment for reinforcement learning. By integrating 3D convolution, spatial attention module and gated recurrent unit to construct deep spatiotemporal encoder to extract spatiotemporal features of assimilation analysis field, a high-dimensional state embedding with both physical constraints and statistical representation capabilities is obtained. The high-dimensional state is embedded into the enhanced near-end policy optimization network built on the Actor-Critic framework. The weights of the mixed background error covariance matrix are adaptively adjusted in the cyclic assimilation-forecasting process to achieve mixed weight regulation, which is used for tropical cyclone data assimilation and forecasting.

2. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 1, characterized in that, The hurricane assimilation-forecasting simulation environment based on WRFDA-WRF integrates WRF and WRFDA to form a closed-loop analysis-forecasting-reanalysis system, and provides the enhanced near-end strategy optimization network with real meteorological report signals as environmental feedback.

3. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 1, characterized in that, In the deep spatiotemporal encoder, the assimilation analysis field state tensor is first processed by the three-dimensional convolution to output multi-scale spatiotemporal features; then the spatial attention module is introduced to capture the multi-scale spatial structure of the multi-scale spatiotemporal features to obtain enhanced features; finally, the enhanced features are input into the gated recurrent unit to capture the temporal dependencies across the assimilation-prediction cycle, thereby obtaining the high-dimensional state embedding.

4. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 1, characterized in that, The enhanced near-end policy optimization network integrates adaptive exploration noise, reward shaping, and priority experience replay mechanisms to achieve stable adaptive control of the data assimilation-forecasting process.

5. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 1, characterized in that, The modeling elements of the hurricane assimilation-forecasting simulation environment based on WRFDA-WRF include state, action, reward, transfer function, and discount factor. The state is composed of the analysis field output by WRFDA. The action is defined as a three-dimensional real-valued vector output in each assimilation-forecasting cycle. The reward is used to quantify the comprehensive contribution of the action performed by the agent in the current assimilation-forecasting cycle to the quality of the analysis field and the forecasting performance. The transfer function is used to define the dynamic evolution process of the assimilation-forecasting environment. The discount factor is used to reflect the long-term continuous characteristics of the assimilation-forecasting cycle.

6. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 5, characterized in that, The dynamic evolution process defined by the transfer function is as follows: at the current assimilation-prediction cycle t, based on the current state s... t Receive action a from the intelligent agent t The actions are mapped to weight vectors through softmax normalization. Subsequently, a mixed background error covariance matrix is ​​formed based on the weighted combination.

7. The hurricane cyclic assimilation prediction method based on spatiotemporal attention reinforcement learning according to claim 3, characterized in that, The three-dimensional convolution adopts a three-layer stacked convolution structure, with each layer using batch normalization and ReLU activation function; the spatial attention module adds non-linear representation through two consecutive 1×1×1 convolutions and double activation function to generate a spatial weight map, and performs element-wise weighting on the spatial weight map to obtain the enhanced features.

8. A hurricane cyclic assimilation prediction system based on spatiotemporal attention reinforcement learning, characterized in that, It includes a simulation environment construction module, a deep spatiotemporal feature extraction module, and a policy optimization module; among which, The simulation environment construction module is used to formalize the assimilation-prediction loop process into a Markov decision process, construct a hurricane assimilation-prediction simulation environment based on WRFDA-WRF, and provide a physically consistent interactive environment for reinforcement learning. The deep spatiotemporal feature extraction module is used to fuse three-dimensional convolution, spatial attention module and gated recurrent unit to construct deep spatiotemporal encoder to extract spatiotemporal features of assimilation analysis field and obtain high-dimensional state embedding with both physical constraints and statistical representation capabilities. The strategy optimization module is used to embed the high-dimensional state into the enhanced near-end strategy optimization network built on the Actor-Critic framework, and adaptively adjust the weights of the mixed background error covariance matrix in the cyclic assimilation-forecasting process to achieve mixed weight control for tropical cyclone data assimilation and forecasting.

Images