A Spatiotemporal Completion Method for Heterogeneous Ocean Data Based on Hybrid Time-Frequency Experts and Dynamic Graphs

Abstract

This invention relates to the field of marine data processing technology, and in particular provides a spatiotemporal incomplete method for heterogeneous marine data based on hybrid time-frequency experts and dynamic graphs. The method includes, during data flow, aligning the original heterogeneous observation sequence with a union set filled USP, and then concatenating it with a fine-grained point-level observation validity mask before mapping it to a high-dimensional feature space; the feature streams enter two feature extraction hubs in parallel, achieving fine decoupling in the time dimension through a hybrid expert time-frequency network; and in the spatial dimension, performing feature reconstruction through an adaptive implicit dynamic graph network; the spatiotemporal deep features are merged into an uncertainty-gated fusion UGF module for adaptive feature concatenation and decoding; and a high-fidelity complete marine data sequence is output in conjunction with a hard residual connection mechanism. This method achieves strong robustness and high fidelity in marine data incomplete.

Description

Technical Field

[0001] This invention relates to the field of marine data processing technology, and in particular to a spatiotemporal filling method for heterogeneous marine data based on hybrid time-frequency experts and dynamic graphs. Background Technology

[0002] With the rapid development of global climate change research and modern ocean observation technology, high-frequency, continuous, and precise monitoring of the marine environment (such as temperature, salinity, dissolved oxygen, and ocean currents) has become the cornerstone of Earth system science and the development of the blue economy.

[0003] Currently, large-scale integrated monitoring networks based on fixed buoys, coastal base stations, and satellite remote sensing have generated massive amounts of multidimensional time-series observation data. However, due to extreme marine environments (such as biofouling, seawater corrosion, and extreme storms) and physical limitations of communication links, the actual in-situ observation data generally suffers from severe and complex multi-mode loss problems, including the loss of instantaneous random points, long-term continuous blocks, and the loss of entire channels due to physical damage to specific sensors. At the same time, the vast marine observation network faces a severe challenge of multi-source heterogeneity: the sensor combinations carried by observation platforms in different sea areas or with different mission types vary, resulting in significant differences in the dimensionality of observation variables, data structures, and potential physical dynamic modes at each station.

[0004] Existing time series interpolation and prediction models exhibit significant limitations when dealing with the complex heterogeneous ocean data mentioned above. On the one hand, traditional deep learning models typically rely on a fixed-dimensional input space. When faced with heterogeneous data from sensors with varying configurations, they often resort to rigid truncation based on prior knowledge, predefined mappings, or isolated modeling for single devices. This rigid approach not only severs the universal characteristic connections of the ocean system globally but also, when faced with channel-level missing data, often fails completely due to a lack of in-depth exploration of implicit physical coupling relationships between multiple variables (such as nonlinear constraints between temperature, salinity, and depth), making cross-variable knowledge transfer and inference impossible. On the other hand, in terms of time series evolution modeling, ocean data simultaneously contains ultra-long-range non-stationary evolution trends and strict macroscopic physical periods (such as tides). Existing recurrent neural networks (such as RNN and LSTM) suffer from severe memory decay and computational efficiency bottlenecks when processing ultra-long, high-frequency sequences. Advanced self-attention models (such as Transformer) face the problem of computational complexity exponentially increasing with sequence length. Moreover, most traditional models only perform simple mean squared error optimization in the pure time domain, which easily leads to mean regression. This results in a degenerate solution that only satisfies the mean squared error constraint, generating overly smooth prediction results. This forcibly erases the inherent physical periodicity and high-frequency fluctuation details of the ocean, generating artifact data with non-physical characteristics. Summary of the Invention

[0005] In view of this, the present invention provides a spatiotemporal incomplete method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs, in order to achieve strong robustness and high fidelity in ocean data incomplete.

[0006] In a first aspect, the present invention provides a spatiotemporal incomplete method for heterogeneous ocean data based on hybrid time-frequency experts and dynamic graphs, the method comprising:

[0007] Step 1: In the data flow, the original heterogeneous observation sequence is aligned by union set filling USP and then concatenated with fine-grained point-level observation validity mask and mapped to a high-dimensional feature space. Step 2: Based on Step 1, the feature streams enter two feature extraction hubs in parallel. In the time dimension, fine decoupling is achieved through a hybrid expert time-frequency network; in the spatial dimension, feature reconstruction is performed through an adaptive implicit dynamic graph network. Step 3: Based on Step 2, the temporal and spatial depth features are incorporated into the uncertainty-gated fusion UGF module for adaptive feature stitching and decoding; and a high-fidelity complete ocean data sequence is output in conjunction with the hard residual connection mechanism.

[0008] Optionally, step 1 includes: The design incorporates a union set filling USP alignment and mask-aware feature embedding mechanism, which transforms unstructured original heterogeneous observation sequences into a unified high-dimensional feature representation through three logical steps. Step 11: Global variable mapping and heterogeneous data alignment; First, extract all types of observed variables from the monitoring network and construct a system containing... Global collection of class variables Subsequently, the inputs of any local station are mapped to the global dimension space to construct a unified global data matrix. , This indicates the length of the time series, i.e., the number of time steps. Represents the set of real numbers; if the station is not equipped with a specific sensor, the current variable column is zero-padded to force all heterogeneous station data to be aligned into a structured tensor with consistent dimensions, that is, to obtain the aligned observation tensor; Step 12: Construction of fine-grained point-level observation validity mask; Constructing a fine-grained point-level observation validity mask matrix If the site is in time Actual observed variables The effective value of , then If the site does not have a specific sensor installed, or if the specific sensor has missing native data at the current moment, then ; Step 13: Mask-aware early feature fusion embedding; The aligned observation tensor and the validity mask are explicitly concatenated along the channel dimension to obtain a multi-channel composite input tensor. The multi-channel composite input tensor is then input into the embedding module of the two-dimensional convolution Conv2d. The convolution kernel performs local temporal smoothing by sliding across time steps but not across variables, fusing numerical and mask channels and mapping them to a high-dimensional feature space. The high-dimensional mask-aware features are then distributed to the downstream network.

[0009] Optionally, step 2 includes: In the time dimension, the hybrid expert time-frequency network dynamically assigns sequences to state-space experts, frequency-domain filtering experts, and local smoothing experts through intelligent temperature routing, so as to finely decouple long-range evolution trends from high-frequency physical cycles; in the spatial dimension, the adaptive implicit dynamic graph network infers the asymmetric implicit physical topology between heterogeneous variables through data-driven node embedding and performs cross-variable information aggregation. A parallel temporal hybrid expert MoE time-frequency network is introduced, which consists of three heterogeneous expert branches and an adaptive routing network: I. Gated bidirectional state-space expert; For high-dimensional time-series features of input First, two independent state-space models, Mamba, are instantiated. One is scanned along the forward time axis to extract forward features. Another approach involves flipping the input feature sequence along the time axis, then flipping the output back to its original state to extract backward features. ; After concatenating the bidirectional features along the channels, a linear mapping is applied. With Sigmoid activation function Generate channel-level gating coefficients Its expression is: ; Finally, time features are generated by weighted concatenation with hard residuals. Its expression is: ; in, This represents the Hadamard product, which is the element-wise multiplication of tensors. II. Frequency domain filtering expert; First, the feature sequence in the time dimension is mapped to the complex frequency domain using the real-valued Fast Fourier Transform (FFT). In the frequency domain, a learnable complex weight tensor is maintained. The complex weight tensor is used to automatically learn and retain key frequency components while suppressing random noise; simultaneously, it adjusts the amplitude and phase of the signal in the frequency domain; after filtering, it is restored to time-domain features via inverse fast Fourier transform (IFFT). ; III. Multi-scale local smoothing expert; A third parallel expert, the local smoothing expert, is introduced. It consists of parallel local convolutional layers with different multi-scale receptive fields, and multi-scale local temporal features are obtained through nonlinear activation mapping. Local smoothing experts are used to perform fast short-time interpolation using adjacent valid points within a very small time window when the Mamba field of view is too large. IV. Dynamic temperature routing and expert integration; The expert opinions adopted for the current data point are dynamically determined by token-level routing weights. Input features are fed into a routing network composed of a multilayer perceptron (MLP). A preset temperature scaling factor is introduced to sharpen the output affinity score, which is then converted into the assigned weights for each expert using a normalization function. Finally, the time branch output is obtained by weighted summation of the features from each expert and normalized layer by layer. .

[0010] Optionally, step 2 further includes: Spatial feature extraction is performed through an adaptive implicit dynamic graph network that does not require any prior graph structure. It is data-driven and uses asymmetric implicit physical topology mining and deep cross-variable information interaction to reconstruct features in extreme missing conditions. V. Dynamic graph construction based on node embedding; Treating various ocean variables as nodes in a graph, in the latent space... The class variable node dynamically initializes two independent sets of learnable parameter matrices: (1) source node embedding matrix (2) Target node embedding matrix , used to characterize the physical properties of the current variable when it acts as an information receiver, where For the embedding dimension, d << ; By calculating the similarity between the source node embedding matrix and the target node embedding matrix, and combining nonlinear activation and sparsification mechanisms, the association score matrix between nodes is adaptively generated. Since two sets of independent embedding features are used, the association score matrix has asymmetry, which fits the unidirectional or asymmetric causal physical correlation in real ocean dynamics. VI. Nonlinear Enhanced Spatial Graph Aggregation; Constructing a dynamic adjacency matrix Subsequently, graph neural networks are used to transmit information along the variable dimension; to handle the complex high-dimensional nonlinear mappings between heterogeneous ocean variables, a deep nonlinear mixing mechanism is introduced, the expression of which is: ; ; in, The input spatial feature matrix; This is the message passing feature matrix after aggregating neighbor node information. The spatial characteristics of the output; It is a deep nonlinear feature hybrid network; This is a layer normalization operation used to stabilize training; When a physical sensor experiences hardware-level failure, i.e., a complete loss of channel, the time reference system of the currently damaged node is completely broken. To address this, a dynamic adjacency matrix is ​​used to borrow health data from strongly correlated variables, and a deep feature hybrid network is employed to fit complex nonlinear mapping rules between heterogeneous variables, achieving high-dimensional semantic transformation across variables. Finally, residual connections and layer normalization are used to output spatial features. .

[0011] Optionally, step 3 includes: An uncertainty-gated fusion UGF module is incorporated to evaluate the prediction variance of the two branches in real time, and adaptive feature concatenation and decoding are performed based on the inverse variance weighting principle; uncertainty-gated fusion module based on the inverse variance weighting principle: a) Local uncertainty variance estimation: Assuming the output of each feature branch follows a Gaussian distribution, its variance represents the uncertainty of the current feature branch with respect to the current prediction; first, two lightweight variance estimation networks are constructed. and Variance maps were extracted from the spatiotemporal features respectively. and Its expression is: ; ; in, Features output by the time branch; Features output by spatial branching; A smooth activation function is used to ensure that the output variance is always positive; It is a very small constant used to prevent numerical instability when divided by zero during calculation; Two sets of variance plots are used to evaluate the prediction confidence of the two branches at the current location point-wise over time. b. Inverse variance weighted allocation: Based on the minimum variance principle in Bayesian inference, branches with smaller variance have higher confidence levels. An inverse variance weighting strategy is applied, where weights are weighted by... Proportional to the time branch, the gating weights are generated by calculating the relative uncertainty. Its expression is: ; c. Cascaded deep feature mixing and decoding: After obtaining dynamic weights, a weighted cascaded-deep mixing strategy is adopted. First, the weighted features are calculated. Its expression is: ; in, This indicates a feature concatenation operation performed along the channel dimension; Subsequently, the weighted feature tensor is concatenated along the channel dimension. The output is mapped to an initial prediction tensor with the same dimensions as the original heterogeneous observation sequence. .

[0012] Optionally, step 3 further includes: The final output is constructed using a hard residual connection mechanism, and its expression is: ; in, This is the validity mask for the original data, i.e., in the validity mask The observation points retain the true original values, only when Missing points are filled in to fill in the network predictions to ensure strict physical consistency; To comprehensively optimize and fill the model while ensuring the physical plausibility of the generated data, the overall loss function will be... The design employs a multi-task joint optimization approach, incorporating time-domain reconstruction errors and introducing frequency-domain constraints, smoothness penalties, and adversarial feedback. The expression is as follows: ; in, and The temporal reconstruction losses are for the mask missing region and the globally effective region, respectively. To balance robustness to extreme outliers and smooth gradients, a hybrid form of L1 and L2 norms is adopted. The frequency domain amplitude error (i.e., the L1 norm based on the fast Fourier transform) is used to force the model to capture the global physical period unique to ocean data and prevent non-physical phase shifts in the generated results. For total variation smoothing loss, high-frequency jitter noise is suppressed by calculating the absolute difference between adjacent time steps; The discrimination error provided to the generative adversarial network is used to improve the realism of local temporal textures; , , , To balance the hyperparameters of various errors; in the early stages of training, Set it to 0 for preheating, then give it a very small weight to refine the texture.

[0013] In a second aspect, embodiments of the present invention provide a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to execute the spatiotemporal filling method for heterogeneous marine data based on hybrid time-frequency experts and dynamic graphs, as described in the first aspect or any possible implementation thereof.

[0014] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the spatiotemporal completion method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs in the first aspect or any possible implementation of the first aspect.

[0015] The technical solution provided by this invention includes the following steps during data flow: The original heterogeneous observation sequence is aligned using a union set filling USP and then concatenated with a fine-grained point-level observation validity mask before being mapped to a high-dimensional feature space. The feature streams enter two feature extraction hubs in parallel. In the time dimension, fine decoupling is achieved through a hybrid expert time-frequency network. In the spatial dimension, feature reconstruction is performed through an adaptive implicit dynamic graph network. The temporal and spatial depth features are merged into an uncertainty-gated fusion UGF module for adaptive feature concatenation and decoding. Finally, a hard residual connection mechanism is used to output a high-fidelity complete ocean data sequence. This method achieves strong robustness and high fidelity in ocean data imputation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a method for spatiotemporal filling of heterogeneous marine data based on hybrid time-frequency experts and dynamic graphs provided in an embodiment of the present invention; Figure 2 The overall architecture of the model provided in the embodiments of the present invention; Figure 3The generator loss convergence curve of the model provided in this embodiment of the invention on moored buoy data; Figure 4 The discriminator loss convergence curve of the model provided in this embodiment of the invention on moored buoy data; Figure 5 This is an image showing the oxygen saturation interpolation effect in a scenario with 10% random point loss, provided by an embodiment of the present invention. Figure 6 This is an image showing the oxygen saturation interpolation effect in a scenario with 30% random point loss, provided by an embodiment of the present invention. Figure 7 This is an image showing the oxygen saturation interpolation effect in a scenario with 50% random point missing, provided by an embodiment of the present invention. Figure 8 This is a diagram illustrating the dissolved oxygen interpolation effect in a scenario with 10% contiguous block loss, provided by an embodiment of the present invention. Figure 9 This is a diagram illustrating the dissolved oxygen interpolation effect in a scenario with 30% contiguous block loss, provided by an embodiment of the present invention. Figure 10 This is a diagram illustrating the dissolved oxygen interpolation effect in a scenario with 50% contiguous block loss, provided by an embodiment of the present invention. Figure 11 This is a diagram illustrating the dissolved oxygen interpolation effect under extreme channel loss scenarios provided in this embodiment of the invention. Figure 12 This is a multivariate spatiotemporal heatmap provided by an embodiment of the present invention for a scenario with an extreme 50% missing rate and random point missing data. Figure 13 This is a multivariate spatiotemporal heatmap for a contiguous block missing scenario with an extreme 50% missing rate, provided by an embodiment of the present invention. Figure 14 This is a multivariate spatiotemporal heatmap for an extreme 50% missing channel scenario provided in an embodiment of the present invention; Figure 15 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0020] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0021] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0022] Figure 1 The flowchart of the spatiotemporal incomplete method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs provided in the embodiments of the present invention is as follows: Figure 1 As shown, the method includes: Step 1: In the data flow, the original heterogeneous observation sequence is aligned by the union set filling USP and then concatenated with the fine-grained point-level observation validity mask and mapped to the high-dimensional feature space.

[0023] This invention employs an end-to-end parallel extraction-adaptive fusion architecture, forming the technical foundation for both high reconstruction accuracy and robustness under extreme conditions. The overall model architecture is as follows: Figure 1 As shown.

[0024] In this embodiment of the invention, step 1 includes: Design a Union Set Padding (USP) alignment and mask-aware feature embedding mechanism. The mechanism transforms unstructured original heterogeneous observation sequences into a unified high-dimensional feature representation through three logical steps. Step 11: Global variable mapping and heterogeneous data alignment; First, extract all types of observed variables from the monitoring network and construct a system containing... Global collection of class variables Subsequently, the inputs of any local station are mapped to the global dimension space to construct a unified global data matrix. , This indicates the length of the time series, i.e., the number of time steps. Represents the set of real numbers; if the station is not equipped with a specific sensor, the current variable column is zero-padding to force all heterogeneous station data to be aligned into a structured tensor with consistent dimensions, that is, to obtain the aligned observation tensor; Step 12: Construction of fine-grained point-level observation validity mask; To accurately distinguish between true observations and missing / filled data, a fine-grained point-level observation validity mask matrix is ​​constructed. If the site is in time Actual observed variables The effective value of , then If the site does not have a specific sensor installed, or if the specific sensor has missing native data at the current moment, then ; Step 13: Mask-aware early feature fusion embedding; To enable the model to acquire the ability to perceive missing boundaries in the initial stage, the aligned observation tensor and the validity mask are explicitly concatenated along the channel dimension to obtain a multi-channel composite input tensor. The multi-channel composite input tensor is then input into the embedding module of the two-dimensional convolution Conv2d. The convolution kernel performs local temporal smoothing by sliding across time steps but not across variables, fusing numerical and mask channels and mapping them to a high-dimensional feature space. The high-dimensional mask-aware features are then distributed to the downstream network.

[0025] Step 2: Based on Step 1, the feature streams enter two feature extraction hubs in parallel. In the time dimension, fine decoupling is performed through a hybrid expert time-frequency network; in the spatial dimension, feature reconstruction is performed through an adaptive implicit dynamic graph network.

[0026] In this embodiment of the invention, step 2 includes: In the time dimension, the hybrid expert time-frequency network dynamically assigns sequences to state-space experts, frequency-domain filtering experts, and local smoothing experts through intelligent temperature routing, so as to finely decouple long-range evolution trends from high-frequency physical cycles; in the spatial dimension, the adaptive implicit dynamic graph network infers the asymmetric implicit physical topology between heterogeneous variables through data-driven node embedding and performs cross-variable information aggregation. To address the challenge that a single network architecture struggles to handle complex dynamic characteristics in ocean time-series data, including long-range non-stationary evolution trends, strict physical periods (such as tides), and transient local mutations, and that these characteristics are prone to mutual interference, a parallel time-series hybrid expert MoE time-frequency network is introduced. This network comprises three heterogeneous expert branches and an adaptive routing network. I. Gated Bi-Mamba Expert; Gated bidirectional state-space experts aim to capture the macroscopic evolutionary trends of extremely long sequences with linear time complexity, overcoming the memory bottleneck of traditional self-attention mechanisms. This is specifically designed for high-dimensional temporal features of the input. First, two independent state-space models, Mamba, are instantiated. One is scanned along the forward time axis to extract forward features. Another approach involves flipping the input feature sequence along the time axis, then flipping the output back to its original state to extract backward features. ; Considering that different missing positions (such as the beginning or end of a sequence) have different degrees of dependence on historical and future information, the bidirectional features are concatenated along the channels and then linearly mapped. With Sigmoid activation function The gating coefficients at the channel level are expressed as follows: ; Finally, time features are generated by weighted concatenation with hard residuals. Its expression is: ; in, This represents the Hadamard product, which is the element-wise multiplication of tensors. II. Frequency Domain Filtering Expert (Fourier Filter Expert) The frequency domain filtering expert is responsible for constraining the physical periodicity of the generated results, preventing the model from degenerating into a simple moving average. First, the time-dimensional feature sequence is mapped to the complex frequency domain using a real-valued Fast Fourier Transform (FFT). In the frequency domain, a learnable complex weight tensor is maintained. The complex weight tensor is used to automatically learn and retain key frequency components (such as the diurnal cycle) and suppress random noise; at the same time, it adjusts the amplitude (amplifying the inherent periodic signal of the ocean and suppressing high-frequency noise) and phase (waveform shift) of the signal in the frequency domain; after filtering, it is restored to the time domain features by inverse fast Fourier transform (IFFT). ; III. Multi-scale Local Smoothing Expert; A third parallel expert, the local smoothing expert, is introduced. It consists of parallel local convolutional layers with different multi-scale receptive fields, and multi-scale local temporal features are obtained through nonlinear activation mapping. Local smoothing experts are used to perform fast short-time interpolation using adjacent valid points within a very small time window when the Mamba field of view is too large. IV. Temperature-Scaled Dynamic Routing. The routing weights at the token level (per time step, per variable) dynamically determine the expert opinions adopted for the current data point. Input features are fed into a routing network composed of a multilayer perceptron (MLP). To avoid ambiguous assignments under extreme conditions, a preset temperature scaling factor is introduced to sharpen the output affinity score. Then, a normalization function converts the affinity score into assignment weights for each expert. This temperature scaling mechanism allows the network to favor Mamba experts with high confidence, for example, when there are consecutive missing blocks; and to resolutely switch to locally smoothing experts when faced with dense random missing blocks. Finally, the time branch output is obtained by weighted summation of the expert features and layer normalization. .

[0027] In this embodiment of the invention, step 2 further includes: Traditional geospatial distance-based graph neural networks often struggle to address the complex physical and biochemical couplings among multidimensional marine observation variables (such as water temperature, salinity, and dissolved oxygen) that lack explicit predefined topological structures. This paper proposes an adaptive implicit dynamic graph network for spatial feature extraction, which is data-driven and utilizes asymmetric implicit physical topology mining and deep intervariate information interaction to reconstruct features under extreme conditions of data loss (such as the failure of an entire array of sensors). V. Dynamic graph construction based on node embedding; To break the dependency on a predefined static adjacency matrix and accurately characterize the asymmetric physical coupling relationships between ocean variables, various ocean variables (such as temperature and salinity) are treated as nodes in a graph, and in the latent space... The class variable node dynamically initializes two independent sets of learnable parameter matrices: (1) source node embedding matrix (2) Target node embedding matrix , used to characterize the physical properties of the current variable when it acts as an information receiver, where For the embedding dimension, d << ; By calculating the similarity between the source node embedding matrix and the target node embedding matrix (such as inner product operation), and combining nonlinear activation and sparsification mechanisms, the association score matrix between nodes is adaptively generated. Since two sets of independent embedding features are used, the association score matrix has asymmetry, which fits the unidirectional or asymmetric causal physical correlation in real ocean dynamics (e.g., temperature changes can significantly affect dissolved oxygen, but the reverse is not true). VI. Nonlinear Enhanced Spatial Graph Aggregation; Constructing a dynamic adjacency matrix Subsequently, a graph neural network is used to transmit information along the variable dimension. Unlike the traditional graph convolutional hybrid network (GCN), which only uses a single-layer linear transformation, a deep nonlinear hybrid mechanism is introduced to handle the complex high-dimensional nonlinear mappings between heterogeneous ocean variables. Its expression is as follows: ; ; in, The input is the spatial feature matrix; This is the message passing feature matrix after aggregating neighbor node information. The spatial characteristics of the output; It is a deep nonlinear feature hybrid network; This is a layer normalization operation used to stabilize training; When a physical sensor experiences hardware-level failure, i.e., a complete loss of channel, the time reference system of the currently damaged node is completely broken. Health data is borrowed from strongly correlated variables (such as salinity and water depth) through a dynamic adjacency matrix, and a deep feature hybrid network is used to fit complex nonlinear mapping rules between heterogeneous variables to achieve high-dimensional semantic transformation across variables. Finally, residual connections and layer normalization are used to output spatial features. This achieves high-precision reasoning results.

[0028] Step 3: Based on Step 2, the temporal and spatial depth features are incorporated into the uncertainty-gated fusion UGF module for adaptive feature stitching and decoding; and a high-fidelity complete ocean data sequence is output in conjunction with the hard residual connection mechanism.

[0029] In real-world marine monitoring environments, the reliability of temporal and spatial information dynamically changes depending on the missing data pattern. For example, when faced with instantaneous, high-frequency random point loss, the temporal branch can achieve high-precision interpolation based on the close correlation between consecutive moments because the local time reference frame remains intact. Conversely, when faced with long-period continuous block loss or even channel loss due to sensor failure, the temporal reference frame of the target variable itself is completely broken. In this case, the temporal branch is in a guessing state, and the system must rely heavily on the spatial branch, utilizing the physical coupling between heterogeneous variables for cross-variable inference. To address the failure of traditional fixed-weight fusion methods in such extreme scenarios, this invention proposes an Uncertainty-Gated Fusion (UGF) module based on the inverse variance weighting principle.

[0030] In this embodiment of the invention, step 3 includes: An uncertainty-gated fusion UGF module is incorporated to evaluate the prediction variance of the two branches in real time, and adaptive feature concatenation and decoding are performed based on the inverse variance weighting principle; uncertainty-gated fusion module based on the inverse variance weighting principle: a) Local uncertainty variance estimation: Assuming the output of each feature branch follows a Gaussian distribution, its variance represents the uncertainty of the current feature branch with respect to the current prediction. First, two lightweight variance estimation networks are constructed. and Variance maps were extracted from the spatiotemporal features respectively. and Its expression is: ; ; in, Features output by the time branch; Features output by spatial branching; A smooth activation function is used to ensure that the output variance is always positive; It is a very small constant used to prevent numerical instability when divided by zero during calculation; Two sets of variance plots are used to evaluate the prediction confidence of the two branches at the current location point-wise over time. b. Inverse variance weighted allocation: Based on the minimum variance principle in Bayesian inference, branches with smaller variance (i.e., lower uncertainty) have higher confidence. An inverse variance weighting strategy is applied, where weights are weighted by... Proportional to the time branch, the gating weights are generated by calculating the relative uncertainty. Its expression is: ; This mechanism enables the model to adaptively and smoothly switch attention between temporal and spatial information based on the degree of corruption of the input data.

[0031] c. Cascaded Deep Feature Fusion and Decoding: After obtaining dynamic weights, traditional methods typically employ simple element-wise addition. However, considering that spatiotemporal feature distributions reside in different manifold spaces, direct addition can easily lead to semantic conflicts. A weighted cascaded-deep fusion strategy is adopted, first calculating the weighted features... Its expression is: ; in, This indicates a feature concatenation operation performed along the channel dimension; Subsequently, the weighted feature tensor is concatenated along the channel dimension. The output is mapped to an initial prediction tensor with the same dimensions as the original heterogeneous observation sequence. .

[0032] In this embodiment of the invention, step 3 further includes: To ensure that the filling results do not corrupt the actual observation data, a hard residual connection mechanism is used to construct the final output, the expression of which is: ; in, This is the validity mask for the original data, i.e., in the validity mask The observation points retain the true original values, only when Missing points are filled in to fill in the network predictions to ensure strict physical consistency; To comprehensively optimize and fill the model while ensuring the physical plausibility of the generated data, the overall loss function will be... The design employs a multi-task joint optimization approach, incorporating time-domain reconstruction errors and introducing frequency-domain constraints, smoothness penalties, and adversarial feedback. The expression is as follows: ; in, and The temporal reconstruction losses are for the mask missing region and the globally effective region, respectively. To balance robustness to extreme outliers and smooth gradients, a hybrid form of L1 and L2 norms is adopted. The frequency domain amplitude error (i.e., the L1 norm based on the fast Fourier transform) is used to force the model to capture the global physical period (such as tidal fluctuations) unique to ocean data, and to prevent non-physical phase shifts in the generated results. The Total Variation Loss is used to suppress high-frequency jitter noise by calculating the absolute difference between adjacent time steps. The discrimination error provided to the Generative Adversarial Network (PatchGAN) is used to improve the realism of local temporal textures; , , , To balance the hyperparameters of various errors; in the early stages of training, Set it to 0 for warm-up, then give it a very small weight to refine the texture.

[0033] Verification of the effectiveness of this invention: (1) Training and testing datasets for heterogeneous ocean data reconstruction; To ensure that the model can learn comprehensive and representative ocean dynamics patterns and to objectively evaluate the filling performance of multi-source heterogeneous sensors, this invention constructs standardized training and testing datasets: ① Global Ocean In-Situ Moored Buoy Dataset; The model training and benchmarking of this invention are based on an authoritative global ocean in-situ observation dataset—the Copernicus Global Ocean In-Situ Near-Real-Time Observations (CMEMS). This dataset integrates data from multiple ocean observation platforms worldwide, exhibiting extremely high equipment heterogeneity and marine environmental complexity.

[0034] In this experiment, a subset of mooring buoys (MOs) was primarily selected, whose extensive and dense global geographical distribution provided the model with abundant real-world samples of ocean spatiotemporal evolution. Addressing the typical equipment heterogeneity issue in ocean observation networks (for example, the Kristineberg site primarily focuses on biochemical parameters such as dissolved oxygen and turbidity, while site 51046 provides comprehensive physical and biochemical observations), this invention constructed a unified set of global variables during the data preprocessing stage, covering multiple key oceanographic physical and biochemical indicators (such as water temperature, salinity, and dissolved oxygen).

[0035] During the data cleaning and reconstruction phase, the system automatically removes invalid infinity values ​​and extreme outliers (noise with an absolute value > 10000) that exceed physical norms, and rigorously divides the dataset into training and test sets in a 7:3 ratio. The data is sliced ​​with overlapping sliding windows of a preset length, and combined with the invention's unique United Set Filling (USP) strategy, a high-quality spatiotemporal feature learning foundation with structural alignment and clear physical meaning is built for the model.

[0036] ② Multi-mode extreme missing data simulation and generalization capability verification test set; To rigorously evaluate the model's ability to repair and reconstruct damaged data in real harsh marine environments, this invention designed three typical equipment failure simulation scenarios in an independent test set (such as a subset of Mediterranean buoy observations). Throughout the evaluation process, the ground truth values ​​from these test sites were manually masked at specific points and used only for final accuracy verification. They were never used for any model parameter updates, ensuring the absolute impartiality of the evaluation results. Random Point Missing (RPM): Simulates discrete data loss caused by momentary equipment failure, high-frequency shot noise from sensors, or transient communication interruptions. Continuous Block Missing (RBM): Simulates large-area, long-period gaps in continuous observation caused by power outages, extreme sea conditions, or prolonged communication blind spots. Channel Missing (CM): Simulates an extreme heterogeneous situation where irreversible hardware damage or systematic calibration failure of a specific type of sensor (e.g., a salinity meter) leads to 100% absence of that variable throughout the entire sequence. The three missing modes were cross-combined and tested at high dynamic missing rates of 10%, 30%, and 50%, respectively. Furthermore, to further verify the generalization ability of the frequency domain filtering expert and dynamic graph topology mining mechanism in this invention to extract the generalized physical rhythm and thermodynamic coupling relationship.

[0037] (2) Experimental setup; The core parameters that need to be set for the model of this invention include: time sliding window size, basic hidden layer dimension, multi-scale convolution kernel size set, state dimension of state space expert, local convolution width, dimension expansion factor, graph node embedding dimension of spatial graph learning branch, saturation scaling factor, temperature scaling factor of dynamic routing, batch size, optimizer type, optimizer momentum parameter, initial learning rate, minimum learning rate, learning rate scheduler type, gradient pruning threshold, number of warm-up rounds for adversarial training, and total number of training steps, etc.

[0038] (3) Model training, evaluation and testing: To verify the effectiveness, accuracy and robustness of the technical solution of this invention in repairing heterogeneous ocean data, a standard experimental procedure was designed, including end-to-end multi-task pre-training and generalization ability testing for multiple missing scenarios. The entire experimental process and model inference were executed on a high-performance computing platform accelerated by NVIDIA T4 Tensor Core GPU, and a fixed global random seed was used to ensure the complete reproducibility of the experimental results.

[0039] The proposed model is trained on a high-quality spatiotemporal dataset aligned with the aforementioned joint set. The training process employs the Adam optimizer, combined with a cosine annealing learning rate scheduling strategy to guide the model to converge stably and efficiently to the optimal state. The model's training objective is not simply mean squared error, but rather a multi-task joint optimization framework aimed at minimizing the overall objective function, which includes reconstruction loss, frequency domain constraint loss, temporal smoothing penalty, and adversarial generation loss. To prevent uncontrollable noise from the generator in the early stages of training, this invention incorporates an adversarial warm-up mechanism. During the initial warm-up phase, the weights of the adversarial generation loss are minimized or temporarily deactivated, allowing the model to focus on learning the underlying spatiotemporal topology based on L1 and L2 reconstruction losses. After the warm-up, adversarial feedback is introduced with small weights to refine the texture of high-frequency data. Simultaneously, gradient pruning effectively prevents gradient explosion, ensuring the stability of the entire generative adversarial network training process. Figure 3 and Figure 4 As shown.

[0040] The following two widely used evaluation metrics in the field of time series analysis are used to quantify the model's point prediction accuracy. It is important to note that, to ensure the rigor of the evaluation, the calculation of these metrics is performed only on regions of the validation set that are intentionally hidden by a manually masked matrix and where the original data contains ground truth values: Mean Absolute Error (MAE): This indicator calculates the average of the absolute values ​​of the differences between the predicted and actual values. MAE provides a clear picture of the actual magnitude of the prediction error, and its calculation formula is as follows: ; in, These are the original observations. is the model imputation value, and m is the total number of samples in the test set that are artificially masked but have true values.

[0041] Root Mean Square Error (RMSE): This metric is calculated as the square root of the mean of the squares of the differences between predicted and actual values. RMSE is more sensitive to larger prediction errors (i.e., outliers) and reflects the predictive stability of the model. Its calculation formula is as follows: ; in, These are the original observations. is the model imputation value, and m is the total number of samples in the test set that are artificially masked but have true values.

[0042] Of the two indicators mentioned above, the lower the value, the smaller the prediction error of the model and the better its performance. Figures 5 to 14As shown, the present invention exhibits excellent predictive performance and strong generalization ability. Especially under extreme scenarios involving continuous block loss, high-frequency random point loss, and severe channel-wide sensor failure, the model maintains extremely high reconstruction fidelity. The MAE and RMSE values ​​calculated on the test set objectively and comprehensively verify the superior reconstruction accuracy and practical application value of the present invention in processing complex, heterogeneous marine data under harsh environments.

[0043] Compared with the prior art, the present invention has the following advantages: (1) It achieves fine decoupling and high-fidelity reconstruction of multi-scale ocean dynamics characteristics.

[0044] Existing deep learning models, when processing ocean time series data, typically employ a single optimization objective, tending to over-smooth the data and thus losing the high-frequency fluctuation details unique to the ocean environment, or failing to explicitly capture strict physical cycles such as tides. This invention constructs a hybrid expert time-frequency network, enabling the model to delegate the processing of complex, superimposed mixed signals to the experts most proficient in the field: using a bidirectional state-space model (Mamba) to anchor long-range evolution trends, using Fourier filtering experts to constrain global physical cycles in the frequency domain, and using multi-scale convolution experts to handle short-term transient changes. Combined with intelligent gated routing with temperature scaling, this invention achieves perfect decoupling of ocean processes, not only accurately repairing missing macroscopic trends but also faithfully reproducing waveform textures with real physical rhythms.

[0045] (2) It breaks through the bottleneck of mining implicit physical coupling between heterogeneous variables and significantly improves the reconstruction capability under extreme missing conditions.

[0046] In real-world ocean monitoring, extreme situations frequently arise where specific sensors are completely destroyed (i.e., channel-level loss), leading to a complete breakdown of the temporal evolution patterns of single variables. Traditional spatial graph models heavily rely on predefined topologies based on geographic distance, failing to characterize the complex asymmetric thermodynamic relationships between ocean hydrological and biochemical variables. This invention abandons the prior geographic matrix and employs a purely data-driven node embedding technique to construct a dynamic implicit topological graph. This mechanism can autonomously uncover asymmetric causal physical topologies, such as temperature fluctuations preceding dissolved oxygen changes, and achieve cross-variable information borrowing and semantic mapping through a deep nonlinear graph mixer. This innovation enables the model to reconstruct highly accurate missing data from seemingly nonexistent data by leveraging correlated variables, even in the face of the complete failure of a certain type of sensor.

[0047] (3) It endows the model with dynamic adaptive perception and intelligent arbitration capabilities in the face of complex missing multi-mode scenarios.

[0048] Real-world marine observation equipment is susceptible to disruptions from typhoons, biological attachments, or communication outages, resulting in highly random and volatile data loss patterns (e.g., a mix of instantaneous random point loss and continuous block loss over months). Traditional spatiotemporal networks typically employ fixed-weight feature stitching, which cannot cope with such dynamic and dramatic changes in the reliability of temporal and spatial information. This invention introduces a deep gated fusion (UGF) mechanism based on uncertainty variance estimation, endowing the model with real-time self-evaluation capabilities. During inference, the model can quantify the prediction uncertainties of temporal and spatial branches point by point and smoothly switch attention according to the inverse variance weighting principle (e.g., absolutely relying on spatial graph inference during continuous gaps, and prioritizing temporal trends during communication packet loss). This adaptive arbitration mechanism ensures that the model maintains extremely high robustness under various extreme conditions.

[0049] (4) It provides a highly inclusive unified processing framework, which completely solves the problem of equipment heterogeneity in marine observation networks.

[0050] Global ocean observation platforms come from diverse sources, and buoys in different sea areas and for different purposes often carry drastically different combinations of sensors. Traditional methods require input with fixed feature dimensions, resulting in the need to train hundreds or thousands of isolated models for different buoys, making engineering deployment extremely cumbersome and unable to leverage the knowledge of the equipment to assist sites with rudimentary equipment. This invention innovatively proposes a Union Set Filling (USP) and point-level validity masking mechanism to uniformly project all unstructured local observation data into a global high-dimensional feature space. This design paradigm, where a single model is compatible with all site structures, greatly simplifies the preprocessing chain of ocean big data, breaks down barriers to the transfer of physical knowledge across sites, and demonstrates excellent generalization and coordination capabilities.

[0051] (5) It balances the long-term dependency capture of ultra-long sequences with linear computation efficiency and has strong potential for business deployment.

[0052] To accurately extrapolate ocean systems over time, models must possess massive memory capacity to trace historical patterns over tens of thousands of hours. However, traditional advanced architectures such as Transformer are limited by quadratic computational complexity, facing severe memory overflow and inference latency when processing high-frequency, ultra-long ocean sequences, making them difficult to implement in real-time operational systems. This invention uses the latest linear state-space model (Mamba) as its core backbone architecture, coupled with a seamless inference mechanism using overlapping sliding windows. While maintaining the ability to remember infinite context, it significantly reduces computational time complexity to the linear level. This design perfectly resolves the contradiction between long-range dependencies and computational overhead, enabling this invention to achieve high-throughput, high-precision data repair with extremely low hardware power consumption, clearing technical obstacles for the implementation of practical engineering forecasting services such as digital twin oceanography and meteorological disaster early warning.

[0053] This invention aims to overcome the limitations of existing marine data processing technologies. It abandons the rigid design of traditional models that rely on fixed input dimensions, predefined geographic topology maps, and static spatiotemporal fusion mechanisms. Instead, it adopts a unified high-dimensional feature mapping driven by all data, and entrusts the task of capturing multidimensional marine spatiotemporal evolution patterns to a hybrid expert time-frequency architecture and implicit dynamic graph learning module. Finally, it achieves dynamic adaptive scheduling of multi-source information through uncertainty assessment. This invention refers to a cutting-edge deep learning spatiotemporal joint reconstruction framework specifically designed to address the severe challenges of inconsistent multi-source data structures, complex spatiotemporal correlations, and varied missing patterns in complex marine observation environments. Traditional models typically rely on fixed-dimensional inputs and can only perform simple single-dimensional interpolation, making them difficult to adapt to heterogeneous marine data (i.e., differences in the types and quantities of variables at different observation stations) caused by varying sensor configurations in practical applications. To address this issue, this invention first designs a Union Set Padding (USP) strategy. By constructing a global variable set covering all observed variables and combining it with fine-grained validity masks, unstructured heterogeneous data is mapped to a unified high-dimensional global feature space, thereby overcoming the limitations of physical devices and achieving cross-site universal feature learning. At the feature extraction level, the core innovation of this method lies in the collaborative design of temporal and spatiotemporal branches. In the temporal dimension, a hybrid expert time-frequency network (MoE-driven Temporal Network) is constructed. This network abandons the limitations of a single architecture and integrates in parallel state-space experts (such as bidirectional Mamba) adept at long-distance trend extrapolation, frequency-domain filtering experts adept at extracting physical periodic rhythms, and local smoothing experts capable of handling high-frequency abrupt changes. Through a temperature-scaled dynamic routing network, the model can adaptively analyze the missing features of the current sequence at the token level, intelligently schedule and combine the most suitable expert resources, thereby deeply analyzing the non-stationary temporal evolution of the marine environment. In the spatial dimension, to address the complex implicit causal relationships among heterogeneous variables that are difficult to measure using geographical distance, this framework introduces an Adaptive Implicit Dynamic Graph Learning mechanism. By assigning learnable node embedding vectors to each type of variable, it dynamically infers asymmetric topological structures reflecting physical mechanisms such as temperature-salt coupling under fully data-driven conditions. This enables efficient cross-variable inference by aggregating effective information from strongly correlated variables even when a certain type of sensor experiences complete channel-level failure. Finally, facing variable and extreme missing data scenarios, this invention employs an Uncertainty-Gated Fusion (UGF) mechanism to evaluate the prediction variance of spatiotemporal features in real time and dynamically allocate fusion weights based on the inverse variance weighting principle.This mechanism enables the model to adaptively switch between historical trend evolution and cross-variable spatial reasoning based on the degree of information loss, giving the model extremely high robustness and high-fidelity data repair capabilities under extreme missing conditions.

[0054] The technical solution provided by this invention includes the following steps during data flow: The original heterogeneous observation sequence is aligned using a union set filling USP and then concatenated with a fine-grained point-level observation validity mask before being mapped to a high-dimensional feature space. The feature streams enter two feature extraction hubs in parallel. In the time dimension, fine decoupling is achieved through a hybrid expert time-frequency network. In the spatial dimension, feature reconstruction is performed through an adaptive implicit dynamic graph network. The temporal and spatial depth features are merged into an uncertainty-gated fusion UGF module for adaptive feature concatenation and decoding. Finally, a hard residual connection mechanism is used to output a high-fidelity complete ocean data sequence. This method achieves strong robustness and high fidelity in ocean data imputation.

[0055] The various steps in the embodiments of the present invention can be performed by an electronic device. This electronic device includes, but is not limited to, tablet computers, portable PCs, and desktop computers.

[0056] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is running, it controls the electronic device containing the computer-readable storage medium to execute the above-described embodiment of the ocean heterogeneous data spatiotemporal filling method based on hybrid time-frequency experts and dynamic graphs.

[0057] Figure 15 A schematic diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 15 As shown, the electronic device 21 includes a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the spatiotemporal filling method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs in the embodiment. To avoid repetition, it will not be described in detail here.

[0058] Electronic device 21 includes, but is not limited to, processor 211 and memory 212. Those skilled in the art will understand that... Figure 15 This is merely an example of electronic device 21 and does not constitute a limitation on electronic device 21. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0059] The processor 211 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0060] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or RAM of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or FlashCard equipped on the electronic device 21. Furthermore, the memory 212 can include both internal and external storage units of the electronic device 21. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 can also be used to temporarily store data that has been output or will be output.

[0061] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A spatiotemporal incomplete method for heterogeneous ocean data based on hybrid time-frequency experts and dynamic graphs, characterized in that, The method includes: Step 1: In the data flow, the original heterogeneous observation sequence is aligned by union set filling USP and then concatenated with fine-grained point-level observation validity mask and mapped to a high-dimensional feature space. Step 2: Based on Step 1, the feature streams enter two feature extraction hubs in parallel. In the time dimension, fine decoupling is achieved through a hybrid expert time-frequency network; in the spatial dimension, feature reconstruction is performed through an adaptive implicit dynamic graph network. Step 3: Based on Step 2, the temporal and spatial depth features are incorporated into the uncertainty-gated fusion UGF module for adaptive feature stitching and decoding; and a high-fidelity complete ocean data sequence is output in conjunction with a hard residual connection mechanism. Step 2 includes: In the time dimension, the hybrid expert time-frequency network dynamically assigns sequences to state-space experts, frequency-domain filtering experts, and local smoothing experts through intelligent temperature routing, so as to finely decouple long-range evolution trends from high-frequency physical cycles; in the spatial dimension, the adaptive implicit dynamic graph network infers the asymmetric implicit physical topology between heterogeneous variables through data-driven node embedding and performs cross-variable information aggregation. A parallel temporal hybrid expert MoE time-frequency network is introduced, which consists of three heterogeneous expert branches and an adaptive routing network: I. Gated bidirectional state-space expert; For high-dimensional time-series features of the input First, two independent state-space models, Mamba, are instantiated. One is scanned along the forward time axis to extract forward features. Another approach involves flipping the input feature sequence along the time axis, then inputting the flipped output to extract backward features. ; After concatenating the bidirectional features along the channels, a linear mapping is applied. With Sigmoid activation function Generate channel-level gating coefficients Its expression is: ； Finally, time features are generated by weighted concatenation with hard residuals. Its expression is: ； in, This represents the Hadamard product, which is the element-wise multiplication of tensors. II. Frequency domain filtering expert; First, the feature sequence in the time dimension is mapped to the complex frequency domain using the real-valued Fast Fourier Transform (FFT). In the frequency domain, a learnable complex weight tensor is maintained. The complex weight tensor is used to automatically learn and retain key frequency components while suppressing random noise; simultaneously, it adjusts the amplitude and phase of the signal in the frequency domain; after filtering, it is restored to time-domain features via inverse fast Fourier transform (IFFT). ; III. Multi-scale local smoothing expert; A third parallel expert, the local smoothing expert, is introduced. It consists of parallel local convolutional layers with different multi-scale receptive fields, and multi-scale local temporal features are obtained through nonlinear activation mapping. Local smoothing experts are used to perform fast short-time interpolation using adjacent valid points within a very small time window when the Mamba field of view is too large. IV. Dynamic temperature routing and expert integration; The expert opinions adopted for the current data point are dynamically determined by token-level routing weights. Input features are fed into a routing network composed of a multilayer perceptron (MLP). A preset temperature scaling factor is introduced to sharpen the output affinity score, which is then converted into the assigned weights for each expert using a normalization function. Finally, the time branch output is obtained by weighted summation of the features from each expert and normalized layer by layer. .

2. The method according to claim 1, characterized in that, Step 1 includes: The design incorporates a union set filling USP alignment and mask-aware feature embedding mechanism, which transforms unstructured original heterogeneous observation sequences into a unified high-dimensional feature representation through three logical steps. Step 11: Global variable mapping and heterogeneous data alignment; First, extract all types of observed variables from the monitoring network and construct a system containing... Global collection of class variables Subsequently, the inputs of any local station are mapped to the global dimension space to construct a unified global data matrix. , This indicates the length of the time series, i.e., the number of time steps. Represents the set of real numbers; if the station is not equipped with a specific sensor, the current variable column is zero-padded to force all heterogeneous station data to be aligned into a structured tensor with consistent dimensions, that is, to obtain the aligned observation tensor; Step 12: Construction of fine-grained point-level observation validity mask; Constructing a fine-grained point-level observation validity mask matrix If the site is in time Actual observed variables The effective value of , then If the site does not have a specific sensor installed, or if the specific sensor has missing native data at the current moment, then ; Step 13: Mask-aware early feature fusion embedding; The aligned observation tensor and the validity mask are explicitly concatenated along the channel dimension to obtain a multi-channel composite input tensor. The multi-channel composite input tensor is then input into the embedding module of the two-dimensional convolution Conv2d. The convolution kernel performs local temporal smoothing by sliding across time steps but not across variables, fusing numerical and mask channels and mapping them to a high-dimensional feature space. The high-dimensional mask-aware features are then distributed to the downstream network.

3. The method according to claim 1, characterized in that, Step 2 also includes: Spatial feature extraction is performed through an adaptive implicit dynamic graph network that does not require any prior graph structure. It is data-driven and uses asymmetric implicit physical topology mining and deep cross-variable information interaction to reconstruct features in extreme missing conditions. V. Dynamic graph construction based on node embedding; Treating various ocean variables as nodes in a graph, in the latent space... The class variable node dynamically initializes two independent sets of learnable parameter matrices: (1) source node embedding matrix (2) Target node embedding matrix , used to characterize the physical properties of the current variable when it acts as an information receiver, where For the embedding dimension, d << ; By calculating the similarity between the source node embedding matrix and the target node embedding matrix, and combining nonlinear activation and sparsification mechanisms, the association score matrix between nodes is adaptively generated. Since two sets of independent embedding features are used, the association score matrix has asymmetry, which fits the unidirectional or asymmetric causal physical correlation in real ocean dynamics. VI. Nonlinear Enhanced Spatial Graph Aggregation; Constructing a dynamic adjacency matrix Subsequently, graph neural networks are used to transmit information along the variable dimension; to handle the complex high-dimensional nonlinear mappings between heterogeneous ocean variables, a deep nonlinear mixing mechanism is introduced, the expression of which is: ； ； in, The input is the spatial feature matrix; This is the message passing feature matrix after aggregating neighbor node information. The spatial characteristics of the output; It is a deep nonlinear feature hybrid network; This is a layer normalization operation used to stabilize training; When a physical sensor experiences hardware-level failure, i.e., a complete loss of channel, the time reference system of the currently damaged node is completely broken. To address this, a dynamic adjacency matrix is ​​used to borrow health data from strongly correlated variables, and a deep feature hybrid network is employed to fit complex nonlinear mapping rules between heterogeneous variables, achieving high-dimensional semantic transformation across variables. Finally, residual connections and layer normalization are used to output spatial features. .

4. The method according to claim 1, characterized in that, Step 3 includes: An uncertainty-gated fusion UGF module is incorporated to evaluate the prediction variance of the two branches in real time, and adaptive feature concatenation and decoding are performed based on the inverse variance weighting principle; uncertainty-gated fusion module based on the inverse variance weighting principle: a) Local uncertainty variance estimation: Assuming the output of each feature branch follows a Gaussian distribution, its variance represents the uncertainty of the current feature branch with respect to the current prediction; first, two lightweight variance estimation networks are constructed. and Variance maps were extracted from the spatiotemporal features respectively. and Its expression is: ； ； in, Features output by the time branch; Features output by spatial branching; A smooth activation function is used to ensure that the output variance is always positive; It is a very small constant used to prevent numerical instability when divided by zero during calculation; Two sets of variance plots are used to evaluate the prediction confidence of the two branches at the current location point-wise over time. b. Inverse variance weighted allocation: Based on the minimum variance principle in Bayesian inference, branches with smaller variance have higher confidence levels. An inverse variance weighting strategy is applied, where weights are weighted by... Proportional to the time branch, the gating weights are generated by calculating the relative uncertainty. Its expression is: ； c. Cascaded deep feature mixing and decoding: After obtaining dynamic weights, a weighted cascaded-deep mixing strategy is adopted. First, the weighted features are calculated. Its expression is: ； in, This indicates a feature concatenation operation performed along the channel dimension; Subsequently, the weighted feature tensor is concatenated along the channel dimension. The output is mapped to an initial prediction tensor with the same dimensions as the original heterogeneous observation sequence. .

5. The method according to claim 4, characterized in that, Step 3 also includes: The final output is constructed using a hard residual connection mechanism, and its expression is: ； in, This is the validity mask for the original data, i.e., in the validity mask The observation points retain the true original values, only when Missing points are filled in to fill in the network predictions to ensure strict physical consistency; To comprehensively optimize and fill the model while ensuring the physical plausibility of the generated data, the overall loss function will be... The design employs a multi-task joint optimization approach, incorporating time-domain reconstruction errors and introducing frequency-domain constraints, smoothness penalties, and adversarial feedback. The expression is as follows: ； in, and The temporal reconstruction losses are for the mask missing region and the globally effective region, respectively. To balance robustness to extreme outliers and smooth gradients, a hybrid form of L1 and L2 norms is adopted. The frequency domain amplitude error (i.e., the L1 norm based on the fast Fourier transform) is used to force the model to capture the global physical period unique to ocean data and prevent non-physical phase shifts in the generated results. For total variation smoothing loss, high-frequency jitter noise is suppressed by calculating the absolute difference between adjacent time steps; The discrimination error provided to the generative adversarial network is used to improve the realism of local temporal textures; , , , To balance the hyperparameters of various errors; in the early stages of training, Set it to 0 for preheating, then give it a very small weight to refine the texture.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the spatiotemporal filling method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs as described in any one of claims 1 to 5.

7. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the device, cause the device to perform the spatiotemporal completion method for ocean heterogeneous data based on hybrid time-frequency experts and dynamic graphs as described in any one of claims 1 to 5.

Images