A spatio-temporal prediction method based on wavelet coding and space-frequency dual-domain feature fusion
By using a non-cyclic method that fuses wavelet encoding and decoding with spatial-frequency dual-domain features, the problems of information loss and single feature modeling in existing spatiotemporal prediction are solved, achieving efficient and accurate spatiotemporal prediction that can adapt to complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SICHUAN UNIV
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-09
AI Technical Summary
Existing spatiotemporal prediction methods have shortcomings in computational efficiency and prediction accuracy. Recurrent methods are inefficient and prone to error accumulation, while acyclic methods are prone to information loss and feature modeling is too simplistic, making it difficult to depict subtle evolutions or rapid transitions.
A non-cyclic end-to-end prediction method based on wavelet encoding and decoding and spatial-frequency dual-domain feature fusion is adopted. Multi-scale spatial semantic features are extracted by wavelet downsampling, and combined with depthwise separable convolution and frequency-domain gated feature interaction to explicitly characterize temporal changes and achieve cross-temporal and spatial feature interaction and accurate prediction.
While maintaining high computational efficiency, it reduces information loss, enhances the ability to represent spatiotemporal changes, improves prediction accuracy and stability, and adapts to scenarios with subtle evolution and rapid transitions.
Smart Images
Figure CN122173824A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of spatiotemporal prediction technology, specifically to a spatiotemporal prediction method based on wavelet encoding and decoding and spatial-frequency dual-domain feature fusion. Background Technology
[0002] Spatiotemporal prediction is a crucial foundational capability for inferring future states, widely serving scenarios such as intelligent transportation and urban governance, meteorological and environmental monitoring, energy dispatching and industrial control, and is crucial to social operational safety and resource allocation efficiency. With the continuous growth in the scale of sensor observations and historical data, application demands are gradually shifting from coarse-grained trend judgments to finer-grained, lower-latency quantitative predictions. Spatiotemporal prediction typically learns by predicting future frames given past frames, representing a typical spatiotemporal modeling problem. This type of data possesses both a clear spatial structure and dynamic patterns evolving over time, with significant differences in numerical range, magnitude of change, and spatial morphology across different tasks. In practical applications, prediction models need to handle long-sequence inputs with high computational efficiency while accurately depicting the co-evolution of slowly changing trends and local disturbances to meet the needs of process monitoring and decision support. Therefore, how to improve the ability to represent spatiotemporal changes while ensuring efficiency remains a continuous focus in this field.
[0003] Existing spatiotemporal prediction learning frameworks can be broadly categorized into two types. The first is a cyclic spatiotemporal modeling method, which typically maintains hidden states through a cyclic structure and recursively generates future sequences step-by-step, capable of characterizing time dependencies and progressively refining predictions. The second is a non-cyclic end-to-end method, which typically employs a "spatial encoding—spatiotemporal transformation—spatial decoding" process, mapping historical sequences to future sequences in a single step, offering better parallel computing efficiency and engineering scalability.
[0004] Both of these methods still have shortcomings. Recurrent methods, due to their reliance on the output of the previous step, are difficult to parallelize fully, limiting overall efficiency. Furthermore, with longer prediction steps, errors tend to accumulate and amplify with each iteration, affecting the reliability of predictions in later time steps. While acyclic methods have higher inference efficiency, they often rely on conventional convolutions for sampling, which can lead to information loss. They also tend to use a single spatial domain modeling approach, and their modeling of temporal changes is relatively indirect, mainly relying on implicit feature learning. This limits their ability to characterize subtle evolutions or rapid transitions, thus restricting prediction accuracy. Summary of the Invention
[0005] To address the aforementioned shortcomings in existing technologies, this invention proposes a spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion. This method solves the problems of easy information loss, single feature domain, and relatively indirect temporal modeling in existing spatiotemporal prediction methods. At the same time, it adopts an acyclic end-to-end prediction paradigm to maintain high computational efficiency and achieve accurate prediction of future spatiotemporal sequences.
[0006] This invention is achieved through the following technical solution:
[0007] A spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion includes:
[0008] Obtain the source spatiotemporal sequence consisting of multiple time steps, and organize it in the batch dimension and time dimension to construct the input sequence and the target sequence;
[0009] By progressively compressing the spatial resolution through wavelet downsampling, the input sequence is encoded step by step to extract spatial semantic representations with multi-scale characteristics.
[0010] The encoded representations are organized into a feature sequence in chronological order for time-series modeling.
[0011] By inputting spatiotemporal features containing state information and change clues into the spatiotemporal transformation network, cross-spatiotemporal feature interaction can be achieved;
[0012] The features after spatiotemporal transformation are decoded step by step, and the spatial resolution is gradually restored through inverse wavelet reconstruction to generate future prediction sequences.
[0013] In the above technical solution, the source spatiotemporal sequence composed of multiple time steps is preprocessed, including:
[0014] For any source spatiotemporal sequence data, the original records are parsed into a time-ordered frame sequence and uniformly represented as tensors;
[0015] By adapting different data to each other, the entire sequence is divided into training, validation and testing segments in chronological order.
[0016] To unify the numerical range and statistical scale of different source data, a linear mapping is performed on each channel within the segment; supervised samples are constructed by using historical and future windows near the valid anchor points to obtain the input sequence and the target sequence.
[0017] In the above technical solution, during the encoding process, shallow features are first obtained and retained as skip connections for detail compensation in the subsequent decoding stage, including:
[0018] For each frame of the input sequence, shallow features of each frame are obtained by a shallow feature extraction operator consisting of two layers of 3×3 convolution and an intermediate nonlinear activation function.
[0019] In the above technical solution, shallow features are executed. Repeated encoding operations yield encoded implicit representations, including:
[0020] Each coding module first performs a two-dimensional discrete wavelet transform on the input features, then concatenates the four sub-bands according to the channel dimension, and completes the sub-band fusion projection through 1×1 convolution; then it undergoes two frequency selection enhancements to obtain the output at the current scale;
[0021] Perform the same encoding operation as above on each frame of the input sequence. Next, the encoded implicit representation is obtained;
[0022] This represents the number of encoding / decoding operations.
[0023] In the above technical solution, short-term change cues that are not easily expressed explicitly in the encoded implicit representation are supplemented by injecting differential features from adjacent frames, including:
[0024] For each frame of encoded features, a frame-by-frame 1×1 convolutional projection is first applied to perform channel mapping, resulting in frame-by-frame independent projection features;
[0025] The difference between adjacent frames is constructed from the projection features, and organized in the channel dimension in time order. A 3×3 depthwise separable convolution and nonlinear activation are applied to the organized features to obtain the motion enhancement term.
[0026] We define a channel-by-channel learnable and spatially broadcast gating coefficient, which is used to control the injection intensity of motion enhancement terms, thereby enhancing features organized sequentially in the channel dimension in the form of residuals.
[0027]
[0028] in, This is the implicit representation after injection. The gating coefficient, This is an element-wise multiplication operation. For strengthening exercises, The features are the projection features of each frame organized in chronological order along the channel dimension. This serves as the anchor index for the sample.
[0029] In the above technical solution, the processed latent representation is input into a spatiotemporal transformation network, spatial and frequency domain features are extracted and fused in parallel, and cross-channel feature interaction is achieved by combining gated channel mixing, and the above process is repeated. Next, including:
[0030] For any sub-block, local texture and global pattern are modeled using a combination of parallel spatial convolution and frequency domain gating:
[0031]
[0032]
[0033] in, , Input for sub-blocks, , For group normalization operators, It is a 9×9 depth separable convolution. and These are the two-dimensional real Fourier transform and its inverse transform, respectively. For learnable complex weights that match the spectral size, For spatial domain branch output features, For frequency domain branch output features;
[0034] The two domains are then fused and combined with residual injection to obtain intermediate features to enhance the spatiotemporal representation;
[0035] Then, normalization and 1×1 convolution are applied to the intermediate feature to expand its channels. It is then divided into two equal parts along the channel dimension, resulting in a gated branch and a gated feature branch. A 3×3 depthwise separable convolution is used to preserve the local structure of the feature branch. After gated multiplication, a 1×1 convolution back projection is performed to obtain the gated channel blending result.
[0036]
[0037] in, It is a 3×3 depthwise separable convolution. The SiLU activation function is used. Represents a 1×1 convolution. For gated branches, For feature branches, This is a result of gated channel mixing;
[0038] The first is obtained through residuals The output of a dual-domain gated converter sub-block;
[0039] Stacking The above sub-blocks are processed to obtain the output. A frame-by-frame 1×1 convolutional projection is applied to the output to map the channels back to the original dimensions and organize them into a frame-by-frame feature sequence in chronological order.
[0040] In the above technical solution, the decoder and encoder maintain symmetry in the number of scales. The aforementioned feature sequence is input into the decoder, and the spatial scale is gradually restored to its initial state through recursion at each scale during the decoding process, thereby obtaining the predicted sequence, including:
[0041] Each decoding module first performs frequency selection enhancement on the input features, then maps the channels to the channel layout corresponding to the four sub-bands through 1×1 convolution, and performs two-dimensional inverse wavelet reconstruction to obtain a higher resolution representation;
[0042] Perform the same decoding operation as above on each frame. This yields full-resolution features;
[0043] This represents the number of encoding / decoding operations.
[0044] In the above technical solution, the full-resolution features are summed with the shallow features retained at the encoding end to restore details. After a 3×3 readout convolution, the predicted frame is obtained, which is then used to form the future prediction sequence.
[0045] The above technical solution also includes:
[0046] End-to-end training optimization is performed using the mean squared error between the predicted sequence and the target sequence as the loss function.
[0047] The trained model is used to predict the test data and calculate the evaluation index.
[0048] The present invention also provides a spatiotemporal prediction device based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion, comprising:
[0049] The acquisition unit is used to acquire a source spatiotemporal sequence consisting of multiple time steps, and organize it in the batch dimension and time dimension to construct the input sequence and the target sequence.
[0050] The coding unit is used to progressively compress the spatial resolution through wavelet downsampling, encode the input sequence step by step, and extract spatial semantic representations with multi-scale characteristics.
[0051] The temporal modeling unit is used to organize the encoded representations into a feature sequence in chronological order for temporal modeling.
[0052] The spatiotemporal transformation unit is used to input spatiotemporal features containing state information and change clues into the spatiotemporal transformation network to achieve cross-spatiotemporal feature interaction;
[0053] The decoding unit is used to decode the features after spatiotemporal transformation step by step, gradually restore the spatial resolution through inverse wavelet reconstruction, and generate future prediction sequences.
[0054] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0055] This invention maintains high computational efficiency by employing a non-cyclic method;
[0056] This invention uses wavelet encoding and decoding to decompose and fuse multiple subbands, taking into account both low-frequency trends and high-frequency disturbances during downsampling and reconstruction, thereby reducing information loss caused by sampling and solving the problem of easy loss of key information caused by conventional convolutional sampling in existing methods.
[0057] This invention integrates parallel spatial large kernel depthwise separable convolution with frequency domain gated feature extraction, enabling the model to utilize both spatial structure and frequency domain response simultaneously. This solves the problem that existing methods mainly rely on a single spatial transformation, resulting in limited feature modeling.
[0058] This invention introduces temporal variations in an explicit differential form through a cross-frame interaction mechanism of differential injection between adjacent frames, which solves the problem that existing methods mainly rely on implicit learning, resulting in relatively indirect temporal modeling. Attached Figure Description
[0059] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:
[0060] Figure 1 This is a schematic diagram of the process of the present invention;
[0061] Figure 2 This is a schematic diagram of the specific process of this embodiment;
[0062] Figure 3 This is a schematic diagram of the apparatus for implementing this embodiment. Detailed Implementation
[0063] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention. Example 1
[0064] like Figure 1 As shown, the overall process of the spatiotemporal prediction method proposed in this embodiment is as follows:
[0065] First, a raw spatiotemporal observation sequence consisting of multiple time steps is received and expanded in the batch dimension and time dimension before being fed into the spatial coding module. The spatial resolution is compressed by multi-level wavelet downsampling, and frequency selection enhancement blocks are introduced at each scale to extract spatial semantic features with multi-scale characteristics while reducing computational complexity. During the coding process, the initial shallow features are retained as skip connections for detail compensation in the subsequent decoding stage.
[0066] Subsequently, the encoded features enter the temporal modeling module. This module first performs channel mapping through frame-by-frame convolution to obtain independent projection features for each frame without temporal mixing. Based on this, it calculates the differential changes of state features at adjacent time steps and extracts local motion patterns through depthwise separable convolution and nonlinear activation functions. At the same time, it uses learnable gating parameters to adaptively adjust the influence intensity of temporal change cues on the original features and fuses them into the original features in a residual manner. This enhances the model's ability to perceive temporal dynamics while ensuring training stability.
[0067] Subsequently, the spatiotemporal features that integrate state information and change cues are fed into a spatiotemporal transformation network composed of multiple spatiotemporal hybrid sub-blocks. In this network, the features undergo spatial local modeling and frequency domain global modulation, are fused in parallel, and then undergo channel nonlinear interaction. Stable deep stacking is achieved through residual and normalization structures to capture high-order dynamic dependencies across time steps.
[0068] After feature evolution is completed, each frame is first mapped back to its original dimension through frame-by-frame convolution. Then, the spatial resolution is restored step by step through a decoding module symmetrical to the encoding module. In the final readout convolution, jump features from the encoding end are introduced to compensate for detail information. Finally, the output is a multi-timestep prediction result that is consistent with the input in spatial size and data format. Through the above process, this invention achieves joint modeling of state evolution and change trends in spatiotemporal sequences without relying on explicit physical modeling, while taking into account prediction accuracy, stability, and generalization ability.
[0069] Based on the above embodiments, a spatiotemporal prediction device based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion is also provided, such as... Figure 3 As shown, it includes:
[0070] The acquisition unit is used to acquire a source spatiotemporal sequence consisting of multiple time steps, and organize it in the batch dimension and time dimension to construct the input sequence and the target sequence.
[0071] The coding unit is used to progressively compress the spatial resolution through wavelet downsampling, encode the input sequence step by step, and extract spatial semantic representations with multi-scale characteristics.
[0072] The temporal modeling unit is used to organize the encoded representations into a feature sequence in chronological order for temporal modeling.
[0073] The spatiotemporal transformation unit is used to input spatiotemporal features containing state information and change clues into the spatiotemporal transformation network to achieve cross-spatiotemporal feature interaction.
[0074] The decoding unit is used to decode the features after spatiotemporal transformation step by step, gradually restore the spatial resolution through inverse wavelet reconstruction, and generate future prediction sequences. Example 2
[0075] like Figure 2 As shown, this embodiment includes the following processing steps in the overall process:
[0076] S1: Preprocess the source spatiotemporal sequence to construct the input and target sequences. This step includes the following sub-steps:
[0077] S11. Parse the source spatiotemporal sequence into a time-ordered frame sequence and unify it into tensor form.
[0078] Record No. Frame observation as ( ),in C represents the total number of frames, H represents the number of channels, and W represents the spatial resolution.
[0079] The entire sequence was then divided into training, validation, and testing segments in chronological order, denoted as... The frame sequence is obtained within partition D. Where k is the index of the partitioned inner frame, The total number of frames within partition D.
[0080] S12, Intra-partial Normalization and Sample Construction
[0081] To unify the numerical scale of data from different sources, a linear normalization parameter is set for each channel c within each partition D. , And perform channel-by-channel normalization on all frames within the partition:
[0082]
[0083] Where c is the channel index. For spatial location index; That is, the k-th frame after normalization, the parameters are concatenated according to the channel dimension and saved as a broadcastable shape, denoted as... , .
[0084] Let the input length be The predicted length is Within partition D, a sliding window sampling is performed using anchor point n. The input sequence is constructed from the historical window, and the target sequence is constructed from the future window.
[0085]
[0086]
[0087] in, For sample anchor index, For the first The input sequence of a sample, The corresponding target sequence; the time interval is determined by the sampling step size of each data point.
[0088] This step enables the unified tensor representation of frame sequences from different source data; completes the partitioning of the dataset to ensure consistent evaluation criteria; and generates input and target sequences through a sliding window to form stable supervised samples.
[0089] S2: Input the input sequence into a wavelet decomposition-based encoder to obtain the latent representation obtained by sub-band feature fusion. This step includes the following sub-steps:
[0090] S21. Frame-by-frame shallow feature extraction, taking the t-th frame of the n-th sample as input... First, the shallow features of each frame are extracted:
[0091]
[0092] in, , ; This represents the number of shallow feature channels; It is a shallow feature extraction operator, consisting of two 3×3 convolutional layers and an intermediate nonlinear activation function.
[0093] S22. Multi-scale wavelet downsampling coding and implicit representation output
[0094] Wavelet downsampling is used to explicitly decompose the features into low-frequency structure and high-frequency details, which are then fused into a compact representation to preserve key information as much as possible during resolution reduction. Let the number of encoding operations be... The scale number is , No. Spatial resolution at each scale satisfies , .
[0095] For each frame at scale Features First, a two-dimensional discrete wavelet transform is applied to obtain four sub-bands:
[0096]
[0097] in, This represents the two-dimensional discrete wavelet transform (DWT). , These represent the low-frequency approximate subband and the high-frequency detail subbands in three directions, respectively; this operation changes the spatial scale from... downsampling .
[0098] The four subbands are then spliced together along the channel dimension and fused and projected using a 1×1 convolution:
[0099]
[0100] in, This indicates a channel-dimensional splicing operator. Represents a 1×1 convolution; For the first The intermediate features are obtained by fusing the information from the four subbands after downsampling in the secondary encoding operation.
[0101] To enhance the characterization capability of key spatial frequency modes in subband fusion features, in the first... In the secondary encoding operation, If two frequency-selective enhancement blocks are stacked in series, the output at this scale will be:
[0102]
[0103] in, The frequency selection enhancement operator is used to enhance the fused features in a frequency-selective manner. Internally, it combines two types of processing in a residual manner: one is a gated spatial enhancement branch, which first performs normalization and generates gated weights, and then enhances key structures and texture patterns through depthwise separable convolution; the other is a point-to-point channel blending branch, which realizes nonlinear interaction and recalibration across channels through point-to-point convolution.
[0104] Perform the same encoding operation on each frame of the input sequence. Next, the encoded hidden representation is obtained. .
[0105] This step preserves both low-frequency structure and high-frequency details, forming a compact implicit representation through sub-band fusion, thus reducing the loss of key information caused by downsampling.
[0106] S3: Inject differential features from adjacent frames into the latent representation. This step includes the following sub-steps:
[0107] S31. Difference between adjacent frames and motion enhancement extraction:
[0108] For the encoded hidden representation sequence For each frame of features, a frame-by-frame 1×1 convolutional projection is first applied, reducing the number of channels from... Mapped to , to obtain frame-by-frame projection features .
[0109] Constructing the difference between adjacent frames:
[0110]
[0111] Will Organize in chronological order along the channel dimension (i.e., incorporate the time dimension into the channel dimension) to obtain Subsequently, a 3×3 depthwise separable convolution and nonlinear activation are applied to it to obtain the motion enhancement term:
[0112]
[0113] in, It is a 3×3 depthwise separable convolution. The SiLU activation function; This is a sports enhancement program.
[0114] S32, Gated Residual Injection:
[0115] Frame-by-frame projection features Similarly, organize the data in chronological order along the channel dimension to obtain... A gate coefficient is set to be learnable per channel and broadcastable in spatial dimension. ,use Control the injection intensity of the motion enhancement term and enhance it using residuals:
[0116]
[0117] in, This is the implicit representation after injection. This is an element-wise multiplication operation.
[0118] This step can explicitly depict changes between adjacent frames, inject short-term dynamic cues, and improve the temporal sensitivity to rapid evolution processes.
[0119] S4: Input the processed latent representation into the spatiotemporal transformation network, extract and fuse spatial and frequency domain features in parallel, and combine gated channel mixing to achieve cross-channel feature interaction.
[0120] Record the network input as The network is... The process consists of stacked dual-domain gated transformation sub-blocks. Each sub-block maintains the same tensor dimension and spatial resolution during forward propagation. Each sub-block contains: parallel spatial domain, frequency domain TokenMixing, and gated ChannelMixing, and includes the following steps:
[0121] S41, TokenMixing within a sub-block:
[0122] TokenMixing models local textures and global patterns through complementary parallel spatial convolution and frequency-domain gating, fusing them into contextual features to enhance spatiotemporal representation. (For sub-block input) First, normalization is performed to obtain the normalized features. ,in For group normalization operators, .
[0123] The spatial domain branch uses a 9×9 depthwise separable convolution:
[0124]
[0125] in, It is a 9×9 depth separable convolution.
[0126] To capture long-range spatial dependencies using the global receptive field, the frequency domain branch applies learnable complex weight gating in the two-dimensional real Fourier domain:
[0127]
[0128] in, and These are the two-dimensional real Fourier transform and its inverse transform, respectively. Learnable complex weights that match the spectral size.
[0129] Fuse dual-domain features and inject residuals:
[0130]
[0131]
[0132] in, For the first The learnable scalar of each sub-block; Use the GELU activation function; This is a global response normalization operator used for channel-level recalibration. These are intermediate features after TokenMixing.
[0133] S42, ChannelMixing within a sub-block:
[0134] ChannelMixing adaptively filters effective features through non-linear information interaction across channels.
[0135] right First, normalization is performed to obtain the normalized features. .
[0136] After 1×1 convolution pairs After channel expansion, the channel is divided into two equal parts along the channel dimension, which are denoted as gated branches. With gated feature branches The feature branches are preserved through 3×3 depthwise separable convolutions, and after gated multiplication, 1×1 convolutional back projection is performed to obtain the gated channel mixing result. :
[0137]
[0138] The first is obtained through residuals The output of a dual-domain gated converter subblock :
[0139]
[0140] through The sub-blocks are stacked to obtain ,right Apply frame-by-frame 1×1 convolutional projection, reducing the number of channels per frame from... Map back The results are then organized into a frame-by-frame feature sequence in chronological order (that is, separating the time dimension from the channel dimension). .
[0141] This step involves parallel modeling of spatial textures and frequency domain patterns, which are then fused to enhance global and local representations. Gated channel interaction improves the efficiency of effective information transmission.
[0142] S5: Input the spatiotemporally transformed representation into the decoder and generate the future prediction sequence through inverse wavelet reconstruction.
[0143] The decoder and encoder are symmetrical in scale number, and the number of decoding operations is the same. And follow the spatial scale relationship , .Will As the coarsest scale at the decoding end The input, that is, let Decoding end Decoding operations are performed step by step, changing the spatial scale from... Gradually recover to It includes the following steps:
[0144] S51. Decoding operations, frame-by-frame enhancement and inverse wavelet reconstruction:
[0145] For the first The input representation at each scale is first enhanced frame-by-frame using a frequency-selective enhancement block; then, a 1×1 convolution maps the channels to the channel layout corresponding to the four sub-bands, and two-dimensional inverse wavelet reconstruction is performed to restore a higher spatial resolution. Therefore, a single decoding operation can be represented as follows:
[0146]
[0147] in, This represents the two-dimensional inverse wavelet reconstruction operator paired with the wavelet decomposition at the encoding end, completing the entire process. After the next decoding operation, full-resolution features are obtained. .
[0148] S52, Skip Connection Fusion and Readout Prediction Sequence:
[0149] Shallow features retained at the encoding end The jump residual is added to the full-resolution features mentioned above, and then passed through a readout convolution to obtain the predicted frame:
[0150]
[0151] in, The future predicted sequence is a 3×3 convolution. .
[0152] This embodiment defaults to ;when At that time, cut off the previous Frame as a result; when In this process, a rolling generation method is used, where the current predicted sequence is used as the input for the next round. The forward pass is repeated, and the outputs of each round are concatenated. If the number of frames generated in the last round exceeds the required number of steps, the frame count is truncated to ensure the final length is [value missing]. .
[0153] In this step, spatial resolution is restored by inverse wavelet and detailed information is supplemented by skip connections to generate a more consistent and stable multi-step prediction sequence.
[0154] S6: Perform end-to-end training optimization using the mean squared error between the predicted and target sequences as the loss function.
[0155] For training samples Model output The mean squared error (MSE) between the predicted sequence and the target sequence is used as the training objective.
[0156]
[0157] in, and The prediction and the target are respectively in time step. ,aisle ,Location The value for a small batch of samples The samples within the batch The average value is taken as the training loss for that batch, where the batch size can be expressed as... .
[0158] Let all learnable parameters be... Gradient descent-based methods are used to optimize the batch loss end-to-end; the learning rate is denoted as... The parameters are updated along the negative gradient direction, and the parameters that perform best on the validation set are retained as the optimal model weights during training.
[0159] In this step, a unified loss constraint is used to constrain the entire sequence error, guiding end-to-end collaborative learning among modules to ensure training convergence stability and generalization ability.
[0160] S7: Predict the test data based on the trained model and calculate the evaluation index.
[0161] S71, Test Inference and Prediction Sequence Generation:
[0162] For any sample anchor index , and its corresponding input sequence Reasoning is performed to obtain the target sequence Dimensionally consistent prediction sequences ,in To load the model with optimal parameters.
[0163] S72. Evaluation domain alignment and evaluation index calculation based on test segment data mapping:
[0164] To unify the numerical standards across different source datasets during the evaluation phase, channel-level affine alignment, i.e., inverse normalization transformation, is performed on the predicted and target sequences during the testing phase.
[0165]
[0166]
[0167] in, The channel-level affine parameters, determined during the testing phase, are applied to the sequence tensor via broadcast; when the source data does not require inverse transformation during evaluation, they are equivalently taken as... .
[0168] Within the evaluation domain, with and The differences are used as the source of error, and commonly used evaluation indicators are calculated to measure the prediction quality. Among them, MSE, MAE, and RMSE are used to characterize the magnitude of numerical error (the smaller the value of the three, the better the effect); PSNR is used to characterize the reconstruction quality, and SSIM is used to characterize the structural similarity (the larger the value of both, the better the effect).
[0169] This step generates prediction sequences in the test segment based on the optimal model. A unified evaluation criterion is used to achieve fair comparison with different methods. Multiple evaluation indicators are combined to comprehensively reflect the prediction performance, as shown below:
[0170] Table 1: Main parameter settings for this embodiment on different datasets. In WeatherBench, the univariate subtasks t2m, tcc, uv10, and r represent the 2m air temperature, total cloud cover, 10m wind speed component, and relative humidity, respectively.
[0171]
[0172] Table 2: Quantitative comparison of existing popular methods on the Moving MNIST dataset
[0173]
[0174] Table 3: Quantitative Comparison of Existing Popular Methods on the TaxiBJ Dataset
[0175]
[0176] Table 4: Quantitative Comparison of Existing Popular Methods on the WeatherBench Univariate Dataset
[0177]
[0178] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion, characterized in that... include: Obtain the source spatiotemporal sequence consisting of multiple time steps, and organize it in the batch dimension and time dimension to construct the input sequence and the target sequence; By progressively compressing the spatial resolution through wavelet downsampling, the input sequence is encoded step by step to extract spatial semantic representations with multi-scale characteristics. The encoded representations are organized into a feature sequence in chronological order for time-series modeling. By inputting spatiotemporal features containing state information and change clues into the spatiotemporal transformation network, cross-spatiotemporal feature interaction can be achieved; The features after spatiotemporal transformation are decoded step by step, and the spatial resolution is gradually restored through inverse wavelet reconstruction to generate future prediction sequences.
2. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in claim 1, characterized in that... Preprocessing of the source spatiotemporal sequence composed of multiple time steps includes: For any source spatiotemporal sequence data, the original records are parsed into a time-ordered frame sequence and uniformly represented as tensors; By adapting different data to each other, the entire sequence is divided into training, validation and testing segments in chronological order. Linear mapping is performed on each channel within the segment, and supervised samples are constructed using historical and future windows near the legal anchor points to obtain the input and target sequences.
3. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in claim 1, characterized in that... During the encoding process, shallow features are first obtained and retained as skip connections for detail compensation in the subsequent decoding stage, including: For each frame of the input sequence, shallow features of each frame are obtained by a shallow feature extraction operator consisting of two layers of 3×3 convolution and an intermediate nonlinear activation function.
4. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in claim 3, characterized in that... Perform on shallow features Repeated encoding operations yield encoded implicit representations, including: Each coding module first performs a two-dimensional discrete wavelet transform on the input features, then concatenates the four sub-bands according to the channel dimension, and completes the sub-band fusion projection through 1×1 convolution; then it undergoes two frequency selection enhancements to obtain the output at the current scale; Perform the same encoding operation as above on each frame of the input sequence. Next, the encoded implicit representation is obtained; This represents the number of encoding / decoding operations.
5. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in claim 1, characterized in that, For short-term change cues that are not easily expressed explicitly in the encoded implicit representation, differential features from adjacent frames are injected, including: For each frame of encoded features, a frame-by-frame 1×1 convolutional projection is first applied to perform channel mapping, resulting in frame-by-frame independent projection features; The difference between adjacent frames is constructed from the projection features, and organized in the channel dimension in time order. A 3×3 depthwise separable convolution and nonlinear activation are applied to the organized features to obtain the motion enhancement term. We define a channel-by-channel learnable and spatially broadcast gating coefficient, which is used to control the injection intensity of motion enhancement terms, thereby enhancing features organized sequentially in the channel dimension in the form of residuals. , in, This is the implicit representation after injection. The gating coefficient, This is an element-wise multiplication operation. For strengthening exercises, The features are the projection features of each frame organized in chronological order along the channel dimension. This serves as the anchor index for the sample.
6. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion according to claim 1, characterized in that, The processed latent representation is input into a spatiotemporal transformation network, where spatial and frequency domain features are extracted and fused in parallel. This is combined with gated channel mixing to achieve cross-channel feature interaction, and the above process is repeated, including: For any sub-block, local texture and global pattern are modeled using a combination of parallel spatial convolution and frequency-domain gating: , , in, , Input for sub-blocks, , For group normalization operators, It is a 9×9 depth separable convolution. and These are the two-dimensional real Fourier transform and its inverse transform, respectively. For learnable complex weights that match the spectral size, For spatial domain branch output features, For frequency domain branch output features; The two domains are then fused and combined with residual injection to obtain intermediate features to enhance the spatiotemporal representation; The intermediate feature is first normalized and then expanded by a 1×1 convolution. It is then divided into two equal parts along the channel dimension, resulting in a gated branch and a gated feature branch. A 3×3 depthwise separable convolution is used to preserve the local structure of the feature branch. After gated multiplication, a 1×1 convolution back projection is performed to obtain the gated channel blending result. , in, It is a 3×3 depthwise separable convolution. The SiLU activation function is used. Represents a 1×1 convolution. For gated branches, For feature branches, This is a result of gated channel mixing; The first is obtained through residuals The output of a dual-domain gated converter sub-block; Stacking The above sub-blocks are processed to obtain the output. A frame-by-frame 1×1 convolutional projection is applied to the output to map the channels back to the original dimensions and organize them into a frame-by-frame feature sequence in chronological order.
7. The spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion according to claim 1, characterized in that, The decoder and encoder maintain symmetry in scale number. The aforementioned feature sequence is input into the decoder, and the spatial scale is recursively restored to its initial state step by step through the scale-by-scale derivation process, including: Each decoding module first performs frequency selection enhancement on the input features, then maps the channels to the channel layout corresponding to the four sub-bands through 1×1 convolution, and performs two-dimensional inverse wavelet reconstruction to obtain a higher resolution representation; Perform the same decoding operation as above on each frame. This yields full-resolution features; This represents the number of encoding / decoding operations.
8. A spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in claim 7, characterized in that... The full-resolution features are summed with the shallow features retained at the encoding end to restore details. The predicted frames are obtained by a 3×3 readout convolution, which in turn form the future prediction sequence.
9. A spatiotemporal prediction method based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion as described in any one of claims 1-8, characterized in that... Also includes: End-to-end training optimization is performed using the mean squared error between the predicted sequence and the target sequence as the loss function. The trained model is used to predict the test data and calculate the evaluation index.
10. A spatiotemporal prediction device based on wavelet encoding / decoding and spatial-frequency dual-domain feature fusion, characterized in that... include: The acquisition unit is used to acquire a source spatiotemporal sequence consisting of multiple time steps, and organize it in the batch dimension and time dimension to construct the input sequence and the target sequence. The coding unit is used to progressively compress the spatial resolution through wavelet downsampling, encode the input sequence step by step, and extract spatial semantic representations with multi-scale characteristics. The temporal modeling unit is used to organize the encoded representations into a feature sequence in chronological order for temporal modeling. The spatiotemporal transformation unit is used to input spatiotemporal features containing state information and change clues into the spatiotemporal transformation network to achieve cross-spatiotemporal feature interaction; The decoding unit is used to decode the features after spatiotemporal transformation step by step, gradually restore the spatial resolution through inverse wavelet reconstruction, and generate future prediction sequences.