A red tide prediction method and system based on time interval and low-rank time-frequency Trasformer

Abstract

The present invention relates to the field of red tide prediction technology, and in particular to a red tide prediction method and system based on time intervals and low-rank time-frequency Transformer. The method comprises acquiring multi-source marine red tide data; performing data preprocessing on the acquired multi-source marine red tide data; performing feature extraction and feature fusion on the multi-source marine red tide data using a multi-scale convolutional neural network; decomposing and reconstructing the fused multi-scale features based on a wavelet transform decoupling mechanism; performing time interval semantic extraction on the reconstructed features, and extracting cross-channel dependencies using a low-rank attention layer. The present invention transforms the implicit ecological semantic features in the marine monitoring data into structured semantic vectors through time interval semantic extraction technology, and combines the cross-domain feature learning capability of the low-rank time-frequency Transformer model to effectively capture the common laws of red tide occurrence in different sea areas and reduce dependence on data from a single region.

Description

Technical Field

[0001] The present invention relates to the technical field of red tide prediction, and in particular to a red tide prediction method and system based on time interval and low-rank time-frequency trasformer. Background Art

[0002] Red tide prediction technology has always been a crucial research topic in the study of marine ecological environmental protection and sustainable utilization of marine resources. As a global marine ecological disaster, red tide outbreaks not only cause the mass mortality of marine life and disrupt the balance of the marine ecosystem, but also cause huge economic losses to fisheries and aquaculture, and even threaten human health through the food chain. In recent years, with the rapid economic development of coastal areas and the increasing pressure on the marine environment, the frequency, scale, and severity of red tides have shown an upward trend. Therefore, it is urgent to establish accurate and efficient red tide prediction methods. Currently, traditional red tide prediction methods mainly include prediction models based on empirical formulas, numerical simulation models, and some prediction algorithms based on machine learning.

[0003] Prediction models based on empirical formulas usually summarize simple mathematical relationships between environmental parameters (such as temperature, salinity, nutrient concentration, etc.) and the occurrence of red tides based on historical observational data. Although such models are simple to calculate and low-cost, they rely too much on specific regions and historical data, and lack a comprehensive consideration of the dynamic changes in complex marine ecosystems. The marine ecosystem is a complex system that includes multiple coupled physical, chemical, and biological factors. The hydrological conditions and community structures in different sea areas vary significantly, making it difficult to directly reuse empirical formulas in cross-regional scenarios. When the marine environment experiences abnormal fluctuations or new influencing factors, the model finds it difficult to accurately capture the nonlinear relationship between the data, resulting in large deviations in the prediction results.

[0004] Numerical simulation models simulate the occurrence and development of red tides by solving a set of equations that describe the physical, chemical, and biological processes of the ocean. Specifically, such models usually require the construction of a complex system of equations including fluid dynamics equations, material transport equations, plankton ecodynamics equations, etc., in order to characterize the spatiotemporal changes of key factors such as temperature, salinity, nutrient concentration, and plankton population dynamics in the marine environment. However, such models require a large amount of initial and boundary condition data, and the determination of model parameters relies on professional knowledge and experience. The cost of data acquisition and model debugging is extremely high. At the same time, due to the complexity of the marine ecosystem, it is often difficult for the model to accurately characterize the coupling relationship between biological, physical, and chemical processes, resulting in large errors between the simulation results and the actual situation, and unstable prediction performance.

[0005] While some machine learning-based red tide prediction algorithms, such as support vector machines and random forests, have improved prediction accuracy to a certain extent, they suffer from significant drawbacks when processing high-dimensional, complex ocean monitoring data. For example, support vector machines (SVMs) inherently rely on the linear separability of data in feature space. When faced with complex, nonlinear data generated by the coupling of multiple variables such as salinity, temperature, and plankton concentration in the ocean environment, they must be mapped into a high-dimensional space using kernel functions. This not only increases computational complexity but also easily leads to the "curse of dimensionality." The random forest algorithm improves generalization by integrating multiple decision trees. However, the inherent greedy splitting strategy of tree models makes it difficult to effectively capture long-range dependencies between various factors involved in red tide formation when learning from time series data. For example, traditional models can only detect superficial, local correlations between a plankton outbreak and abnormal water temperature fluctuations and tidal changes from a week prior.

[0006] In summary, existing red tide prediction technologies suffer from low prediction accuracy, poor universality, high computational cost, and an inability to fully exploit data features. The red tide prediction method proposed in this paper, based on time interval semantic extraction and low-rank time-frequency transformers, aims to mine the deep semantics of data through time interval semantic extraction technology, reduce data dimensionality using low-rank methods, and capture long-range dependencies in combination with the Transformer architecture. This addresses the problems of insufficient data processing, weak model adaptability, and low prediction efficiency in existing technologies, achieving high-precision, efficient, and real-time prediction of red tides. Summary of the Invention

[0007] In order to solve the above-mentioned problems, the present invention provides a red tide prediction method and system based on time interval and low-rank time-frequency Trasformer.

[0008] In the first aspect, the present invention provides a red tide prediction method based on time interval and low-rank time-frequency Trasformer, which adopts the following technical solutions:

[0009] A red tide prediction method based on time interval and low-rank time-frequency Trasformer, including:

[0010] Obtain multi-source data on marine red tides;

[0011] Perform data preprocessing on the acquired marine red tide multi-source data;

[0012] Use multi-scale convolutional neural networks to extract and fuse features from multi-source data of marine red tides;

[0013] Decompose and reconstruct the fused multi-scale features based on the wavelet transform decoupling mechanism;

[0014] Perform time interval semantic extraction on the reconstructed features and use the low-rank attention layer to extract cross-channel dependencies;

[0015] Output the prediction results.

[0016] Furthermore, the obtained marine red tide multi-source data is preprocessed, including time alignment and missing value processing of the original data collected by the sensor, and a method combining linear interpolation and sliding average is used to uniformly sample all data to the same time interval to construct a multi-feature time series matrix. ,in H represents the time step, V is the feature dimension; for missing values, the weighted average of the data of adjacent time points is used to fill in the missing values, and the weight is determined according to the distance of the time interval; the Z-score standardization method is used to map each feature data to a standard normal distribution space with a mean of 0 and a variance of 1, and the standardized data is nonlinearly transformed. For some features that show exponential growth or have a long-tail distribution, the natural logarithm transformation is used, which is expressed as:

[0017] ,

[0018] in is the log-transformed data matrix.

[0019] Furthermore, the multi-scale convolutional neural network is used to extract and fuse features from multi-source data of marine red tide, including inputting features in each bottleneck block. , the number of channels is C , after a 1×1 convolutional layer, it is then divided into B branches ,in ,then, B The branch features are recursively fed into their respective 3×3 convolutional layers and added to the output of the previous branch, expressed as:

[0020] ,

[0021] Feature Collection The receptive field of each feature in the Aggregate information from previous branches; the receptive field of the convolutional layer output depends on the convolution kernel size k , step length s And the input receptive field ,Right now The final features are split into multiple branches and s = 1 for processing; the features are concatenated and fused through a 1×1 convolutional layer to generate the output features:

[0022]

[0023] in, Added to the output features via shortcut connections.

[0024] Furthermore, the wavelet transform decoupling mechanism is used to decompose and reconstruct the fused multi-scale features, including accurately separating and strengthening the high-frequency short-term fluctuation information and low-frequency long-term evolution trend in the data by decomposing and reconstructing the multi-scale features, so as to improve the multi-scale convolutional neural network's ability to represent time patterns, wherein the wavelet transform is used In the multi-scale convolutional neural network module B The feature set output by each branch Decoupling into low-frequency components With high frequency components :

[0025]

[0026] in w Indicates the number of decomposition layers, followed by inverse wavelet transform Building a short-term model and long-term patterns , expressed as:

[0027]

[0028] in, The operation generates features that match the input dimensions but are padded with zero values.

[0029] Furthermore, the time interval semantic extraction of the reconstructed features includes introducing a frequency domain representation method of learnable frequency, using generalized DCT to calculate customized frequency domain components, and generalizing DCT to a form with customized frequency to realize frequency learning, which is expressed as:

[0030] ,

[0031]

[0032] in, is a set of custom frequency coefficients, ,and ,set up To preserve the mean, generalize DCT to a length of N The time series mapping is frequency domain components, suitable for obtaining compact data representation in time series forecasting.

[0033] Furthermore, the use of a low-rank attention layer to extract cross-channel dependencies includes using a low-rank attention layer to perform lightweight attention calculations in the channel dimension to generate a low-rank correction to the output of the CI Transformer encoder, wherein the cross-channel sequence is first mapped to a low-dimensional representation and then linearly projected back to the original space to generate an update; secondly, a time-frequency independent TFI setting is adopted to share the correction amount in the time-frequency dimension, and the resource-intensive multi-head self-attention MSA in the channel-dimensional Transformer encoder is replaced with a lightweight multi-head self-attention LMSA, where the LMSA represents the standard MSA as:

[0034]

[0035] in , , LMSA is defined as:

[0036] ,

[0037]

[0038] The input 、 , the learnable matrix 、 , h For the number of heads, is the model width.

[0039] Furthermore, the method of extracting cross-channel dependencies using the low-rank attention layer also includes obtaining the CI representation of the low-rank attention input by the Transformer encoder, which is recorded as ,in and are the lengths of time domain and frequency domain representations, respectively, according to TFI settings, Reshaped into , at this time the low-rank attention is expressed as:

[0040] ,

[0041] in, is a learnable query vector, whose length ; is a compact learnable position embedding; is a learnable projection matrix; By repeating along the time-frequency dimension, Map to , and then compare it with Add.

[0042] Furthermore, the output prediction results include compressing the spatial dimension of the output feature matrix of the low-rank attention layer, aggregating the spatial information to the channel dimension through global average pooling, and obtaining the channel-level feature vector ,Right now:

[0043]

[0044] in Indicates the b Batch, c channels, spatial positions ( i , j ) at the eigenvalue, is the mean feature of the channel in the spatial dimension. If the input data contains T The time series information of time steps is G Expanded to time series feature sequence , and input gated recurrent units to capture temporal dependencies. The hidden state update process of GRU is:

[0045]

[0046] in for t The hidden state of the moment, is the hidden layer dimension, is the hidden state of the previous time step, GRU updates the gate and reset gate Control the retention and forgetting of historical information, specifically:

[0047]

[0048] in is the sigmoid activation function, represents element-by-element multiplication, [,;,] is a vector concatenation operation, , , is the weight matrix, , , is the bias vector.

[0049] Furthermore, the output prediction result also includes extracting the hidden state of the last time step As a time series feature representation, it is input into a multi-layer perceptron for nonlinear transformation. The MLP contains two hidden layers, and the batch normalization and dropout mechanisms are introduced to improve the generalization ability of the model, which is expressed as:

[0050]

[0051] in 、 is the weight matrix, 、 is the bias vector, , is the hidden layer dimension, p is the Dropout inactivation rate, ReLU is the linear rectification activation function, and finally the features are mapped to the red tide prediction target space through the output layer. The output layer is a fully connected layer, and the calculation formula is:

[0052]

[0053] in To predict the results, V is the feature dimension; is the output layer weight matrix, is the output bias vector.

[0054] The second aspect is a red tide prediction system based on time intervals and low-rank time-frequency Trasformer, including:

[0055] The data acquisition module is configured to acquire multi-source data of marine red tides;

[0056] The preprocessing module is configured to perform data preprocessing on the acquired marine red tide multi-source data;

[0057] The feature module is configured to use a multi-scale convolutional neural network to perform feature extraction and feature fusion on multi-source data of marine red tide;

[0058] The reconstruction module is configured to decompose and reconstruct the fused multi-scale features based on the wavelet transform decoupling mechanism;

[0059] The extraction module is configured to perform time interval semantic extraction on the reconstructed features and extract cross-channel dependencies using a low-rank attention layer;

[0060] The prediction module is configured to output a prediction result.

[0061] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a red tide prediction method based on time intervals and low-rank time-frequency Trasformer.

[0062] In a fourth aspect, the present invention provides a terminal device comprising a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor. A red tide prediction method based on time intervals and low-rank time-frequency Trasformer.

[0063] In summary, the present invention has the following beneficial technical effects:

[0064] In terms of improving the universality of the model, the traditional prediction model based on empirical formulas is limited by the historical data and environmental parameters of a specific sea area, and is difficult to adapt to different hydrological conditions and differences in biological community structure. The present invention uses time interval semantic extraction technology to convert the implicit ecological semantic features in the ocean monitoring data (such as the duration of water temperature anomalies, the trend of changes in nutrient concentrations, etc.) into structured semantic vectors. Combined with the cross-domain feature learning ability of the low-rank time-frequency Transformer model, it can effectively capture the common laws of red tide occurrence in different sea areas and reduce dependence on data from a single region. After actual measurement verification in multiple sea areas, the average absolute error of the model in cross-regional prediction scenarios is reduced by more than 30% compared with traditional methods, which significantly improves the adaptability to different marine environments and solves the limitation of "regional exclusivity" of traditional models.

[0065] To address the high computational cost and insufficient accuracy of numerical simulation models, the present invention utilizes low-rank matrix decomposition technology to reduce the dimensionality of high-dimensional ocean data (e.g., monitoring parameters with over 50 dimensions, including water temperature, salinity, dissolved oxygen, etc.). While retaining core feature information, the data dimensions are compressed to 1 / 5 to 1 / 3 of the original, significantly reducing data processing and storage requirements. Simultaneously, the Transformer architecture's multi-head attention mechanism can efficiently capture the long-range dynamic correlations between multiple factors during red tides (e.g., the co-evolutionary relationship between phytoplankton growth and light and ocean current velocity), avoiding the prediction biases of traditional numerical models caused by simplified ecological coupling relationships. Compared to numerical simulation methods that rely on high-precision initial conditions, the present invention reduces model training costs by over 60%, and improves prediction stability by 40% in complex ocean environments, achieving dual optimization of computational efficiency and prediction accuracy.

[0066] To overcome the data processing bottlenecks of traditional machine learning, this paper addresses the dimensionality curse and inadequate capture of long-range dependencies in high-dimensional time series data. By using a temporal semantic extraction module, this method transforms discrete monitoring time series into ecologically meaningful semantic units (e.g., "water temperature exceeds 25°C for three consecutive days" or "salinity weekly variation exceeds a threshold"), providing the model with more easily understandable structured input. Low-rank dimensionality reduction techniques further eliminate data redundancy, reducing the model parameter size by over 50% and mitigating the risk of overfitting. The self-attention mechanism of the Transformer architecture effectively models long-range dependencies across time steps (e.g., capturing the correlation between abnormally elevated nutrient concentrations and subsequent red tide outbreaks 10 days in advance). Compared to traditional algorithms such as support vector machines and random forests, this method improves feature extraction for long-term red tide incubation processes by 70%, significantly enhancing its ability to analyze complex data patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 3 is a schematic diagram of a red tide prediction method based on time interval and low-rank time-frequency Trasformer according to Example 1 of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described in detail below with reference to the accompanying drawings.

[0069] Example 1

[0070] Reference Figure 1 In this embodiment, a red tide prediction method based on time interval and low-rank time-frequency Trasformer includes:

[0071] S1. Data input and preprocessing,

[0072] In research on red tide prediction based on multi-source data, high-dimensional, complex data collected from the Taizhou waters and Ala Wai Harbor encompass a wide range of marine environmental information, including but not limited to chlorophyll-a concentration, water temperature, salinity, dissolved oxygen content, pH, nutrient concentrations (such as nitrate and phosphate), and meteorological parameters (wind speed, light intensity, and rainfall). These data reflect the ecological and environmental conditions for red tide occurrence from various perspectives. Due to the diverse data sources, different dimensions, and complex nonlinear relationships, directly using raw data results in inefficient model training and insufficient prediction accuracy. Therefore, a systematic data preprocessing process is necessary.

[0073] First, the raw data collected by various sensors are time-aligned and missing values ​​are processed. Since the collection frequency and start time of different types of data vary, a method combining linear interpolation and sliding average is used to uniformly sample all data to the same time interval (such as hourly level) to construct a multi-feature time series matrix. ,in H represents the time step, V It is a feature dimension, including the above-mentioned marine environmental parameters. For missing values, the weighted average of the data of adjacent time points is used to fill in the missing values. The weight is determined according to the time interval. The formula is as follows:

[0074] (1)

[0075] in, The first i Time point j dimensional eigenvalues, p and q Respectively represent the number of time points selected forward and backward, For the k Time point relative toi The closer the distance, the greater the weight.

[0076] Then, data normalization is performed to eliminate the dimension effect. Using the Z-score normalization method, each feature data is mapped to a standard normal distribution space with a mean of 0 and a variance of 1. The formula is:

[0077] (2)

[0078] in, is the standardized data matrix, Indicates the i Time point j Normalized values ​​of dimensional features, and Respectively j This operation ensures that all features have equal importance in model training and prevents features with larger values ​​from dominating the model learning process.

[0079] To further enhance the expressive power of data features, the standardized data is transformed nonlinearly. For some features that show exponential growth or long-tail distribution (such as the change in nutrient concentration during red tide outbreaks), natural logarithm transformation is used:

[0080] (3)

[0081] in This is the logarithmically transformed data matrix. This step compresses the dynamic range of the data, effectively reduces data dimensionality, removes noise, and extracts core features closely related to the occurrence and development of red tides. This provides high-quality data input for subsequent time interval semantic extraction and low-rank time-frequency Transformer model analysis. This reduces the impact of outliers while highlighting subtle differences in data changes, making the data distribution more consistent with model training requirements.

[0082] S2. Multi-scale convolutional neural network,

[0083] The goal of red tide prediction is to base on historical red tide observation data. , predicting the future T Red tide occurrence in time steps This paper introduces a novel multi-scale convolutional neural network architecture for learning multi-scale features from input time series, which can capture both short-term and long-term patterns.

[0084] Red tide monitoring data has complex temporal characteristics, and the short-term dynamics and long-term trends it contains are both key elements for accurate prediction. The former reflects local fluctuations and periodic changes, while the latter contains macro-trends reflecting ecological evolution and predictive information for future trends. Traditional convolutional neural networks (CNNs) are limited by the design of convolutional layers with fixed receptive fields. Their output features can only cover a limited time range, resulting in insufficient ability to capture single temporal patterns. In view of this, the present invention proposes a novel multi-scale convolutional neural network architecture based on bottleneck block stacking. The core of this architecture is that each bottleneck block implements multi-scale feature learning through parallel branches: the small receptive field branches focus on the details of the periodic fluctuations in the data, while the large receptive field branches focus on extracting macro-features that represent the overall trend, effectively addressing the limitations of traditional models in capturing multi-dimensional temporal patterns.

[0085] In each bottleneck block, the input features (The number of channels is C ) first passes through a 1×1 convolutional layer and is then split into B branches ,in .then, B The branch features are recursively fed into their respective 3×3 convolutional layers and added to the output of the previous branch ( Except as follows:

[0086] (4)

[0087] Feature Collection The receptive field of each feature in increases successively, because each (like i >1) will aggregate information from previous branches.

[0088] In convolutional neural networks, the receptive field refers to the input region associated with each output in a convolutional layer. The size of the receptive field determines the range of inputs observed by the network during the convolution operation. Generally speaking, the receptive field of a convolutional layer output depends on the size of the convolution kernel. k , step length s And the input receptive field , and its calculation formula is:

[0089] (5)

[0090] In the multi-scale convolutional neural network module, features are divided into multiple branches and s = 1 is processed by a parallel 3×3 convolution layer. Given by Equation (4) B The feature set obtained by the branch , we set the receptive field of the input feature Rin =1 and the number of branches B =4, the receptive fields of the multi-branch features are [3, 5, 7, 9], respectively. Specifically, the small receptive field branch focuses on extracting the periodic fluctuation details in the short-term pattern, while the large receptive field branch focuses on capturing the overall trend characteristics implied by the long-term pattern. Through this differentiated multi-scale feature learning mechanism, the convolutional neural network designed in this paper can effectively integrate the dual temporal pattern characteristics of short-term dynamics and long-term trends, achieving comprehensive capture of the multi-dimensional temporal characteristics of red tide monitoring data.

[0091] Finally, these features are concatenated and fused through a 1×1 convolutional layer to generate the output features :

[0092] (6)

[0093] in, is added to the output features via shortcut connections. Subsequently, It is input into the subsequent multi-scale convolutional neural network module to generate multi-scale features for LLM. .

[0094] S3. Temporal mode decoupling,

[0095] The high-frequency and low-frequency components of marine red tide monitoring data correspond to key short-term and long-term temporal patterns, respectively. To further explore the temporal characteristics of this data, this paper proposes an innovative pattern decoupling mechanism based on wavelet transforms. This mechanism decomposes and reconstructs multi-scale features, accurately separating and enhancing high-frequency short-term fluctuations and low-frequency long-term evolution trends in the data. This significantly enhances the ability of multi-scale convolutional neural networks to represent temporal patterns.

[0096] First, we use wavelet transform In the multi-scale convolutional neural network module B The feature set output by each branch Decoupling into low-frequency components With high frequency components :

[0097] (7)

[0098] in w represents the number of decomposition layers. Then, through inverse wavelet transform Building a short-term model and long-term patterns :

[0099] (8)

[0100] in, The operation generates features that match the input dimensions but are padded with zero values.

[0101] S4. Temporal pattern reconstruction,

[0102] For feature sets The receptive fields of the features in the local-to-global / global-to-local assembly process will expand in turn. Features with smaller receptive fields can more effectively capture local periodic fluctuations, while features with larger receptive fields focus on broader global trends. Therefore, in the local-to-global / global-to-local assembly process, short-term / long-term patterns The following enhancements have been made:

[0103] (9)

[0104] After assembly is complete, The features in are reconstructed by combining short-term and long-term patterns:

[0105] (10)

[0106] Reconstructed features It has both high-frequency fluctuation sensitivity and low-frequency trend foresight, and is used in red tide early warning tasks. It can capture the short-term abnormal signal of chlorophyll concentration suddenly increasing by 0.5μg / L within 6 hours, and at the same time The fused historical data indicates that this period coincided with the passage of the local spring warm current (a long-term trend), enabling a comprehensive assessment of the probability of a red tide outbreak. This feature reconstruction mechanism effectively addresses the feature fragmentation problem of traditional single-scale models when processing complex ocean time series data. It avoids false alarms caused by focusing solely on short-term fluctuations and overcomes the warning lag associated with single-scale long-term trend analysis.

[0107] S5. Time interval semantic extraction,

[0108] Large language models are pre-trained on massive amounts of semantically rich text data, where each word has a clear meaning. In contrast, red tide detection data is semantically sparse, requiring the entire sequence to convey specific content. Therefore, LLMs pre-trained on text data struggle to accurately interpret the semantics of time series data. The present invention utilizes a low-rank Transformer module with an encoder-decoder architecture to extract semantic information from time series data for LLMs.

[0109] First, input red tide monitoring data Divided into P fragments ,in , LRepresents the length of the segment. Randomly mask about 75\% of the segments. During training, retaining all frequency components may lead to overfitting because many high-frequency changes are caused by noise. At the same time, it is also crucial to use frequency domain sparsity to reduce computational and memory complexity. Therefore, a key issue of the frequency domain transformer is how to select a part of the frequency components to represent temporal semantics. Unlike retaining only low-frequency components, the present invention introduces a frequency domain representation method that can learn frequencies. A customized discrete cosine transform (generalized DCT) is used to calculate customized frequency domain components to achieve frequency learning. Generalized DCT is a generalized form of discrete cosine transform (DCT), which represents sequences by superposition of real-valued cosine functions of different frequencies. Compared with the discrete Fourier transform (DFT) using a complex exponential function, DCT can roughly halve the length of the real sequence and has a strong energy compression characteristic. Therefore, it is more popular in the field of compression and is particularly suitable for representing sequences with a small number of frequency domain components. A commonly used form of DCT is:

[0110] (11)

[0111] (12)

[0112] matrix are orthogonal, so the inverse transform is:

[0113] (13)

[0114] In order to take advantage of sparsity, some unimportant frequency domain components in DCT can be ignored. The red tide monitoring data F is transformed into the frequency domain by multiplying a set of orthogonal cosine function bases, the frequencies of which are uniformly distributed in the frequency domain. k However, these specific frequencies may not be sufficient to represent certain real-world phenomena. For example, when the sampling rate is 1, multiple frequency domain components are required to represent a simple function cos(1.1πt / N), but there is actually only one frequency domain component.

[0115] In order to further improve the representation capability while fixing a small number of frequency domain components, the present invention generalizes DCT to a form with custom frequencies to achieve this goal:

[0116] (14)

[0117] (15)

[0118] in, is a set of custom frequency coefficients, ,and We set To preserve the mean. Generalize DCT to lengthN The time series mapping is Frequency domain (FD) components, which are suitable for obtaining compact data representation in time series forecasting. Considered as a constant, the complexity of the generalized DCT is O ( N However, unlike the DCT, the generalized DCT basis is usually not orthogonal and has no intuitive inverse transform. Thanks to modern deep learning frameworks, it is convenient to recover the time domain (TD) sequence from the sparse frequency domain representation obtained by the generalized DCT by learning the projection matrix.

[0119] By As a learnable parameter, the generalized DCT supports adaptive learning of frequencies. This flexibility allows the generalized DCT to adjust frequencies to better approximate the most significant frequency components in the data, which may not align with the uniform grid points of the DCT. In the implementation, The initialization is based on the high energy frequencies obtained by applying DCT to a randomly sampled subset of the dataset. Although the time complexity of generalizing DCT based on DCT frequency initialization is O ( LlogL ), but this initialization phase is relatively fast, especially compared to training time. Because time series datasets are typically smaller than those in CV and NLP, in many practical scenarios, it is acceptable to perform the DCT on the entire dataset without sampling. Therefore, this short initialization phase can be loosely ignored when analyzing overall complexity.

[0120] S6. Low-rank attention layer extracts cross-channel dependencies,

[0121] There are two intuitive approaches to capturing cross-channel dependencies. The first embeds data points from all channels at the same time step into a unified feature vector, while the second applies the Transformer architecture along the channel dimension. By replacing the self-attention mechanism with two small-scale attentions, the computational and memory overhead of the channel-dimensional Transformer is reduced. However, empirical studies have shown that these methods produce larger errors than the channel-independent (CI) PatchTST.

[0122] To capture cross-channel dependencies with less redundancy, we use a low-rank attention layer. Low-rank attention is a computationally efficient method for incorporating cross-channel information into CI modeling. It performs lightweight attention computation in the channel dimension, generating low-rank modifications to the output of the CI Transformer encoder.

[0123] Low-rank attention aims to further improve the accuracy of CI models, which already have high-precision temporal dependencies. This process is prone to overfitting, which low-rank attention mitigates by constraining updates to a low-rank space in two ways: first, mapping the cross-channel sequence to a low-dimensional representation, then generating updates through linear projection back to the original space; second, adopting a time-frequency independence (TFI) setting so that the corrections are shared across the time and frequency dimensions. Furthermore, the number of parameters is reduced by simplifying the attention mechanism and moving the position embedding from the input space to the low-dimensional representation space.

[0124] The resource-intensive multi-head self-attention (MSA) in the channel-wise Transformer encoder is replaced with lightweight multi-head self-attention (LMSA). LMSA aggregates information from each channel into a low-rank space using short learnable queries and adds a compact learnable position embedding at the output to represent the positional relationship in the low-rank space. The result is mapped to the number of channels via linear projection, the aggregated information is assigned to each channel, and the result is replicated in the time-frequency dimension according to the TFI setting. The subsequent residual connections, layer normalization (LayerNorm), and multi-layer feedforward network (MLP) processing methods are consistent with the standard Transformer encoder.

[0125] LMSA (Linear Multi-Head Self-Attention) is a simplified version of MSA with reduced computational requirements and number of parameters. In LMSA, keys and values ​​share the same linear projection. Since it is used with learnable queries, the linear projection of the query is also omitted. Standard MSA is represented as:

[0126] (16)

[0127] in , LMSA is defined as:

[0128] (17)

[0129] (18)

[0130] The input 、 , the learnable matrix 、 . h For the number of heads, is the model width.

[0131] The input of the low-rank attention is the CI representation obtained by the Transformer encoder, denoted as ,in and are the lengths of time domain and frequency domain representation respectively. According to TFI settings, Reshaped into . At this time, the low-rank attention can be expressed as:

[0132] (19)

[0133] in, is a learnable query vector, whose length ; is a compact learnable position embedding; is a learnable projection matrix; By repeating along the time-frequency dimension, Map to , and then compare it with Add.

[0134] Due to its low-rank property, low-rank attention has high computational efficiency. D The complexity is , which may lead to significant computational overhead, but low-rank attention reduces the complexity to .when When considered as a constant, the complexity can be approximated as O ( D ).

[0135] S6. Output of prediction results,

[0136] After the low-rank attention layer completes the extraction of cross-channel dependencies, the output feature matrix , where B represents the batch size, C is the number of feature channels, and H and W represent the spatial height and width of the feature map, respectively. Considering that the red tide prediction task requires integrating spatiotemporal features and capturing temporal dynamics, the subsequent processing flow revolves around feature dimensionality reduction, temporal modeling, and prediction mapping.

[0137] First of all, Perform spatial dimension compression and aggregate spatial information into channel dimension through global average pooling (GAP) to obtain channel-level feature vector , and its calculation formula is:

[0138] (20)

[0139] in Indicates the b Batch, c channels, spatial positions ( i , j ) at the eigenvalue, is the mean feature of the channel in the spatial dimension.

[0140] If the input data contains T Time series information of time steps (such as ocean monitoring data for several consecutive days), then G Expanded to time series feature sequence , and inputs a Gated Recurrent Unit (GRU) to capture temporal dependencies. The hidden state update process of the GRU is as follows:

[0141] (twenty one)

[0142] in for t The hidden state of the moment, is the hidden layer dimension, is the hidden state of the previous time step (initial state Set to zero vector). GRU updates the gate and reset gate Control the retention and forgetting of historical information. The specific calculation is:

[0143] (twenty two)

[0144] in is the sigmoid activation function, represents element-by-element multiplication, [,;,] is a vector concatenation operation, , , is the weight matrix, , , is the bias vector.

[0145] Extract the hidden state of the last time step As a time series feature representation, it is input into a multi-layer perceptron (MLP) for nonlinear transformation. The MLP contains two hidden layers and introduces batch normalization (BN) and dropout mechanisms to improve the model's generalization ability. The calculation process is as follows:

[0146] (twenty three)

[0147] in 、 is the weight matrix, 、 is the bias vector, , is the hidden layer dimension, p is the Dropout inactivation rate, and ReLU is the linear rectification activation function.

[0148] Finally, the features are mapped to the red tide prediction target space through the output layer. The output layer is a fully connected layer, and the calculation formula is:

[0149] (twenty four)

[0150] in To predict the results, V It is a characteristic dimension, including the above-mentioned multiple marine environmental parameters; is the output layer weight matrix, The output bias vector is y. This mapping process combines the cross-channel dependency features extracted by the low-rank attention layer with the temporal dynamic features captured by the GRU, ultimately generating a continuous value or categorical prediction result Y suitable for red tide prediction, providing data-driven decision support for marine ecological monitoring.

[0151] Experimental verification:

[0152] Table 1 Comparison of monitoring effects

[0153]

[0154] Experimental results demonstrate that the proposed model maintains a consistent lead across diverse data distributions (globally applicable HAEDAT vs. regionally specific CASEarth) and data scales (over 100,000 HAEDAT samples vs. 12,000 samples in the core region of CASEarth), validating the invention's technical advantages of high universality and robustness. Specifically, on the CASEarth dataset, the model's prediction accuracy for the "red tide impact level" (89.3% match rate) improved by 25% compared to traditional methods, providing a reliable basis for marine forecasting agencies to implement graded warnings.

[0155] In summary, the dataset comparison experiment proves from a quantitative perspective that the present invention systematically solves the bottlenecks of traditional models in precision, recall and cross-domain adaptability through the collaborative design of "temporal semantic extraction + low-rank dimensionality reduction + Transformer architecture". Its performance advantages are closely aligned with the technical solution of the invention, providing solid experimental support for the implementation of the technology.

[0156] Example 2

[0157] This embodiment provides a red tide prediction system based on time intervals and low-rank time-frequency Trasformer, including:

[0158] The data acquisition module is configured as follows:

[0159] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a red tide prediction method based on time intervals and low-rank time-frequency Trasformer.

[0160] A terminal device includes a processor and a computer-readable storage medium, the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor. A red tide prediction method based on time intervals and low-rank time-frequency trasformer is described.

[0161] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A red tide prediction method based on time interval and low-rank time-frequency Trasformer, characterized in that: include: Obtain multi-source data on marine red tides; Perform data preprocessing on the acquired marine red tide multi-source data; Use multi-scale convolutional neural networks to extract and fuse features from multi-source data of marine red tides; Decompose and reconstruct the fused multi-scale features based on the wavelet transform decoupling mechanism; Perform time interval semantic extraction on the reconstructed features and use the low-rank attention layer to extract cross-channel dependencies; Output prediction results; The multi-scale convolutional neural network is used to extract and fuse features from multi-source data of marine red tide, including inputting features in each bottleneck block. , the number of channels is C , after a 1×1 convolutional layer, it is then divided into B branches ,in ,then, B The branch features are recursively fed into their respective 3×3 convolutional layers and added to the output of the previous branch, expressed as: , feature set The receptive field of each feature in the Aggregate information from previous branches; The receptive field of the convolutional layer output depends on the convolution kernel size k , step length s And the input receptive field ,Right now , the final features are divided into multiple branches and s = 1 for parallel 3×3 convolutional layers; The features are concatenated and fused through a 1×1 convolutional layer to generate output features. in, Added to the output features via shortcut connections; The decomposition and reconstruction of the fused multi-scale features based on the wavelet transform decoupling mechanism includes decomposing and reconstructing the multi-scale features, accurately separating and strengthening the high-frequency short-term fluctuation information and low-frequency long-term evolution trend in the data, so as to improve the multi-scale convolutional neural network's ability to represent time patterns, wherein the wavelet transform is used In the multi-scale convolutional neural network module B The feature set output by each branch Decoupling into low-frequency components With high frequency components : in w Indicates the number of decomposition layers, followed by inverse wavelet transform Building a short-term model and long-term patterns , expressed as: in, The operation generates features that match the input dimensions but are filled with zero values; The time interval semantic extraction of the reconstructed features includes introducing a frequency domain representation method for learnable frequencies, using generalized DCT to calculate customized frequency domain components, and generalizing DCT to a form with customized frequencies to achieve frequency learning, which is expressed as: ， in, is a set of custom frequency coefficients, ,and ,set up To preserve the mean, generalize DCT to a length of N The time series mapping is frequency domain components, suitable for obtaining compact data representation in time series forecasting.

2. The red tide prediction method based on time interval and low-rank time-frequency Trasformer according to claim 1 is characterized in that: The data preprocessing of the acquired marine red tide multi-source data includes time alignment and missing value processing of the original data collected by the sensor, and the use of a method combining linear interpolation and sliding average to uniformly sample all data to the same time interval to construct a multi-feature time series matrix. ,in H represents the time step, V is the feature dimension; for missing values, the weighted average of the data of adjacent time points is used to fill in the missing values, and the weight is determined according to the distance of the time interval; the Z-score standardization method is used to map each feature data to a standard normal distribution space with a mean of 0 and a variance of 1, and the standardized data is nonlinearly transformed. For some features that show exponential growth or have a long-tail distribution, the natural logarithm transformation is used, which is expressed as: ， in is the log-transformed data matrix.

3. The red tide prediction method based on time interval and low-rank time-frequency Trasformer according to claim 2 is characterized in that: The method uses a low-rank attention layer to extract cross-channel dependencies, including using the low-rank attention layer to perform lightweight attention calculations in the channel dimension to generate low-rank corrections to the output of the CI Transformer encoder, wherein the cross-channel sequence is first mapped to a low-dimensional representation and then linearly projected back to the original space to generate updates; secondly, a time-frequency independent TFI setting is adopted to share the correction amount in the time-frequency dimension, replacing the resource-intensive multi-head self-attention MSA in the channel-dimensional Transformer encoder with a lightweight multi-head self-attention LMSA, wherein the LMSA represents the standard MSA as: in , , LMSA is defined as: ， The input 、 , the learnable matrix 、 , h For the number of heads, is the model width.

4. The red tide prediction method based on time interval and low-rank time-frequency Trasformer according to claim 3 is characterized in that: The method of extracting cross-channel dependencies using a low-rank attention layer also includes obtaining the CI representation of the low-rank attention input from the Transformer encoder, which is denoted as ,in and are the lengths of time domain and frequency domain representations, respectively, according to TFI settings, Reshaped into , at this time the low-rank attention is expressed as: in, is a learnable query vector, whose length ; is a compact learnable position embedding; is a learnable projection matrix; By repeating along the time-frequency dimension, Map to , and then compare it with Add.

5. The red tide prediction method based on time interval and low-rank time-frequency Trasformer according to claim 4 is characterized in that: The output prediction results include compressing the spatial dimension of the output feature matrix of the low-rank attention layer, aggregating the spatial information to the channel dimension through global average pooling, and obtaining the channel-level feature vector ,Right now: in Indicates the b Batch, c channels, spatial positions ( i , j ) at the eigenvalue, is the mean feature of the channel in the spatial dimension. If the input data contains T The time series information of time steps is G Expanded to time series feature sequence , and input the gated recurrent unit to capture the temporal dependency, the hidden state update process of GRU: in for t The hidden state of the moment, is the hidden layer dimension, is the hidden state of the previous time step, GRU updates the gate and reset gate Control the retention and forgetting of historical information, specifically: ， in is the sigmoid activation function, represents element-by-element multiplication, [,;,] is a vector concatenation operation, , , is the weight matrix, , , is the bias vector.

6. The red tide prediction method based on time interval and low-rank time-frequency Trasformer according to claim 5 is characterized in that: The output prediction result also includes extracting the hidden state of the last time step As a time series feature representation, it is input into a multi-layer perceptron for nonlinear transformation. The MLP contains two hidden layers. Batch normalization and Dropout mechanisms are introduced to improve the generalization ability of the model, which is expressed as: in 、 is the weight matrix, 、 is the bias vector, , is the hidden layer dimension, p is the Dropout inactivation rate, ReLU is the linear rectification activation function, and finally the features are mapped to the red tide prediction target space through the output layer. The output layer is a fully connected layer, and the calculation formula is: in To predict the results, V is the feature dimension; is the output layer weight matrix, is the output bias vector.

7. A red tide prediction system based on time interval and low-rank time-frequency Trasformer, executing a red tide prediction method based on time interval and low-rank time-frequency Trasformer according to any one of claims 1 to 6, characterized in that: include: The data acquisition module is configured to acquire multi-source data of marine red tides; The preprocessing module is configured to perform data preprocessing on the acquired marine red tide multi-source data; The feature module is configured to use a multi-scale convolutional neural network to perform feature extraction and feature fusion on multi-source data of marine red tide; The reconstruction module is configured to decompose and reconstruct the fused multi-scale features based on the wavelet transform decoupling mechanism; The extraction module is configured to perform time interval semantic extraction on the reconstructed features and extract cross-channel dependencies using a low-rank attention layer; The prediction module is configured to output a prediction result.

Images