Time sequence prediction method and system based on local clustering
By optimizing the cluster centers through local clustering method and dual-constraint loss function, the problem of insufficient utilization of historical information in time series data prediction is solved, and the prediction accuracy and operation efficiency are improved.
Patent Information
- Application Number
- CN202510627650.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2025-09-16
AI Technical Summary
Existing time series data prediction methods find it difficult to effectively combine historical information, resulting in limited prediction accuracy. In addition, expanding the input sequence length will increase memory space and running time, affecting the timeliness and effectiveness of decision-making.
A local clustering method is used for dimensionality reduction. By constructing a dual-constrained clustering loss function, the cluster centers are optimized on the entire data set to obtain the hidden layer representation, retain the long-term dependencies of time series data, and generate future trend modeling.
It improves the operating efficiency in long time series scenarios, achieves more accurate future trend modeling, and reduces the requirements for memory space and running time.
Smart Images
Figure CN120654869A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of time series data prediction, and in particular to a time series prediction method and system based on local clustering. Background Art
[0002] Time series data is widely present in scenarios such as meteorology, finance, and transportation. Accurately forecasting this data is crucial for effective decision-making. Due to the enormous scale of time series data, forecasting methods for this data typically infer future trends based on information within a local window of the most recent period. Given that future data trends are closely related to historical data, the time series data input to these forecasting methods typically only covers recent data, making it difficult to incorporate historical information that is critical to the forecast, limiting forecast accuracy. Increasing the input time series window and incorporating more historical data into the forecasting model can significantly improve forecast accuracy. However, as the length of the input sequence increases, the memory space required and the runtime of the forecasting model increase, impacting the timeliness and effectiveness of decision-making. Summary of the Invention
[0003] The present invention aims to provide a time series prediction method and system based on local clustering. This method uses local clustering to reduce the dimensionality of input data, eliminating the need to directly process the original high-dimensional time series data and improving operational efficiency in long time series scenarios. Cluster centers jointly optimize a dual clustering loss function across the entire dataset, ensuring that the reduced hidden layer representations are deeply aligned with the inherent evolutionary patterns of the time series data. Compared to dimensionality reduction methods that rely solely on local statistical properties, this method effectively preserves the long-term dependencies of time series data through a global perception mechanism, ultimately achieving more accurate modeling of future trends.
[0004] The present invention is achieved through the following technical solutions:
[0005] Time series prediction methods based on local clustering include:
[0006] Step S1, performing segmented preprocessing on the collected time series data to obtain several clusters;
[0007] Step S2, constructing a clustering loss function containing dual constraints;
[0008] Step S3: Based on the clustering loss function, cluster training is performed to obtain the final k cluster centers.
[0009] Step S4, obtaining the hidden layer representation of the input time series based on the final k cluster centers;
[0010] Step S5: Generate future prediction results based on the hidden layer representation.
[0011] Optionally, before step S1, the method further includes:
[0012] The collected time series data are stored in a time series database in chronological order; wherein the time series data includes N variables, and each variable corresponds to an observation value of T time steps.
[0013] Optionally, in step S1, the collected time series data is pre-processed in segments to obtain several clusters, including:
[0014] Set a segmentation window p with a fixed time length, divide each variable into n segments; then divide the n segments into several clusters, where each cluster includes data corresponding to several segments.
[0015] Optionally, in step S1, a segmentation window p with a fixed time length is set to divide each variable into n segments, including:
[0016] If the number of time steps T is divisible by p, then n = T / p;
[0017] If the number of time steps T is not divisible by p, then in, Indicates rounding down the result of T / p.
[0018] Optionally, in step S2, a clustering loss function containing dual constraints is constructed include:
[0019] Construct the reconstruction loss as follows (1)
[0020]
[0021] In the above formula (1), Represents clusters The mean vector, c j Represents clusters The cluster center of k represents the number of clusters; the above formula (1) describes each cluster The mean vector and cluster center c j The Euclidean distance and sum;
[0022] Construct the correlation loss as follows (2):
[0023]
[0024] In the above formula (2), Represents each cluster All time periods and cluster center c jThe degree of similarity Sum, where the similarity The Pearson correlation coefficient is used for calculation, where k represents the number of clusters. Represents clusters Medium time period the number of
[0025] Construct the clustering loss function as follows (3):
[0026]
[0027] In the above formula (3), α represents a preset weight coefficient.
[0028] Optionally, in step S3, clustering is trained based on a clustering loss function to obtain the final k cluster centers, including:
[0029] Randomly initialize k cluster centers and use the clustering loss function The definition assigns all segments to the nearest cluster center;
[0030] Then according to the clustering loss function Update cluster centers using the gradient direction propagation method;
[0031] The above two processes are iterated until the clustering loss converges to obtain the final k cluster centers.
[0032] Optionally, in step S4, obtaining a hidden layer representation of the input time series based on the final k cluster centers includes:
[0033] The input time series is divided into multiple segments according to the segmentation length l, and then a linear layer is used to obtain the initial hidden layer representation of all ends. in Includes n items;
[0034] Calculate the preliminary hidden layer representation of the final k cluster centers in Includes k items;
[0035] Processing preliminary hidden layer representations through cross-attention mechanism and the initial hidden layer representation Get the hidden layer representation of the input time series The hidden layer represents It includes N variables, each of which has n hidden layers.
[0036] Optionally, in step S5, generating future prediction results based on the hidden layer representation includes:
[0037] Use Gaussian distribution initialization to form m readout representations;
[0038] A cross-attention mechanism is performed on the n hidden layer representations and m readout representations of each variable to obtain a set of m final representations;
[0039] After flipping the n hidden layer representations of each variable, the cross-attention mechanism is calculated with the m readout representations to obtain another set of m final representations;
[0040] Using the following equations (4) and (5), according to the m final representations and the other group of m final representations, a linear layer with a gating mechanism is projected as the final prediction result That is, generating future prediction results,
[0041]
[0042] In the above formulas (4)-(5), CrossAttn() represents the cross attention mechanism calculation, represents the m readout representations, represents n hidden layer representations, Indicates that the n hidden layers are flipped, that is, The dimensions of the two matrices are swapped, represents a set of m final representations, represents another set of m final representations, β represents the preset gating weight, Indicates the intermediate quantity, Indicates that Projected via a linear layer.
[0043] Optionally, after step S5, the method further includes:
[0044] According to the average mean square error of the future prediction results, the neural network parameters involved in steps S3 and S4 are updated through the gradient back propagation method.
[0045] The time series prediction system based on local clustering includes:
[0046] The preprocessing module is used to perform segmented preprocessing on the collected time series data to obtain several clusters;
[0047] Loss function construction module, used to construct clustering loss function containing dual constraints;
[0048] Cluster training module, used to train clustering and obtain the final k cluster centers based on the clustering loss function;
[0049] Hidden layer representation acquisition module, used to obtain the hidden layer representation of the input time series based on the final k cluster centers;
[0050] The prediction result generation module is used to generate future prediction results based on the hidden layer representation.
[0051] Compared with the prior art, the present invention has the following beneficial effects:
[0052] The time series prediction method and system based on local clustering provided in this application uses local clustering to achieve dimensionality reduction of input data, eliminating the need to directly process the original high-dimensional time series data, thereby improving operational efficiency in long time series scenarios. The cluster centers jointly optimize the dual clustering loss function on the entire data set, ensuring that the hidden layer representation after dimensionality reduction deeply aligns with the inherent evolutionary laws of the time series data. Compared to dimensionality reduction methods that rely solely on local statistical characteristics, this invention effectively preserves the long-term dependencies of time series data through a global perception mechanism, ultimately achieving more accurate future trend modeling. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. Among them:
[0054] Figure 1 This is a flow chart of the time series prediction method based on local clustering provided by the present invention.
[0055] Figure 2 This is an architectural diagram of the time series prediction method based on local clustering provided by the present invention.
[0056] Figure 3 This is a schematic diagram of the structure of the time series prediction system based on local clustering provided by the present invention. DETAILED DESCRIPTION
[0057] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some, rather than all, structures related to the present application are shown in the accompanying drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0058] As used herein, the terms "comprise," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.
[0059] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0060] See also Figure 1 and 2 As shown, an embodiment of the present application provides a time series prediction method based on local clustering. The time series prediction method based on local clustering includes:
[0061] Step S1, performing segmented preprocessing on the collected time series data to obtain several clusters;
[0062] Step S2, constructing a clustering loss function containing dual constraints;
[0063] Step S3: Based on the clustering loss function, cluster training is performed to obtain the final k cluster centers.
[0064] Step S4, obtaining the hidden layer representation of the input time series based on the final k cluster centers;
[0065] Step S5: Generate future prediction results based on the hidden layer representation.
[0066] The beneficial effects of the above-described embodiments are as follows: the time series prediction method based on local clustering uses a local clustering method to achieve dimensionality reduction of the input data, eliminating the need to directly process the original high-dimensional time series data, thereby improving operational efficiency in long time series scenarios; the cluster centers jointly optimize the dual clustering loss function on the entire data set, so that the hidden layer representation after dimensionality reduction deeply aligns with the inherent evolutionary laws of the time series data. Compared to dimensionality reduction methods that rely solely on local statistical characteristics, the present invention effectively preserves the long-term dependencies of time series data through a global perception mechanism, ultimately achieving more accurate future trend modeling.
[0067] In another embodiment, before step S1, the method further includes:
[0068] The collected time series data are stored in a time series database in chronological order; the time series data includes N variables, each variable corresponds to T time step observations; the time step refers to the basic unit of discretized time in the time series data.
[0069] In actual scenarios, time series data generated by scenarios such as meteorological monitoring and transportation monitoring can be collected. After the time series data collection is completed, the original collected time series data is stored in the time series database in the order of their collection time. Index information can also be generated based on the collection time information of the stored time series data to facilitate the rapid and accurate retrieval of the required time series data in the time series database. In addition, the collection of time series data can be performed for multiple variables, that is, multiple variables can be collected at the same time. For example, in a meteorological monitoring scenario, multiple variables such as temperature, humidity, wind speed / direction, and rainfall can be collected at the same time. The total number of time steps for the collection and monitoring of each variable is T, that is, the collection and monitoring process of each variable goes through T time step intervals, and the corresponding variable is collected and monitored in each time step interval to obtain the observation value of the corresponding variable.
[0070] In another embodiment, in step S1, the collected time series data is segmented and pre-processed to obtain a number of clusters, including:
[0071] Set a segmentation window p with a fixed time length, divide each variable into n segments; then divide the n segments into several clusters, where each cluster includes data corresponding to several segments.
[0072] In another embodiment, in step S1, a segmentation window p with a fixed time length is set to divide each variable into n segments, including:
[0073] If the number of time steps T is divisible by p, then n = T / p;
[0074] If the number of time steps T is not divisible by p, then in, Indicates rounding down the result of T / p.
[0075] Considering the large scale and randomness of time series data collection, the collection of time series data in different monitoring scenarios is not always implemented according to a unified collection timing pattern, and there are large variations in the length and number of collection time steps. To facilitate the subsequent unified processing of time series data and reduce the data processing workload, it is necessary to perform segmentation preprocessing on the time series data. This segmentation divides the original time series data into several time periods of fixed length to achieve unified and standardized processing of the time series data.
[0076] Specifically, when the number of time steps T is divisible by p, then n = T / p, so that the original time series data can be directly segmented and preprocessed without any padding. When the number of time steps T is not divisible by p, then In the above case, because the number of time steps T cannot be divided by p, there are remaining unfilled time periods when the time series data is segmented according to the segmentation window p of a fixed time length. At this time, the remaining unfilled data segments are filled with the above unfilled time periods. For example, the average value of the existing data values in the above unfilled time period can be used to fill the remaining position of the above unfilled time period, so that the above unfilled time period is completed into a complete segment of length p, thereby realizing large-scale unified processing of time series data.
[0077] In another embodiment, in step S2, a clustering loss function containing dual constraints is constructed, including:
[0078] Construct the reconstruction loss as follows (1)
[0079]
[0080] In the above formula (1), Represents clusters The mean vector, c j Represents clusters The cluster center of k represents the number of clusters; the above formula (1) describes each cluster The mean vector and cluster center c j The Euclidean distance and sum;
[0081] Construct the correlation loss as follows (2):
[0082]
[0083] In the above formula (2), Represents each cluster All time periods and cluster center c j The degree of similarity Sum, where the similarity The Pearson correlation coefficient is used for calculation, where k represents the number of clusters. Represents clusters Medium time period The number of; among them, the correlation loss The negative value design of makes the higher the correlation, the smaller the loss;
[0084] Construct the clustering loss function as follows (3):
[0085]
[0086] In the above formula (3), α represents a preset weight coefficient.
[0087] Through the above process, a clustering loss function with dual constraints is constructed, while considering the numerical differences between time periods in the clustering loss formula (corresponding to the reconstruction loss ) and correlation (corresponding to the correlation loss ), which is more conducive to establishing correlations between different time periods than previous clustering loss definitions that only consider numerical differences. Furthermore, cluster centers are generated by jointly optimizing a dual-constrained clustering loss function (comprising reconstruction loss and correlation loss) across the entire dataset, ensuring that the hidden layer representation after dimensionality reduction deeply aligns with the inherent evolutionary patterns of time series data. Compared to dimensionality reduction methods that rely solely on local statistical properties, this global perception mechanism effectively preserves the long-term dependencies of time series data, enabling more accurate modeling of future trends.
[0088] In another embodiment, in step S3, clustering is trained based on the clustering loss function to obtain the final k cluster centers, including:
[0089] Randomly initialize k cluster centers and use the clustering loss function The definition assigns all segments to the nearest cluster center;
[0090] Then according to the clustering loss function Update cluster centers using the gradient direction propagation method;
[0091] The above two processes are iterated until the clustering loss converges to obtain the final k cluster centers.
[0092] Through the above process, the mean update center of the data segments within the cluster is calculated, and the clustering loss function is directly optimized using the gradient backpropagation method, so that the center update process simultaneously incorporates the bidirectional constraints of numerical distance (i.e., Euclidean distance) and temporal similarity (i.e., Pearson correlation coefficient), thereby capturing the potential correlation structure between data segments and converging to a more globally representative temporal pattern.
[0093] In another embodiment, in step S4, obtaining a hidden layer representation of the input time series based on the final k cluster centers includes:
[0094] The input time series is divided into multiple segments according to the segmentation length l, and then a linear layer is used to obtain the initial hidden layer representation of all ends. in Includes n items;
[0095] Calculate the preliminary hidden layer representation of the final k cluster centers in Includes k items;
[0096] Processing preliminary hidden layer representations through cross-attention mechanism and the initial hidden layer representation Get the hidden layer representation of the input time series The hidden layer represents It includes N variables, each of which has n hidden layers. In the above formula, CrossAttn() represents the cross attention mechanism calculation.
[0097] Through the above process, since the initial hidden layer representation It only contains fixed k items. Therefore, the time complexity of the algorithm is much smaller than that of the existing full attention-based method. Reduced to The operating efficiency when processing long-length input time series data has been greatly improved.
[0098] In another embodiment, in step S5, generating a future prediction result based on the hidden layer representation includes:
[0099] Use Gaussian distribution initialization to form m readout representations;
[0100] Perform cross-attention calculations on the n hidden layer representations and m readout representations of each variable to obtain m final representations;
[0101] A cross-attention mechanism is performed on the n hidden layer representations and m readout representations of each variable to obtain a set of m final representations;
[0102] After flipping the n hidden layer representations of each variable, the cross-attention mechanism is calculated with the m readout representations to obtain another set of m final representations;
[0103] Using the following equations (4) and (5), according to the m final representations and the other group of m final representations, a linear layer with a gating mechanism is projected as the final prediction result That is, generating future prediction results,
[0104]
[0105] In the above formulas (4)-(5), CrossAttn() represents the cross attention mechanism calculation, represents the m readout representations, represents n hidden layer representations, Indicates that the n hidden layers are flipped, that is, The dimensions of the two matrices are swapped, represents a set of m final representations, represents another set of m final representations, β represents the preset gating weight, Indicates the intermediate quantity, Indicates that Projected via a linear layer.
[0106] Through the above process, the readout representation is combined with the price difference attention mechanism to replace the previous output method of feature splicing and large-scale linear layers, which greatly reduces the parameter scale and thus reduces the running memory overhead.
[0107] In another embodiment, after step S5, the method further includes:
[0108] According to the average mean square error of the future prediction results, the neural network parameters involved in steps S3 and S4 are updated through the gradient back propagation method.
[0109] Specifically, the average mean square error of the future prediction results is first calculated; based on the above average mean square error, combined with the chain rule, the error gradient is reversely calculated layer by layer from the last layer of network parameters; according to the error gradient of each layer of network parameters, the neural network parameters involved in steps S3 and S4 are updated through Adam and AdamW optimizers.
[0110] See also Figure 3 As shown, an embodiment of the present application provides a time series prediction system based on local clustering. The time series prediction system based on local clustering includes:
[0111] The preprocessing module is used to perform segmented preprocessing on the collected time series data to obtain several clusters;
[0112] Loss function construction module, used to construct clustering loss function containing dual constraints;
[0113] Cluster training module, used to train clustering and obtain the final k cluster centers based on the clustering loss function;
[0114] Hidden layer representation acquisition module, used to obtain the hidden layer representation of the input time series based on the final k cluster centers;
[0115] The prediction result generation module is used to generate future prediction results based on the hidden layer representation.
[0116] The operation and effects of the local clustering-based time series prediction system of the present invention are corresponding to and consistent with the above-mentioned local clustering-based time series prediction method, and the local clustering-based time series prediction system will not be repeated here.
[0117] The above is only a specific embodiment of the present invention, and any other improvements made based on the concept of the present invention are considered to be within the protection scope of the present invention.
Claims
1. The time series prediction method based on local clustering is characterized by: include: Step S1, performing segmented preprocessing on the collected time series data to obtain several clusters; Step S2, constructing a clustering loss function containing dual constraints; Step S3: Based on the clustering loss function, cluster training is performed to obtain the final k cluster centers. Step S4, obtaining the hidden layer representation of the input time series based on the final k cluster centers; Step S5: Generate future prediction results based on the hidden layer representation.
2. The time series prediction method based on local clustering according to claim 1, characterized in that: Before step S1, the method further includes: The collected time series data are stored in a time series database in chronological order; wherein the time series data includes N variables, and each variable corresponds to an observation value of T time steps.
3. The time series prediction method based on local clustering according to claim 2, characterized in that: In step S1, the collected time series data is segmented and preprocessed to obtain several clusters, including: Set a segmentation window p with a fixed time length, divide each variable into n segments; then divide the n segments into several clusters, where each cluster includes data corresponding to several segments.
4. The time series prediction method based on local clustering according to claim 3, characterized in that: In step S1, a segmentation window p with a fixed time length is set to divide each variable into n segments, including: If the number of time steps T is divisible by p, then n = T / p; If the number of time steps T is not divisible by p, then in, Indicates rounding down the result of T / p.
5. The time series prediction method based on local clustering according to claim 4, characterized in that: In step S2, a clustering loss function with dual constraints is constructed include: Construct the reconstruction loss as follows (1) In the above formula (1), Represents clusters The mean vector, c j Represents clusters The cluster center of k represents the number of clusters; the above formula (1) describes each cluster The mean vector and cluster center c j The Euclidean distance and sum; Construct the correlation loss as follows (2): In the above formula (2), Represents each cluster All time periods and cluster center c j The degree of similarity Sum, where the similarity The Pearson correlation coefficient is used for calculation, where k represents the number of clusters. Represents clusters Medium time period the number of Construct the clustering loss function as follows (3): In the above formula (3), α represents a preset weight coefficient.
6. The time series prediction method based on local clustering according to claim 5, characterized in that: In step S3, based on the clustering loss function, clustering is trained and the final k cluster centers are obtained, including: Randomly initialize k cluster centers and use the clustering loss function The definition assigns all segments to the nearest cluster center; Then according to the clustering loss function Update cluster centers using the gradient direction propagation method; The above two processes are iterated until the clustering loss converges to obtain the final k cluster centers.
7. The time series prediction method based on local clustering according to claim 6, characterized in that: In step S4, the hidden layer representation of the input time series is obtained based on the final k cluster centers, including: The input time series is divided into multiple segments according to the segmentation length l, and then a linear layer is used to obtain the initial hidden layer representation of all ends. in Includes n items; Calculate the preliminary hidden layer representation of the final k cluster centers in Includes k items; Processing preliminary hidden layer representations through cross-attention mechanism and the initial hidden layer representation Get the hidden layer representation of the input time series The hidden layer represents It includes N variables, each of which has n hidden layers.
8. The time series prediction method based on local clustering according to claim 7, characterized in that: In step S5, based on the hidden layer representation, future prediction results are generated, including: Use Gaussian distribution initialization to form m readout representations; A cross-attention mechanism is performed on the n hidden layer representations and m readout representations of each variable to obtain a set of m final representations; After flipping the n hidden layer representations of each variable, the cross-attention mechanism is calculated with the m readout representations to obtain another set of m final representations; Using the following equations (4) and (5), according to the m final representations and the other group of m final representations, a linear layer with a gating mechanism is projected as the final prediction result That is, generating future prediction results, In the above formulas (4)-(5), CrossAttn() represents the cross attention mechanism calculation, represents the m readout representations, represents n hidden layer representations, Indicates that the n hidden layers are flipped, that is, The dimensions of the two matrices are swapped, represents a set of m final representations, represents another set of m final representations, β represents the preset gating weight, Indicates the intermediate quantity, Indicates that Projected via a linear layer.
9. The time series prediction method based on local clustering according to claim 1, wherein: After step S5, the method further includes: According to the average mean square error of the future prediction results, the neural network parameters involved in steps S3 and S4 are updated through the gradient back propagation method.
10. A time series prediction system based on local clustering, characterized in that: include: The preprocessing module is used to perform segmented preprocessing on the collected time series data to obtain several clusters; Loss function construction module, used to construct clustering loss functions containing dual constraints Clustering training module, used for clustering loss function Train clustering and obtain the final k cluster centers; Hidden layer representation acquisition module, used to obtain the hidden layer representation of the input time series based on the final k cluster centers; The prediction result generation module is used to generate future prediction results based on the hidden layer representation.