A global-local divergence based multivariate time series unsupervised dimensionality reduction method

Abstract

The application discloses a kind of multivariate time series unsupervised dimension reduction method based on global-local divergence, it includes the following steps: S1, the covariance matrix of multivariate time series is calculated, the upper triangular element of covariance matrix is extracted, it is combined as feature sequence, obtain the feature set Fea of multivariate time series={f i |i=1,2,…,n};S2, using k near neighbor and Euclid Distance (Euclid Distance, ED) measure is established neighborhood set N k (f i )={f j |j=1,2,…,k};S3, after finding the neighborhood of each sample point, the neighborhood center sequence m i Of each feature sequence f i In feature set is calculated;S4, with the neighborhood variance of sample point after projection, local divergence is characterized.First, the variance of each sample point neighborhood set after projection is calculated, then the sum of the accumulation of these variances is obtained, to obtain local divergence;S5, according to the field center point obtained in step S3, the variance of field center point is calculated to obtain global variance.Finally, the low-dimensional projection sequence obtained by the experimental result shows that the method can represent original MTS, and obvious dimension reduction is realized.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to an unsupervised dimensionality reduction method for multivariate time series based on global-local divergence. Background Technology

[0002] Time series typically refers to a series of data collected from various sensors that change over time. It is widely used in fields such as environmental science, medicine, and finance. Based on the number of variables, time series can be divided into univariate time series (UTS) and multiple time series (MTS). MTS can be viewed as a combination of corresponding UTS generated by different factors within the same system. Compared to UTS, MTS not only possesses a higher dimensionality in terms of time but also in terms of feature dimensionality, and there are correlations between these features. Therefore, data mining of MTS is more complex than that of UTS.

[0003] Currently, data mining using MTS (Multi-Dimensional Synthesis) is applied in industrial fault monitoring. Since most fault monitoring data is acquired through sensors, it contains a significant amount of noise. Furthermore, the monitoring data collected by sensors has both feature and time dimensions, and this high dimensionality makes data processing difficult and inefficient during fault monitoring. Therefore, effectively reducing the dimensionality of multivariate time-series data in fault monitoring is key to solving this problem.

[0004] Time series data mining typically includes clustering, classification, prediction, anomaly detection, and correlation analysis. These tasks are usually related to the scale and complexity of the data. Since time series data (MTS) possesses high dimensionality in two dimensions, dimensionality reduction or feature representation is often necessary to reduce data complexity and mitigate interference from redundant information during these mining tasks. Currently, existing technologies mainly fall into three categories: dimensionality reduction based on the feature dimension, dimensionality reduction based on the time dimension, and dimensionality reduction based on both. However, dimensionality reduction based on the feature dimension cannot address the technical problem of unequal time lengths in MTS, leading to difficulties in subsequent similarity measurement. Dimensionality reduction based on the time dimension ignores the correlation characteristics of variables in the feature dimension and may still result in information redundancy. Dimensionality reduction based on both dimensions typically requires bidirectional dimensionality reduction techniques, which are computationally expensive. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an unsupervised dimensionality reduction method for multivariate time series based on global-local divergence.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An unsupervised dimensionality reduction method for multivariate time series based on global-local divergence, characterized by the following steps:

[0008] S1: Obtain fault monitoring data through sensors, combine the acquired fault monitoring data multivariate time series into a multivariate time series original dataset D, calculate the covariance matrix of each multivariate time series in the dataset, extract the upper triangular elements of the covariance matrix, combine them into a feature sequence, and form a feature sequence set from all feature sequences, with each feature sequence having the same length.

[0009] S2: Based on the obtained feature sequence set, use the k-nearest neighbor number and Euclidean distance to establish the neighborhood set of each sample in the feature sequence set;

[0010] S3: Based on the neighborhood set obtained in step S2, calculate the neighborhood center sequence for each neighborhood set in the feature set;

[0011] S4: Based on the neighborhood set obtained in step S2, calculate the variance of each neighborhood set of the sample points to be projected, and then sum these variances to calculate the local divergence.

[0012] S5: Calculate the global variance of the neighborhood based on the neighborhood center points obtained in step S3, and obtain the global divergence.

[0013] S6: Solve for the projection matrix based on the local and global divergences obtained in steps S4 and S5;

[0014] S7: Based on the projection matrix obtained in step S6, project the feature sequence set obtained in step S1 to obtain the dimensionality-reduced feature sequence:

[0015] y i =W T f i (6);

[0016] S8: Based on the dimensionality reduction feature sequence, obtain the dimensionality reduction feature set D′={y} of the fault monitoring data. i |i=1,2,…,n}, where,

[0017] S9: Process the dimensionality-reduced fault monitoring data to obtain the fault monitoring results.

[0018] Furthermore, the specific operational steps of step 1 include:

[0019] S11: The acquired fault monitoring data, in a multivariate time series format, is used to construct a multivariate time series raw dataset D = {X}. i |i=1,2,…,n}, where n is the number of samples. Let x represent the i-th MTS sample in the original dataset.i (i = 1, 2, ..., m) represents a series of observations of the i-th variable, where m is the number of variables, and t i X is the time length of the i-th multivariate time series; and each multivariate time series is zero-mean processed: X i =X i -E(X i );

[0020] S12: Calculate the covariance matrix for each zero-mean multivariate time series, and the covariance matrix of the i-th MTS is:

[0021]

[0022] S13: The covariance matrix Extract the elements of the upper triangular matrix and arrange them into row vectors in order:

[0023] f i =(a 11 ,a 12 ,…,a 1m ,a 21 ,…,a 2m ,…,a mm (8)

[0024] f i As the feature sequence of the i-th MTS, the feature set Fea={f i |i=1,2,…,n}.

[0025] Further, in step S3, the neighborhood center sequence m is calculated. i The calculation formula is:

[0026]

[0027] Where k is the nearest neighbor number, m i f i The sequence of neighborhood centers of its k nearest neighbors.

[0028] Furthermore, the specific steps of S4 include:

[0029] S41: Based on the neighborhood set, calculate the variance of each neighborhood set of the projected sample points, and sum these variances to obtain the local divergence calculation formula:

[0030]

[0031] Where, p i =W T m i The low-dimensional projection of the neighborhood center point; y jIt refers to the low-dimensional projection of the sample points, where W is the projection matrix and the subscript L is the abbreviation for Local.

[0032] S42: By transforming equation (2), we can obtain:

[0033]

[0034] in, For the i-th feature sequence f i Local divergence matrix:

[0035]

[0036] Furthermore, the specific steps of S5 include:

[0037] S51: Based on the neighborhood center points in the neighborhood center sequence, calculate the global variance of the neighborhood to obtain the global divergence calculation formula:

[0038]

[0039] Where, p i =W T m i It is a low-dimensional projection of the neighborhood center point; the subscript G is an abbreviation for Global.

[0040] S52: By transforming equation (3), we can obtain:

[0041]

[0042] Among them, S G The global scatter matrix is:

[0043]

[0044] S53: Simplifying equation (12), we get:

[0045]

[0046] in, It is the center point of all neighborhood centers, i.e., the global neighborhood center.

[0047] Furthermore, the specific steps of S6 include:

[0048] S61: Combining the local divergence obtained from formula (9) and the global divergence obtained from formula (11) yields formula (4):

[0049]

[0050] S62: Transform equation (4) into a problem of solving generalized eigenvalues, and obtain the projection matrix by solving equation (4):

[0051] S G ω=λS L ω (5)

[0052] Where: ω is the generalized eigenvector, and λ is the generalized eigenvalue;

[0053] S63: Solving equation (5) yields the first d largest eigenvalues ​​λ1, λ2, ..., λ d (d<p), corresponding eigenvectors ω1,ω2,…,ω d And the projection matrix W = (ω1, ω2, ..., ω d ).

[0054] Compared with the prior art, the advantages of this invention are as follows:

[0055] This invention first proposes a feature sequence extraction method, extracting the upper triangular elements of the covariance matrix of a multivariate time series and combining them into a feature sequence. Then, based on the fundamental idea of ​​"minimizing local divergence and maximizing global divergence," an unsupervised dimensionality reduction model is proposed, preserving global information as much as possible while maintaining local nearest neighbor relationships. Using the feature sequence as input, the model minimizes the sum of the variances of the neighborhoods of all sample points and maximizes the variance of the neighborhood centroids. The projection matrix obtained by solving the model enables dimensionality reduction of multivariate time series. Finally, this invention experimentally verifies the proposed dimensionality reduction method and related comparative methods using 20 publicly available datasets. Experimental results show that the proposed dimensionality reduction method can effectively reduce the dimensionality of the MTS dataset, thereby improving the accuracy of fault monitoring. Detailed Implementation

[0056] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0057] This invention discloses an unsupervised dimensionality reduction method for multivariate time series based on global-local divergence, specifically including the following steps:

[0058] S1. Collect fault monitoring data through sensors to obtain the corresponding multivariate time series, and combine them into a multivariate time series raw dataset D. Then, transform the multivariate time series raw dataset D into a set of equal-length feature sequences Fea = {f i |i=1,2,…,n};

[0059] Specifically, the specific operating steps of S1 include:

[0060] S11. Combine the obtained multivariate time series data to form the original multivariate time series dataset D = {X} i |i=1,2,…,n}, where n is the number of samples. Let x represent the i-th MTS sample in the dataset. i (i = 1, 2, ..., m) represents a series of observations of the i-th variable, where m is the number of variables, and t i X is the time length of the i-th MTS. For each MTS, zero-mean processing is performed, i.e., X... i =X i -E(X i );

[0061] S12. Calculate the covariance matrix for each zero-mean multivariate time series, and the covariance matrix of the i-th MTS is:

[0062]

[0063] Wherein, the covariance matrix It is a symmetric matrix;

[0064] σ i Extract the elements of the upper triangular matrix and arrange them into row vectors in order:

[0065] f i =(a 11 ,a 12 ,…,a 1m ,a 21 ,…,a 2m ,…,a mm (8)

[0066] f i As the feature sequence of the i-th MTS, we obtain the feature set Fea={f i |i=1,2,…,n}.

[0067] S2. Based on the obtained feature set Fea, k-nearest neighbors and Euclidean distance (ED) are used as metrics to obtain the neighborhood set N of each sample in the feature set Fea. k (f i )={f j |j=1,2,…,k}

[0068] S3. Based on the neighborhood set obtained in step S2, calculate the neighborhood center sequence m for each neighborhood set in the feature set Fea. i m i The calculation formula is:

[0069]

[0070] Where k is the nearest neighbor number, m i f i The sequence of neighborhood centers of its k nearest neighbors;

[0071] S4. Based on the neighborhood set obtained in step S2, obtain the variance of each neighborhood set of the projected sample points. Then, sum these variances to obtain the formula for calculating local divergence:

[0072]

[0073] Where, p i =W T m i The low-dimensional projection of the neighborhood center point; y j It refers to the low-dimensional projection of the sample points, where W is the projection matrix and the subscript L is an abbreviation for Local.

[0074] Specifically, S4 also includes:

[0075] S41. By transforming equation (2), we can obtain:

[0076]

[0077] in, S Li For the i-th feature sequence f i The local divergence matrix, i.e.:

[0078]

[0079] S5. Based on the neighborhood center points obtained in step S3, calculate the global variance of the neighborhood to obtain the global divergence:

[0080]

[0081] Where, p i =W T m i It represents the low-dimensional projection of the neighborhood center point; the subscript G is an abbreviation for Global.

[0082] Specifically, S5 also includes:

[0083] S51. By transforming equation (3), we can obtain...

[0084]

[0085] in

[0086]

[0087] S52, S in formula (12) G The global scatter matrix can be simplified to obtain:

[0088]

[0089] in, The center point of all neighborhood centers is called the global neighborhood center;

[0090] S6. Combine the local divergence (formula (9)) and global divergence (formula (11)) obtained in steps S4 and S5 to obtain equation (4). Solve equation (4) to obtain the projection matrix W = (ω1, ω2, ..., ω d ):

[0091]

[0092] Equation (4) is transformed into a problem of solving generalized eigenvalues:

[0093] S G ω=λS L ω (5)

[0094] Where ω is the generalized eigenvector and λ is the generalized eigenvalue.

[0095] By solving equation (5), we can obtain the first d largest eigenvalues ​​λ1, λ2, ..., λ. d (d<p) and the corresponding eigenvectors ω1,ω2,…,ω d And the projection matrix W = (ω1, ω2, ..., ω d );

[0096] S7. Based on the projection matrix obtained in step S6, project the sample points (i.e., elements in the feature sequence set Fea) obtained in step S1 to obtain the dimensionality-reduced feature sequence:

[0097] y i =W T f i (6)

[0098] Finally, the dimensionality-reduced feature set D′={y i |i=1,2,…,n}, where,

[0099] Example

[0100] To verify the dimensionality reduction method proposed in this invention (hereinafter referred to as GLSUP), relevant experiments were conducted.

[0101] 1. Dataset Selection

[0102] Let the MTS dataset D = {X} i |i=1,2,…,n}, where n is the number of samples. It is the i-th MTS in the dataset, x i (i = 1, 2, ..., m) represents a series of observations of the i-th variable, where m is the number of variables, and the length of the feature sequence is (m+1)m / 2. Assume the average length of the MTS in the dataset is t, and the number of low-dimensional features selected is d (d << m). 2 ).

[0103] 2. Algorithm Complexity

[0104] The training process of this invention mainly includes three parts: feature extraction, model solving, and projection. In the feature extraction stage, the computational cost is mainly the calculation of the MTS covariance matrix, with a computational complexity of O(nm). 2 t). In the model solution stage, the computational cost mainly involves calculating the local divergence matrix and generalized eigenvalues, with computational complexities of O(nm) and O(nm), respectively. 4 ) and O(m 6 After obtaining the projection matrix W, the GLSUP method projects the feature sequences in the feature set Fea, with a computational complexity of O(dm). 2 Therefore, the complexity of the GLSUP method is O(nm). 2 t+nm 4 +m 6 ).

[0105] 3. MTS Data Dimensionality Reduction Experiment Process

[0106] This embodiment selects 20 multivariate time series datasets from different fields, as shown in Table 1, namely: LP1, LP2, LP3, LP4, LP5, Daily and Sports Activities (DSA), Finger Movements (FM), Hand Movement Direction (HMD), NATOPS, Cricket, Racket Sports (RS), Epilepsy, Basic Motions (BM), LSST, Articular Word Recognition (AWR), EEGeye, Wafer, WalkvsRun (WR), KickvsPunch (KP), and Australian Sign Language (ASL). The first 15 are equal-length datasets, and the last 5 are unequal-length datasets.

[0107] Table 1 MTS Dataset Information

[0108]

[0109] Dimensionality reduction effectiveness refers to the degree to which information is retained to characterize the MTS features after dimensionality reduction. The experiment inputs the dimensionality reduction results into a KNN (K=1) classifier, and the classification accuracy is used to evaluate the effectiveness of dimensionality reduction. Therefore, the experimental process can be described as follows: The original MTS dataset is dimensionality reduced using a dimensionality reduction algorithm to obtain a dimensionality-reduced dataset. Samples are sequentially selected from the dimensionality-reduced dataset and input into the classifier. A nearest neighbor query is used to obtain the sample most similar to the query sample. The label value of this sample is taken as the category to which the query sample belongs. If the label value matches that of the query sample, it is a correct classification; otherwise, it is an incorrect classification. After performing the operation on all samples, the classification accuracy is obtained:

[0110] ε=n true / n

[0111] Where ε is the classification accuracy, n true The number of correctly classified samples is n, where n is the number of samples.

[0112] Five unsupervised dimensionality reduction methods—PCA, CPCA, LPP, PBLDA, and VPCA—were selected as comparison methods. Since VPCA is only applicable to datasets of equal length, experiments were conducted on only 15 datasets of equal length. On the unequal-length MTS dataset, the dimensionality reduction results of PCA and CPCA remained unequal-length sequences. Euclidean distance can only measure sequences of equal length. Therefore, on the five unequal-length datasets, Dynamic Time Warping (DTW) distance was used to measure the dimensionality reduction results of PCA and CPCA, achieving KNN classification.

[0113] The parameters involved in PCA, CPCA, and VPCA methods include: variance contribution rate σ; the parameters involved in the LPP method include: nearest neighbor number k, heat kernel parameter t, and number of low-dimensional features d; the parameters involved in the PBLDA method include: feature dimension p. c Time dimension p r The GLSUP method involves the following parameters: the number of nearest neighbors k and the number of low-dimensional features d. In the experiment, the variance contribution rate σ of the PCA, CPCA, and VPCA methods was set to 80%. The number of nearest neighbors k and the heat kernel parameter t were both set to 1. The time dimension p r While keeping the original time series length unchanged, the feature dimension p is adjusted. c The optimal matching accuracy of the three dimensionality reduction algorithms, PBLDA, GLSUP, and LPP, is obtained by using the low-dimensional feature number d.

[0114] The experimental results on the effectiveness of dimensionality reduction are shown in Table 2, with the highest classification accuracy in each row highlighted in bold. The results show that the GLSUP method achieves good classification accuracy across all 20 datasets. The GLSUP method transforms the MTS into a feature sequence of equal length, preserving the correlation information between different variables, and then simultaneously considers both global and local information of the dataset, thus achieving dimensionality reduction of the feature sequence.

[0115] Table 2. Experimental results on the effectiveness of dimensionality reduction.

[0116]

[0117]

[0118] 4. Experimental Conclusions

[0119] Among the other methods, CPCA offers a significant improvement in dimensionality reduction effectiveness compared to PCA. This is because CPCA projects the MTS to a common low-dimensional subspace, while PCA projects to different low-dimensional subspaces. However, both methods only reduce the dimensionality of the variables, without reducing the sequence length. VPCA achieves high classification accuracy but is only suitable for datasets of equal length. PBLDA reduces dimensionality in both the variable and time dimensions, but its performance is poor on some datasets because it truncates unequal-length sequences to equal length, resulting in information loss. LPP extracts feature sequences from the MTS and performs LPP dimensionality reduction, solving the unequal-length problem, but it only considers local information and ignores global information. Furthermore, this method requires singular value decomposition for each MTS, leading to high computational complexity.

[0120] Furthermore, in the multi-class ASL dataset, the GLSUP method has a significant advantage over other methods. This is because the GLSUP method considers both global and local information of the dataset, and can project samples into well-separable clusters even in unlabeled datasets.

[0121] In summary, the dimensionality reduction method proposed in this invention uses the covariance matrix of the MTS as the feature sequence of the MTS, which can transform MTSs with different time lengths into feature sequences of equal length. Moreover, by projecting the feature sequences of equal length onto the same common low-dimensional space, the resulting low-dimensional projection sequence can represent the original MTS, thus achieving a significant dimensionality reduction effect.

[0122] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A multivariate time series unsupervised dimensionality reduction method based on global-local divergence, characterized in that, Includes the following steps: S1: Acquire fault monitoring data through sensors, and combine the acquired fault monitoring data into a multivariate time series raw dataset. Calculate the covariance matrix for each multivariate time series in the dataset, extract the upper triangular elements of the covariance matrix, and combine them into a feature sequence; then, construct a feature sequence set from all the feature sequences, with each feature sequence having the same length. S2: Based on the obtained feature sequence set, use Nearest neighbor count and Euclidean distance are used to establish the neighborhood set of each sample in the feature sequence set; S3: Based on the neighborhood set obtained in step S2, calculate the neighborhood center sequence for each neighborhood set in the feature set; S4: Based on the neighborhood set obtained in step S2, calculate the variance of each neighborhood set of the sample points to be projected, and then sum these variances to calculate the local divergence. S5: Calculate the global variance of the neighborhood based on the neighborhood center points obtained in step S3, and obtain the global divergence. S6: Solve for the projection matrix based on the local and global divergences obtained in steps S4 and S5; S7: Based on the projection matrix obtained in step S6, project the feature sequence set obtained in step S1 to obtain the dimensionality-reduced feature sequence: （6）； S8: Obtain the dimensionality-reduced feature set of the fault monitoring data based on the dimensionality-reduced feature sequence. ,in, ; S9: Process the dimension-reduced fault monitoring data to obtain fault monitoring results; The specific steps in step 1 include: S11: Combine the acquired fault monitoring data into a multivariate time series dataset. ,in For the number of samples, Represents the first in the original dataset One MTS sample, Indicates the first A series of observations of a variable, For the number of variables, It is the first The time lengths of the multivariate time series are determined, and each multivariate time series is zero-mean normalized. ; S12: Calculate the covariance matrix for each zero-mean multivariate time series, and the... The covariance matrix of each MTS is: （7） ； S13: The covariance matrix Extract the elements of the upper triangular matrix and arrange them into row vectors in order: （8） Will As the first The feature sets of MTS are obtained by analyzing the feature sequences of each MTS. ; In step S3, the neighborhood center sequence is calculated. The calculation formula is: （1） in, The nearest neighbor number, for With The neighborhood center sequence of the nearest neighbor points; Furthermore, the specific steps of S4 include: S41: Based on the neighborhood set, calculate the variance of each neighborhood set of the projected sample points, and sum these variances to obtain the local divergence calculation formula: （2） in, The low-dimensional projection of the neighborhood center point; It refers to the low-dimensional projection of the sample points, where W is the projection matrix and the subscript L is the abbreviation for Local. S42: By transforming equation (2), we can obtain: （9） in, , For the first Feature sequences Local divergence matrix: （10）； The specific steps of S5 include: S51: Based on the neighborhood center points in the neighborhood center sequence, calculate the global variance of the neighborhood to obtain the global divergence calculation formula: （3） in, It is a low-dimensional projection of the neighborhood center point; the subscript G is an abbreviation for Global. S52: By transforming equation (3), we can obtain: （11） in, The global scatter matrix is: （12）； S53: Simplifying equation (12), we get: （13） in, It is the center point of all neighborhood centers, i.e., the global neighborhood center.

2. The unsupervised dimensionality reduction method for multivariate time series based on global-local divergence as described in claim 1, characterized in that, The specific steps of S6 include: S61: Combining the local divergence obtained from formula (9) and the global divergence obtained from formula (11) yields formula (4): （4）； S62: Transform equation (4) into a problem of solving generalized eigenvalues, and obtain the projection matrix by solving equation (4): （5） in: For generalized eigenvectors, These are generalized eigenvalues; S63: Solving equation (5) yields the previous... The largest eigenvalue , corresponding eigenvectors and projection matrix .

Images