A data dimensionality reduction method based on principal component analysis in Fourier domain

Abstract

The present invention discloses a data dimensionality reduction method based on Fourier domain principal component analysis. By projecting high-dimensional data into the Fourier domain and utilizing the properties of circulant matrices and Fourier matrices, the problem of solving eigenvectors in principal component analysis is transformed into finding a meaningful Fourier domain basis. Since the Fourier domain basis is predefined and the principal component distribution of the data is ordered, the training can be accelerated by inputting the training samples in batches until the required Fourier basis is stable and ordered. The number of Fourier bases and the projection matrix are determined, and the projection matrix is ​​multiplied by the high-dimensional data set to obtain a low-dimensional data set to facilitate rapid data processing. The data dimensionality reduction method provided by the present invention, based on principal component analysis and fast Fourier transform, can remove noise and redundant information in high-dimensional data sets, reduce unnecessary calculation processes in data processing, and improve the running speed and memory efficiency of the algorithm.

Description

Technical Field

[0001] The invention belongs to the field of computer science and image processing technology, and in particular is a data dimension reduction method based on Fourier domain principal component analysis. Background Art

[0002] Traditional data processing methods are no longer able to effectively analyze massive amounts of data. At the same time, as the data dimensions generated by big data processing and cloud computing continue to increase, it is even more necessary to perform dimensionality reduction on high-dimensional data in order to remove noise and redundant information from high-dimensional data sets, reduce unnecessary computational processes in data processing, and improve the efficiency of algorithms.

[0003] Principal Component Analysis (PCA) is an advanced data exploration algorithm that can be used to find patterns in data and find data transformation representations that can emphasize these patterns. PCA rotates the coordinate axes of the original data set orthogonally so that the originally scattered sample points are concentrated near certain feature coordinate axes after rotation. When the principal components that carry less original information are ignored, the dimensionality of the original data is reduced and data compression is achieved. PCA is a powerful data transformation technique that you can apply to further analysis. This method is very useful when you encounter high-dimensional data sets. It can help understand the underlying data structure, cluster analysis, regression analysis, and many other tasks.

[0004] However, although principal component analysis has shown good performance, its application in massive data processing problems is limited due to its high computational complexity. In order to process large-scale data, many optimization techniques have been proposed to accelerate the principal component analysis algorithm. According to different strategies to solve this problem, the existing optimization techniques can be roughly divided into the following two categories: one is to use Nystrom's matrix approximation technology, which reduces the computational cost of the eigendecomposition step by using the calculated submatrix eigenvectors to approximate the original matrix eigenvectors. Another method is to use Random Fourier Features to approximate the matrix, which can transform the original KPCA problem into a high-dimensional linear PCA problem. However, although the above methods solve the application processing problems of massive data, they are still not fully utilized in terms of speed and memory efficiency. The fast and efficient calculation of massive data is still a problem we face. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention proposes a data dimensionality reduction method based on Fourier domain principal component analysis, which uses the fast Fourier transform method to observe each data point in the sequence from the perspective of the frequency domain, and constructs a new Fourier transform-based principal component analysis algorithm. By optimizing the problem of solving the eigenvector of the principal component analysis to find a meaningful Fourier domain basis, and batch input training, the eigenvalue distribution of the global sample is approximated by using the stable and ordered eigenvalues ​​of some samples. This improves the computing speed and memory utilization of data dimensionality reduction, and provides support and acceleration for principal component analysis of large-scale data.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A data dimension reduction method based on Fourier domain principal component analysis comprises the following steps:

[0008] Step 1: Data initialization: collect the data sample set X as the required data set, where X is an M×N dimensional matrix; and initialize the current batch number j, the initial M×N dimensional zero matrix Λ 0 , random Fourier basis set P 0 and discrete Fourier matrix F; M represents the dimension of the data set X, and N is the number of samples of the data;

[0009] Step 2: Construct batch samples and Fourier data representation of batch samples, and randomly input a batch sample set X with a number b b ∈R M×b , for X b The sample vector x in i Perform Fourier transform to get

[0010] Step 3: For each batch of randomly input batch sample set X b , calculate the batch sample set X of this batch b The obtained eigenvalue matrix Λ b , expressed as: yes The complex conjugate matrix of ;

[0011] As small batch samples are continuously input, the batch sample set X of each batch b The obtained eigenvalue matrix Λ b Add to Λ j , using Λ j Represents the accumulation of feature values ​​after inputting the jth batch of samples. The process is expressed as: j ←Λ j-1 +Λ b ; Among them, Λ j-1 Indicates that when inputting j-1 batch sample set X b The characteristic values ​​obtained are accumulated;

[0012] Step 4: Obtain the Fourier projection basis of the batch sample set and convert Take it as the column vector of F; for the eigenvalue matrix Λ j The diagonal elements λ 1 ,λ 2 , ..., λ M Sort in ascending order and select the smallest first r eigenvalues ​​λ 1 ,λ 2 , ...λ r The corresponding Fourier basis in the matrix F Constructs the current projection set r is the preset number of required Fourier projection bases;

[0013] Step 5: If the set P j With P j-1 If the same, then end the execution of steps 2 to 4 and obtain the required Fourier basis As the final Fourier projection basis, otherwise execute steps 2 to 4 and update the current input batch number, j←j+1;

[0014] Step 6: For the set P j Perform an inverse Fourier transform on each Fourier projection basis i=1,...,r, forming a projection matrix V′=[p 1 p 2 … p r ]; Combine the high-dimensional dataset X with the projection matrix V′ T Multiply them together to get the reduced-dimensional data set X′=V′ T X.

[0015] Further, set the threshold g, batch sample set X b The number of samples is b=N*g, b<<M.

[0016] Furthermore, the threshold value g is set between 0.5% and 5%.

[0017] Further, for the sample vector x i Perform Fourier transform to get It is expressed as:

[0018]

[0019] in, is the generating vector of the Fourier transform, Represents the vector x i Perform fast Fourier transform, where F is the discrete Fourier matrix;

[0020] Further, obtain the sample vector x iThe method is: represent the data sample set as X = [x 1 x 2 ... x N ], the sample vector x is formed by the i-th column data samples in the data sample set X i , i = 1, 2, ... N, N represents the number of data samples; the sample vector x i Contains data samples of M dimensions in the i-th column.

[0021] Further, in this embodiment, is the sample x i The constructed circulant matrix is ​​expressed as:

[0022]

[0023] Among them, circ represents the vector x i Shift the corresponding circulant matrix The characteristic of this circulant matrix is ​​that it can be diagonalized by Fourier transform. F H =(F * ) T is the conjugate transpose of the Fourier matrix F, and H represents the conjugate transpose operation. The batch samples X of the current batch are obtained as follows b The characteristic value of:

[0024]

[0025] Where λ is the Lagrangian factor; b is the number of batch samples; yes The complex conjugate matrix of ; ⊙ is the element-wise multiplication operation of the matrix; diag means converting the vector into a diagonal matrix whose main diagonal is the vector element; is the main projection vector of the training data set X, that is, the feature vector. For each batch of randomly input samples X b , we can get Λ b :

[0026]

[0027] Among them, Λ b is the eigenvalue matrix obtained for this batch of samples.

[0028] Further, the Fourier domain basis vector is:

[0029]

[0030] Where V is the set of projection column vectors v, that is, V = [v 1 v 2 … vn ].

[0031] Beneficial effects of the present invention:

[0032] 1. Use the repeatability of the data sequence to model the data in the Fourier domain. Use the fast Fourier transform method to observe each data point in the sequence from the perspective of the frequency domain, and construct a new principal component analysis algorithm based on the Fourier domain. Finding the projection target of the principal component analysis can be achieved by finding a pre-defined meaningful Fourier basis.

[0033] 2. Due to the operational properties of the Fourier domain, we can avoid complex matrix inversion operations in the time domain by performing simple matrix dot product operations in the Fourier domain.

[0034] 3. In order to obtain the Fourier basis meaningfully, the training process does not need to load all the data samples, but only needs to load several batches of data samples until the order of the Fourier basis is stable, which can undoubtedly make more efficient use of memory.

[0035] 4. By optimizing the problem of solving the eigenvector of principal component analysis to find a meaningful Fourier domain basis, and by batch input training, the eigenvalue distribution of the global sample is approximated with the stable and orderly eigenvalues ​​of some samples. This improves the computing speed and memory utilization of the data dimensionality reduction process, and provides support and acceleration for principal component analysis of massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the main flow chart of the method proposed by the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] like Figure 1 A data dimensionality reduction method based on Fourier domain principal component analysis is shown, comprising the following steps:

[0039] Step 1: Dataset preparation: collect data sample set X(M×N) as the required data set, and represent the data sample set as X=[x 1 x 2 ... x N ], the sample vector x is formed by the i-th column data samples in the data sample set X i , i = 1, 2, ... N, N represents the number of data samples; the sample vector x i Contains data samples of M dimensions in the i-th column.

[0040] Initialization parameters: j, Λ 0 , F, P 0 . Where j represents the current batch number of batch training, and j = 1; Λ 0 represents the initial M×M dimensional zero matrix; F is the discrete Fourier matrix (DFT); P 0 is a random Fourier basis set, P 0 The elements of the set are column vectors of the discrete Fourier matrix (DFT) F. The discrete Fourier matrix (DFT) F is represented as:

[0041]

[0042] Among them, V is the set of projection column vectors v, that is, V = [v 1 v 2 … v n ], n is the number of columns of the set V; ω is a complex number and can be expressed as ω = e -2πi / M , i is the imaginary unit.

[0043] Step 2: construct batch samples and their Fourier data representation.

[0044] According to the threshold g, randomly input a batch of samples X with the number b = N*g b ∈R M×b , g is 0.5% to 5%. b Perform fast Fourier transform and use the fast Fourier transform method to observe the data from the perspective of frequency domain, which is expressed as follows:

[0045]

[0046] in, Represents the vector x i Perform fast Fourier transform; F is the discrete Fourier matrix; is the vector generated after Fourier transform, and ∧ is used to represent the vector generated after fast Fourier transform. i Represents the i-th column vector in the data sample set X.

[0047] Step 3: Get the feature values ​​of the current batch of samples.

[0048] Get the current batch of samples X as follows b The characteristic values ​​of:

[0049]

[0050] Where λ is the Lagrangian factor; b is the number of batch samples; yes The complex conjugate matrix of ; ⊙ is the dot product operation of the elements in the matrix; diag means converting the vector into a diagonal matrix with the main diagonal as the vector elements; is the main projection vector of the training data set X, that is, the feature vector; F H is the conjugate transpose of the Fourier matrix F, and H represents the conjugate transpose operation. According to formula (2), for each batch of randomly input samples X b , we can get:

[0051]

[0052] Among them, Λ b is the eigenvalue matrix obtained for this batch of samples. We use Λ j represents the accumulation of eigenvalues ​​after the jth batch of samples is input, and j represents the number of batches currently input. As small batches of samples are continuously input, the eigenvalue matrix Λ obtained for each batch of samples is b Add to Λ j ,

[0053] Λ j ←Λ j-1 +Λ b (4)

[0054] Among them, Λ j-1 Represents the accumulated eigenvalues ​​obtained after inputting j-1 batches of samples.

[0055] Step 4: Obtain the Fourier projection basis of the batch samples.

[0056] According to formula (2), Take it as the column vector of F, for the eigenvalue matrix Λ j The diagonal elements λ 1 ,λ 2 , ..., λ M Sort in ascending order and select the first r smallest eigenvalues ​​λ 1 ,λ 2 , ...λ r The Fourier basis in the corresponding matrix F Constructs the current projection set Wherein, r is the preset number of Fourier projection bases required, and the value here is 50.

[0057] Step 5: If the set P j With P j-1 If the same, then end the execution of steps 2 to 4 and obtain the required Fourier basis As the final Fourier projection basis. Otherwise, execute steps 2 to 4 and update the current input batch number, j←j+1.

[0058] Step 6: For the set Pj Perform an inverse Fourier transform on each Fourier projection basis i=1,...,r, obtain the projection matrix V′=[p 1 p 2 … p r ] The high-dimensional dataset X is combined with the projection matrix V′ T Multiply them together to get the reduced-dimensional data set X′=V′ T X.

[0059] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A data dimensionality reduction method based on Fourier domain principal component analysis, characterized in that: The steps include: Step 1: Data initialization, obtain sample vector x i The method is: represent the data sample set as X = [x1 x2 … x N ], the sample vector x is formed by the i-th column data samples in the data sample set X i , i = 1, 2, ... N, N represents the number of data samples; the sample vector x i Contains data samples of M dimensions in the i-th column; collects the data sample set X as the required data set, X is an M×N dimensional matrix; and initializes the current batch number j, the initial M×M dimensional zero matrix Λ0, the random Fourier basis set P0 and the discrete Fourier matrix F; M represents the dimension of the data set X, and N is the number of data samples; Step 2: Construct batch samples and Fourier data representation of batch samples, and randomly input a batch sample set X with a number b. b ∈R M×b , for X b The sample vector x in i Perform Fourier transform to get Get the current batch of samples X as follows b The characteristic value of: Where λ is the Lagrangian factor; b is the number of batch samples; yes The complex conjugate matrix of ; ⊙ is the element-wise multiplication operation of the matrix; diag means converting the vector into a diagonal matrix whose main diagonal is the vector element; is the main projection vector of the training data set X, that is, the feature vector; for each batch of randomly input samples X b , we can get Λ b : Among them, Λ b The eigenvalue matrix obtained for this batch of samples; Step 3: For each batch of randomly input batch sample set X b , calculate the batch sample set X of this batch b The obtained eigenvalue matrix Λ b , expressed as: yes The complex conjugate matrix of ; As small batch samples are continuously input, the batch sample set X of each batch b The obtained eigenvalue matrix Λ b Add to Λ j , using Λ j Represents the accumulation of feature values ​​after inputting the jth batch of samples. The process is expressed as: j ←Λ j-1 +Λ b ; Among them, Λ j-1 Indicates that when inputting j-1 batch sample set X b The characteristic values ​​obtained are accumulated; Step 4: Obtain the Fourier projection basis of the batch sample set and convert Take it as the column vector of F; for the eigenvalue matrix Λ j The diagonal elements λ1,λ2,…,λ M Sort in ascending order and select the smallest first r eigenvalues ​​λ1,λ2,…λ r The corresponding Fourier basis in the matrix F Constructs the current projection set r is the preset number of required Fourier projection bases; Step 5: If the set P j With P j-1 If the same, then end the execution of steps 2 to 4 and obtain the required Fourier basis As the final Fourier projection basis, otherwise execute steps 2 to 4 and update the current input batch number, j←j+1; Step 6: For the set P j Perform an inverse Fourier transform on each Fourier projection basis Construct the projection matrix V'=[p1 p2 ... p r ]; Combine the high-dimensional dataset X with the projection matrix V' T Multiply them together to get the reduced-dimensional data set X'=V' T X.

2. The data dimension reduction method based on Fourier domain principal component analysis according to claim 1, characterized in that: Set the threshold g, batch sample set X b The number of samples is b = N * g, b < <M。 3. The data dimension reduction method based on Fourier domain principal component analysis according to claim 2 is characterized in that: The threshold value g is between 0.5% and 5%.

4. The data dimension reduction method based on Fourier domain principal component analysis according to claim 1, characterized in that: For the sample vector x i Perform Fourier transform to get It is expressed as: in, is the generating vector of the Fourier transform, Represents the vector x i Perform fast Fourier transform, where F is the discrete Fourier matrix.

5. The data dimension reduction method based on Fourier domain principal component analysis according to claim 4, characterized in that: The Fourier domain basis vectors are: Where V is the set of projection column vectors v, that is, V = [v1 v2 … v n ], n is the number of columns of the set V; ω is a complex number and can be expressed as ω = e -2πi / M , i is the imaginary unit.

Images