A comparative principal component data analysis method based on deep neural network

Abstract

The present invention proposes a comparative principal component data analysis method based on a deep neural network, which can flexibly perform nonlinear mapping on data through training, and then extract comparative information in the data. It includes: mean centering the original data vector from which comparative information needs to be extracted, randomly selecting a batch of data from the training data, inputting the data into the neural network for forward propagation, and mapping the data to a new feature space through the target network and the background network; performing mean centering operation on the output of the target network and the background network again, calculating the sample covariance matrix, performing comparative principal component analysis, and obtaining a comparative projection direction matrix; calculating the gradient value of the loss function for the network output, iterating the weights of the target network and the background network through the back-propagation algorithm, performing forward propagation and comparative principal component analysis on all data, and projecting the network output in the comparative projection direction to obtain the comparative principal component.

Description

Technical Field

[0001] The present invention relates to a comparative principal component data analysis method based on a deep neural network, which belongs to the field of data analysis, and in particular to high-dimensional nonlinear data analysis technology generated in the fields of intelligent manufacturing, biomedicine, quantitative finance, etc. Background Art

[0002] With the continuous development of data acquisition and storage technology, a large amount of high-dimensional complex data continues to emerge in the process of intelligent manufacturing. Although high-dimensional data contains rich information, the high dimensionality also brings challenges to data analysis. Data dimensionality reduction and feature extraction technology can reduce the dimensionality of data and reduce the impact of redundant features in the data. Therefore, it occupies an important position in the process of fusion analysis of high-dimensional data.

[0003] Principal Component Analysis (PCA) is a classic feature extraction method that maximizes the variance of the transformed data by finding a linear transformation. PCA transforms the original features to obtain new features, reducing the dimension of the feature space while retaining the information contained in the original data to the greatest extent. However, the principal components obtained by principal component analysis do not necessarily contain the information that the researcher is interested in. For example, the data set to be processed is regarded as computer numerical control machine tool milling processing data, and what is wanted to extract is the relevant patterns of abnormal processing conditions, such as the workpiece clamping level or the tool wear state. If PCA is performed on only a single data set, it is more likely to produce patterns related to other information in the data set, such as feed rate, rather than the patterns that the researcher is interested in.

[0004] Abdi et al. proposed the contrastive principal component analysis method (contrastive PCA, cPCA) in the literature (Exploring patterns enriched in a dataset with contrastive principal component analysis, Nat. Commun., vol. 9, no. 1, pp. 1–7, May 2018.), which introduces a background dataset and makes it contain background information that is irrelevant to the information of interest, such as the gene expression levels of healthy people in the above example. This method looks for directions in the target data with higher variance than the background data, and projects the target data to this direction to obtain a pattern unique to the target data.

[0005] Based on the cPCA method, researchers have made various extensions. Li et al. proposed the probabilistic cPCA (PcPCA) method, which gave a probabilistic model of cPCA and improved the interpretability and robustness of cPCA to noise. Boileau et al. proposed the sparse cPCA (scPCA) method, which can extract sparse, stable, interpretable and relevant biological signals. Chen et al. proposed the discriminative PCA (dPCA) method in the literature (Nonlinear dimensionality reduction for discriminative analytics of multiple datasets. IEEE Trans. Signal Process., vol. 67, no. 3, pp. 740–752, Dec. 2018). The dPCA method combines the linear discriminant analysis method to find the idea of ​​projection that can increase the variance between classes while reducing the variance within classes, and maximize the ratio of the variance of the target data after projection to the variance of the background data. dPCA has no adjustable parameters, which avoids the consumption of time and computing resources caused by parameter adjustment. As a nonlinear extension of cPCA, kernel contrastive component analysis (KcPCA) uses a suitable kernel function to map the original data to a high-dimensional space and performs cPCA on the mapped features. However, nonlinear representation is limited by fixed kernel functions and cannot usually be selected a priori. In many practical applications, fixed kernel functions may limit the flexibility of nonlinear structure extraction. At the same time, the kernel function method is a non-parametric method that requires storing all training sample data and calculating a new kernel matrix when mapping new data. The complexity of the kernel function-based method grows roughly as a cubic function of the number of data, and faces the challenge of storage and computing resource consumption when processing large-scale data sets.

[0006] Therefore, in order to solve the above problems, it is necessary to design an efficient nonlinear feature extraction algorithm based on learning, which can flexibly perform nonlinear mapping on data through training, and then extract the contrast information in the data. So far, there is no method that can effectively improve the ability to extract contrast information from nonlinear data in the real world. Summary of the invention

[0007] The present invention proposes a comparative principal component data analysis method based on a deep neural network, which can flexibly perform nonlinear mapping on data through training, and then extract comparative information existing in the data.

[0008] The present invention is achieved through the following technical solutions.

[0009] A comparative principal component data analysis method based on a deep neural network, comprising:

[0010] Step 1: Initialize the relevant parameters of the target neural network and the background neural network;

[0011] Step 2: Mean-center the original data vector for which contrast information needs to be extracted, randomly select a batch of data from the training data, input the data into the neural network for forward propagation, and perform nonlinear mapping on the data through the target neural network and the background neural network;

[0012] Step 3: perform mean centering operation on the outputs of the target neural network and the background neural network again, calculate the sample covariance matrix, solve the contrast principal component analysis optimization problem, and obtain the contrast projection direction matrix;

[0013] Step 4: Solve the network model parameter optimization problem, calculate the gradient value of the loss function for the network output, iteratively update the parameters of the target neural network and the background neural network through the back propagation algorithm, and determine whether the number of iterations meets the requirements. If not, continue to perform the alternating optimization steps until the required number of iterations is reached. If it meets the requirements, continue;

[0014] Step 5: Perform forward propagation and comparative principal component analysis on all data, and project the network output in the comparative projection direction to obtain the comparative principal component.

[0015] Beneficial effects of the present invention:

[0016] The present invention gradually learns the optimal nonlinear transformation of target data and background data through a deep neural network. The method has both the flexibility of nonlinear representation learning and the statistical ability of mining contrast information, and can adaptively identify the rich nonlinear patterns in the target data set relative to the background data set. At the same time, it avoids the "dimensionality curse" in non-parametric methods and the limitation of hyperparameter adjustment in cPCA. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of an embodiment of a comparative principal component data analysis method based on a deep neural network of the present invention. DETAILED DESCRIPTION

[0018] The present invention is described in detail below with reference to the accompanying drawings and embodiments. The following embodiments or drawings are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0019] The implementation principle of the present invention is as follows: the comparative principal component data analysis method based on a deep neural network of the present invention is applied to a nonlinear data set, the original data is nonlinearly mapped through a deep neural network, the deep neural network model is iteratively updated according to the loss function of the comparative principal component analysis, and the optimal nonlinear mapping function and linear transformation are found, so that the variance of the target data after mapping is larger and the variance of the background data is smaller.

[0020] like Figure 1 As shown, the comparative principal component data analysis method based on a deep neural network according to a specific embodiment of the present invention specifically includes the following steps:

[0021] Step 1: Initialize the relevant parameters of the target neural network and the background neural network;

[0022] In this embodiment, the relevant parameters include network model parameters and Iteration number T, learning rate η;

[0023] Step 2: Mean-center the original data vector for which contrast information needs to be extracted, randomly select a batch of data from the training data, input the data into the neural network for forward propagation, and perform nonlinear mapping on the data through the target neural network and the background neural network;

[0024] In this embodiment, a batch of data is randomly selected from the training data and input into the neural network for forward propagation, specifically in the following manner: and background dataset Randomly select a batch of centralized data target data X bt and background data Y bt , and input the two data matrices into the target neural network and the background neural network respectively.

[0025] In this embodiment, the target neural network and the background neural network use the same network model, which can be a fully connected feedforward network, or expanded to a recurrent neural network or a residual network; specifically:

[0026] Assume that the network model contains K layers, and the kth hidden layer has c k There are O neurons in the output layer of the network (O ≥ d). For the target neural network, the output of the k-th layer network is expressed as

[0027]

[0028] in is the activation function applied in the network, is the parameter matrix of the k-th layer network;

[0029] Using DcPCA to find the parameter matrix and The target neural network and the background neural network are defined as nonlinear mapping functions respectively. and And find the direction U that maximizes the contrast variance of the target data relative to the background data.

[0030] Step 3: perform mean centering operation on the outputs of the target neural network and the background neural network again, calculate the sample covariance matrix, solve the contrast principal component analysis optimization problem, and obtain the contrast projection direction matrix;

[0031] In this embodiment, the outputs of the target neural network and the background neural network are respectively expressed as and

[0032] In this embodiment, the mean centering operation is specifically as follows:

[0033] make express The i-th column vector in , p represents the number of data in this batch, Subtract the sample mean Perform mean centering.

[0034] In this embodiment, the sample covariance matrix is ​​specifically:

[0035]

[0036]

[0037] in, It is the data after nonlinear mapping and The sample covariance matrix corresponds to the target neural network output and the background neural network output when the input data are the target data matrix and the background data matrix respectively.

[0038] In this embodiment, the comparative principal component analysis optimization problem is specifically:

[0039]

[0040] in, is the difference between the target neural network output covariance matrix and the background neural network output covariance matrix. Solving this problem is equivalent to finding C t The largest d eigenvalues ​​of The optimal solution obtained by comparing the principal component analysis method is the matrix composed of the eigenvectors corresponding to the largest d eigenvalues.

[0041] Step 4: Solve the network model parameter optimization problem, calculate the gradient value of the loss function for the network output, iteratively update the parameters of the target neural network and the background neural network through the back propagation algorithm, and determine whether the number of iterations meets the requirements. If not, continue to perform the alternating optimization steps until the required number of iterations is reached. If it meets the requirements, continue;

[0042] In this embodiment, the network model parameter optimization problem is specifically expressed as:

[0043]

[0044] Calculating the loss function For all parameters and The gradient of the loss function L for the network output and Gradient

[0045]

[0046] Perform back propagation to calculate the loss function L for all parameters and The gradient of the network is used to update the parameters of the network

[0047]

[0048] Step 5: Input all the data into the target and background neural networks for forward propagation, center the obtained data, and then perform comparative principal component analysis. Project the network output in the comparative projection direction to obtain the comparative principal component.

[0049] In this embodiment, after all the data are forward propagated and centralized, the data is represented as and The contrast projection direction is expressed as Then the final result of the deep contrast principal component data analysis method can be expressed as

Claims

1. A comparative principal component data analysis method based on deep neural network, characterized in that: The dataset to be processed is CNC machine tool milling processing data, used to extract patterns related to abnormal processing conditions; including: Step 1: Initialize the target neural network and background neural network related parameters, including network model parameters and Step 2: Mean-center the original data vectors for which contrast information needs to be extracted, and randomly select a batch of centralized data from the training data, including the target data. and background data The data is input into the neural network for forward propagation, and the data is nonlinearly mapped through the target neural network and the background neural network; the outputs of the target neural network and the background neural network are respectively expressed as and Step 3: Perform mean centering on the outputs of the target neural network and the background neural network again to calculate the sample covariance matrix and Solve the contrast principal component analysis optimization problem and obtain the contrast projection direction matrix Step 4: Solve the network model parameter optimization problem: Use the contrast projection direction matrix Constructing loss function Calculate the gradient value of the loss function for the network output, iteratively update the parameters of the target neural network and the background neural network through the back propagation algorithm, and determine whether the number of iterations meets the requirements. If not, continue to perform the alternating optimization steps, that is, reselect small batch centralized data, forward propagate to obtain the output of the target neural network and the background neural network, and perform steps three and four until the required number of iterations is reached. If the requirements are met, continue; Step 5: Perform forward propagation and comparative principal component analysis on all data, and project the network output in the comparative projection direction to obtain the comparative principal component.

2. The method according to claim 1, characterized in that The relevant parameters include network model parameters, number of iterations, and learning rate.

3. The method according to claim 2, characterized in that The method randomly selects a batch of data from the training data and inputs the data into the neural network for forward propagation, specifically in the following manner: randomly selects a batch of centralized data target data and background data from the target data set and the background data set, and inputs the two data matrices into the target neural network and the background neural network respectively.

4. The method according to claim 3, characterized in that The target neural network and the background neural network use the same network model, which is a fully connected feedforward network, or is expanded to a recurrent neural network or a residual network.

5. The method according to claim 4, characterized in that The network model is specifically: Assume that the network model contains K layers, and the kth hidden layer has c k neurons, the output layer of the network has O neurons, where O ≥ d. For the target network, the output of the k-th layer network is expressed as d indicates that the number of eigenvalues ​​to be selected in solving the optimization problem of comparative principal component analysis in step 3 is d; in is the activation function applied in the network, is the weight matrix of the k-th layer network; Using DcPCA to find the weight matrix and The target neural network and the background neural network are defined as nonlinear mapping functions respectively. and And find the direction U that maximizes the contrast variance of the target data relative to the background data.

6. The method according to claim 5, characterized in that The mean centering operation is specifically as follows: make express The i-th column vector in , i = 1, 2, ..., p, p represents the number of data in this batch, Subtract the sample mean Perform mean centering.

7. The method according to claim 6, characterized in that The sample covariance matrix is ​​specifically: in, It is the data after nonlinear mapping and The sample covariance matrix corresponds to the target neural network output and the background neural network output when the input data are the target data matrix and the background data matrix respectively.

8. The method according to claim 7, characterized in that The comparative principal component analysis optimization problem is specifically: in, is the difference between the target neural network output covariance matrix and the background neural network output covariance matrix. Solving this problem is equivalent to finding C t The largest d eigenvalues ​​of The optimal solution obtained by comparing the principal component analysis method is the matrix composed of the eigenvectors corresponding to the largest d eigenvalues.

9. The method according to claim 8, characterized in that The network model parameter optimization problem is specifically stated as: Calculating the loss function For all parameters and The gradient of the loss function L is first calculated for the network output and Gradient Perform back propagation to calculate the loss function L for all parameters and The gradient of the network is used to update the parameters of the network

Images