Complex data prediction method and device for multi-view learning based on key information

Abstract

According to the complex data prediction method and device based on multi-view learning of key information, due to the fact that low correlation exists between features and labels in complex data such as financial table data, in the prediction method oriented to the complex data, feature selection plays an important role in the aspect of filtering the key information. In order to pay attention to key information and increase information diversity, the invention provides a strategy based on feature importance to organize a multi-view feature space, including linear correlation, nonlinear correlation and similarity measurement, further, outputs in different views are obtained through multi-layer perception to serve as local view knowledge, key local view knowledge is obtained, and the multi-view feature space is obtained. And the interference of less important information is reduced. For all features in the data, a global feature space is obtained through a multi-layer perceptron, and for the feature space, an information fusion mechanism is learned through multi-layer perception, so that effective information in the complex data can be fully utilized, and a robust and high-accuracy prediction result is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of complex data prediction, and in particular relates to a complex data prediction method and device based on multi-view learning of key information. Background Art

[0002] A lot of common information, such as user information, credit records, etc., is stored in the form of tables. Currently, tabular data is the most common information in the financial industry, such as historical information and identity representation information. However, these tabular data often face the problem of low correlation between features and labels, which means that most of the information in the features does not have a significant impact on the prediction task, and there are only a few features that have a certain correlation with the labels. In this case, traditional machine learning methods may not be able to fully utilize the effective information in the data, resulting in a decrease in prediction performance.

[0003] Existing research usually solves related problems based on feature screening, feature combination and interaction, and incremental feature methods. First, feature screening strategies are used to filter key information to reduce the interference of unimportant and redundant features on model prediction. At present, feature selection based on genetic algorithms, random forests (RF) or regularization strategies is beneficial to model performance. However, feature screening strategies based on a single perspective are prone to information loss. In addition, more feature relationships can be captured through feature combination and interaction, which can improve the performance of the model. However, this will increase the dimension of the feature space and may lead to an increase in computational complexity. At the same time, the selection of feature combinations and interactions requires certain domain knowledge or experience. Incremental feature selection gradually selects features, which can handle the correlation between features and find the feature subset that is most helpful for model prediction. This method can effectively reduce the dimension of the feature space and improve the interpretability of the model. Selection may require multiple model training and feature evaluation, which increases computational and time overhead. In addition, the results of incremental feature selection may be affected by the selection and order of the initial feature set, and may not be able to find the globally optimal feature subset. Summary of the invention

[0004] The present invention is made to solve the above problems, and aims to provide a complex data prediction method and device that can make full use of effective information in complex data, thereby improving the prediction accuracy of prediction tasks for complex data such as financial table data. The present invention adopts the following technical solutions:

[0005] The present invention provides a complex data prediction method based on multi-view learning of key information, which is characterized by comprising the following steps: a nonlinear view acquisition step: calculating the feature importance of each feature in a training set based on at least one nonlinear algorithm, and extracting a predetermined proportion of the most important features from the training set according to the feature importance, to obtain a nonlinear local view; a linear view acquisition step: calculating the feature importance of each feature in the training set based on at least one linear algorithm, and extracting a predetermined proportion of the most important features from the training set according to the feature importance, to obtain a linear local view; a similarity view acquisition step: clustering the features in the training set based on a clustering algorithm to obtain multiple clusters, and selecting the cluster with the largest number of features. A local view of multiple clusters as similarity measures; a global view acquisition step: acquiring a global view containing all features in the training set; a prediction model training step: using the training set to train a complex data prediction model, wherein the complex data prediction model includes a global multilayer perceptron for learning knowledge of the global view, a plurality of local multilayer perceptrons respectively used to learn knowledge of multiple local views, a splicing layer for splicing outputs of the global multilayer perceptron and the local multilayer perceptron, and a fully connected layer for processing the output of the splicing layer; a complex data prediction step: inputting the complex data to be predicted into the trained complex data prediction model to obtain a prediction result.

[0006] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the nonlinear view acquisition step, the information gain of each feature in the training set is calculated based on the decision tree, gradient boosting decision tree and random forest respectively, and then the feature importance of each feature is calculated based on the information gain, and the most important features of a predetermined proportion are screened out to form multiple nonlinear local views.

[0007] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the linear view acquisition step, the optimal coefficient and Pearson correlation coefficient of each feature in the training set are calculated based on logistic regression and Pearson method respectively, and then the feature importance of each feature is calculated based on the corresponding optimal coefficient and the corresponding Pearson correlation coefficient respectively, and the most important features of a predetermined proportion are screened out to form multiple linear local views,

[0008] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the nonlinear view acquisition step and the linear view acquisition step, the predetermined proportion is 30%.

[0009] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the similarity view acquisition step, the features in the training set are clustered based on the K-means algorithm, and all features are divided into 4 clusters.

[0010] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the complex data prediction model, the fully connected layer includes a first fully connected layer and a second fully connected layer, both of which are fully connected layers based on the Sigmoid activation function, the number of neurons in the second fully connected layer is smaller than that in the first fully connected layer, and the splicing layer, the first fully connected layer, and the second fully connected layer are connected in sequence.

[0011] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the complex data prediction model, the global multi-layer perceptron includes a global first fully connected layer, a global batch regularization layer, and a global second fully connected layer connected in sequence, the global first fully connected layer and the global second fully connected layer are both fully connected layers based on the Relu activation function, and the global batch normalization layer is used to batch normalize the output of the global first fully connected layer.

[0012] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein each of the local multilayer perceptrons includes a local first fully connected layer, a local batch regularization layer, a local second fully connected layer, and a local third fully connected layer connected in sequence, the local first fully connected layer and the local second fully connected layer are both fully connected layers based on the Relu activation function, the local third fully connected layer is a fully connected layer based on the Sigmoid activation function, and the local batch normalization layer is used to batch normalize the output of the local first fully connected layer.

[0013] The complex data prediction method based on multi-view learning of key information provided by the present invention may also have such a technical feature, wherein, in the prediction model training step, the cross entropy loss function is used as the loss function to optimize the complex data prediction model, and the complex data prediction model is optimized based on the Adam optimizer.

[0014] The present invention provides a complex data prediction device based on multi-view learning of key information, characterized in that it includes: a nonlinear view acquisition unit, which calculates the feature importance of each feature in a training set based on at least one nonlinear algorithm, and extracts a predetermined proportion of the most important features from the training set according to the feature importance, so as to obtain a nonlinear local view; a linear view acquisition unit, which calculates the feature importance of each feature in the training set based on at least one linear algorithm, and extracts a predetermined proportion of the most important features from the training set according to the feature importance, so as to obtain a linear local view; a similarity view acquisition unit, which clusters the features in the training set based on a clustering algorithm to obtain multiple clusters, and selects the cluster with the largest number of features as the cluster. A local view for similarity measurement; a global view acquisition unit, used to acquire a global view containing all features in the training set; a prediction model training unit, used to train a complex data prediction model using the training set, wherein the complex data prediction model includes a global multilayer perceptron for learning knowledge of the global view, a plurality of local multilayer perceptrons respectively used to learn knowledge of a plurality of local views, a splicing layer for splicing outputs of the global multilayer perceptron and the local multilayer perceptrons, and a fully connected layer for processing the output of the splicing layer; and a complex data prediction unit, used to input the complex data to be predicted into the trained complex data prediction model to obtain a prediction result.

[0015] Function and Effect of the Invention

[0016] According to the complex data prediction method and device based on multi-view learning of key information of the present invention, the method includes a nonlinear view acquisition step, a linear view acquisition step, a similarity view acquisition step, a global view acquisition step, a prediction model training step and a complex data prediction step. Since there is a low correlation between features and labels in complex data such as financial table data, feature selection plays an important role in filtering key information in the prediction method for these complex data. In order to focus on key information and increase information diversity, the present invention proposes a strategy based on feature importance to organize multi-view feature space, mainly including linear correlation, nonlinear correlation and similarity measurement. Further, the present invention obtains outputs in different views as local view knowledge through a multi-layer perceptron, thereby obtaining key local view knowledge and reducing the interference of less important information. In addition, for all features in the data, a global feature space is obtained through a multi-layer perceptron, and for all these feature spaces, an information fusion mechanism is learned through multi-layer perception, so that effective information in complex data can be fully utilized to obtain robust and highly accurate prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1is a flow chart of a complex data prediction method based on multi-view learning of key information in an embodiment of the present invention;

[0018] Figure 2 is a framework structure diagram of multi-view learning based on key information in an embodiment of the present invention;

[0019] Figure 3 is a flow chart of a nonlinear view acquisition step in an embodiment of the present invention;

[0020] Figure 4 is a flow chart of feature screening based on a decision tree in an embodiment of the present invention;

[0021] Figure 5 is a flow chart of a linear view acquisition step in an embodiment of the present invention;

[0022] Figure 6 is a flow chart of feature screening based on logistic regression in an embodiment of the present invention;

[0023] Figure 7 is a flowchart of a similarity view acquisition step in an embodiment of the present invention;

[0024] Figure 8 is a flow chart of performing feature segmentation using the K-means algorithm in an embodiment of the present invention;

[0025] Fig. 9 is a schematic diagram of the architecture of a complex data prediction model in an embodiment of the present invention;

[0026] Fig.10 4 is a block diagram of a complex data prediction device based on multi-view learning of key information in an embodiment of the present invention.

[0027] Reference numerals:

[0028] A complex data prediction device 100 based on multi-view learning of key information; a prediction model storage unit 101; a training data acquisition unit 102; a nonlinear view acquisition unit 103; a linear view acquisition unit 104; a similarity view acquisition unit 105; a global view acquisition unit 106; a prediction model training unit 107; a complex data prediction unit 108; a prediction result output unit 109; and a complex data prediction control unit 110. DETAILED DESCRIPTION

[0029] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the complex data prediction method and device based on multi-view learning of key information of the present invention are specifically described below in combination with embodiments and drawings.

[0030] <Example>

[0031] In daily life, a common concept of "strong combination" is that by combining individuals with unique advantages, favorable results can be obtained. Different joint groups usually achieve different favorable results. Similarly, in feature selection, different methods can produce different favorable views. In order to learn the multi-view information provided by these "strong combination" feature sets, this embodiment provides a complex data prediction method based on multi-view learning of key information, which is used to perform predetermined prediction tasks on input complex data.

[0032] The complex data refers to data with a large amount of data and different data sources, such as tabular data in the financial industry. In this embodiment, the method is used to predict user credit default based on tabular data in the financial industry.

[0033] Figure 1 is a flow chart of a complex data prediction method based on multi-view learning of key information in this embodiment.

[0034] Figure 2 4 is a framework diagram of multi-view learning based on key information in this embodiment.

[0035] like Figure 1 and Figure 2 As shown, the method comprises the following steps:

[0036] Non-linear view acquisition step S1: Calculate the feature importance of each feature in the training set based on multiple predetermined non-linear algorithms, and extract a predetermined proportion of the most important features from the training set according to the feature importance, so as to obtain multiple non-linear local views.

[0037] Linear view acquisition step S2: Calculate the feature importance of each feature in the training set based on multiple predetermined linear algorithms, extract a predetermined proportion of the most important features from the training set according to the feature importance, and obtain multiple linear local views.

[0038] Similarity view acquisition step S3: clustering the features in the training set based on a predetermined clustering algorithm, and selecting the cluster with the most feature data as a local view for similarity measurement.

[0039] Global view acquisition step S4: Acquire a global view containing all features in the training set.

[0040] Prediction model construction step S5: construct a complex data prediction model, which includes multiple multi-layer perceptrons, a splicing layer and a fully connected layer, wherein the multiple multi-layer perceptrons are used to learn the knowledge of the above-mentioned global view and multiple local views respectively, and the splicing layer is used to splice the outputs of multiple multi-layer perceptrons.

[0041] Prediction model training step S6: Use the training set to train the complex data prediction model, wherein the global view and multiple local views are respectively input into different multi-layer perceptrons, so that the prediction model learns multi-view knowledge.

[0042] Complex data prediction step S7: input the complex data to be predicted into the trained complex data prediction model to obtain the prediction result.

[0043] The above steps will be described in detail below.

[0044] Nonlinear field of view acquisition step S1: Calculate the feature importance of each feature of the training set based on multiple predetermined nonlinear algorithms, and extract a predetermined proportion of the most important features from the training set according to the feature importance, so as to obtain multiple nonlinear local views.

[0045] In this embodiment, decision tree (DT), gradient boosted decision tree (GBDT) and random forest (RF) are used in step S1. These algorithm models filter feature sets with nonlinear correlation importance according to information gain, which is conducive to making the complex data prediction model of this embodiment (hereinafter referred to as the prediction model) focus on the nonlinear mapping relationship between labels and features, that is, it is conducive to the prediction model to obtain a nonlinear correlation perspective.

[0046] Figure 3 FIG. 4 is a flow chart of the nonlinear view acquisition steps in this embodiment.

[0047] like Figure 3 As shown, in this embodiment, step S1 specifically includes the following sub-steps:

[0048] Step S1-1: Calculate the feature importance of each feature in the training set based on the decision tree, and select the most important features of a predetermined proportion, thereby forming a nonlinear local view.

[0049] Figure 4 It is a flowchart of feature screening based on decision tree in this embodiment.

[0050] like Figure 4 As shown, step S1-1 specifically includes the following sub-steps:

[0051] Step S1-1-1: For the training set (sample set) D = {x1, x2, ..., x n ,y}, where x1,x2,…,x n For each feature, y is the target variable, and the information gain of the target variable y is calculated:

[0052] Γ(y)=-∑(Θ(y) / l)*Θ(Θ(y) / l), (1)

[0053]

[0054] Where Θ(y) refers to the number of times positive and negative samples appear in y, and l is the number of samples in the training set used.

[0055] Step S1-1-2: For each feature x in the training set i , calculate its information gain

[0056]

[0057] In the formula, Γ(y|x i ) is a given feature x i The entropy of y.

[0058] Step S1-1-3: Among all features, select information gain The highest feature x i As the current partition node.

[0059] Step S1-1-4: Determine whether the predetermined recursive stop condition is reached.

[0060] The recursive stopping conditions include the following four situations: all samples belong to the same class; there are no more features to choose from; the maximum depth of the tree is reached; there are not enough instances to split into a node. The recursion is stopped when one of these situations is reached.

[0061] Step S1-1-5: When the judgment in step S1-1-4 is no, the selected feature x i Create a branch for each possible value of , and recursively repeat steps S1-1-1 to S1-1-4 on the data subset corresponding to each branch.

[0062] Step S1-1-6: When the judgment in step S1-1-4 is yes, the majority class of the instances in the current subset is designated as the class label of the leaf node, thereby constructing a decision tree.

[0063] Step S1-1-7: Based on the constructed decision tree, calculate each feature x i The feature importance of .

[0064] That is, the feature importance of the corresponding feature is calculated by accumulating the information gain of each feature in the process of constructing the decision tree.

[0065] Step S1-1-8: Based on each feature x iThe features with the top 30% feature importance are extracted to form a local view.

[0066] Step S1-2: Calculate the feature importance of each feature in the training set based on the gradient boosting decision tree, and select the most important features of a predetermined proportion, thereby forming a nonlinear local view.

[0067] Step S1-3: Calculate the feature importance of each feature in the training set based on the random forest, and select the most important features of a predetermined proportion, thereby forming a nonlinear local view.

[0068] The process of feature screening based on gradient boosting decision tree and random forest is similar to the specific process of step S1-1, that is, all features x in the training set are calculated based on GBDT and RF respectively. i The information gain of , and based on this information gain, the feature x is calculated i The importance of the features will not be elaborated.

[0069] Linear view acquisition step S2: Calculate the feature importance of each feature of the training set based on a plurality of predetermined linear algorithms, and extract a predetermined proportion of the most important feature set from the training set according to the feature importance to obtain a plurality of linear local views.

[0070] In this embodiment, in step S2, logistic regression (LR) and Pearson correlation coefficient are used to calculate feature importance.

[0071] Figure 5 4 is a flow chart of the linear view acquisition steps in this embodiment.

[0072] like Figure 5 As shown, in this embodiment, step S2 specifically includes the following sub-steps:

[0073] Step S2-1: Calculate the feature importance of each feature in the training set based on logistic regression, and filter out the most important features of a predetermined proportion, thereby forming a linear local view.

[0074] Figure 6 It is a flow chart of feature screening based on logistic regression in this embodiment.

[0075] like Figure 6 As shown, step S2-1 specifically includes the following sub-steps:

[0076] Step S2-1-1: For the training set D = {x1, x2, ..., x n ,y}, build the LR model:

[0077] f(x)=β0+β1·x1+β2·x2+…+β n ·x n , (4)

[0078] Where β1,β2,…,β n is the coefficient of each feature.

[0079] Step S2-1-2: Optimize the model based on the loss function and L2 regularization to obtain the optimal result of the LR model, and solve the optimal result of the LR model based on the gradient descent method to obtain the optimal feature coefficient.

[0080] In order to obtain an LR model with better generalization ability, in this embodiment, the above LR model is optimized based on L2 regularization, and its optimization target can be defined as:

[0081] J(w)=-(1 / l)*∑[y i *log(p i )+(1-y i )*log(1-p i )]+λ*||w|| 2 , (5)

[0082] Where J(β) is the objective function, which represents the combination of the loss function and the regularization term; β is the weight vector of the model, l is the number of samples, and y i is the true label of the i-th sample; p i is the probability that the i-th sample is a positive example, calculated according to the model and the weight vector; λ is the regularization parameter, which is used to control the weight of the regularization term. A larger λ value will lead to a model that tends to be a simple solution, while a smaller λ value allows the model to fit the data more freely; ||w|| 2 is the square of the L2 norm of the weight vector w.

[0083] The optimal LR model f(x) is obtained by minimizing the sum of the loss function and the regularization term, and the optimal result of the model is solved based on the gradient descent method to obtain the optimal feature coefficient The solution algorithm based on the gradient descent method is an existing technology and will not be described in detail.

[0084] Step S2-1-3: Based on the optimal feature coefficient, calculate the feature importance of each feature in the training set.

[0085] Among them, for the optimal Calculate its absolute value as the feature importance of the corresponding feature, that is,

[0086] Step S2-1-4: Based on the feature importance, the most important features of a predetermined proportion are screened out to form a linear local view.

[0087] That is, according to The values ​​are sorted by size, according to β i The absolute value of selects the top 30% of features to form a nonlinear local view.

[0088] Step S2-2: Calculate the feature importance of each feature in the training set based on the Pearson correlation coefficient, and screen out the most important features of a predetermined proportion, thereby obtaining a linear local view.

[0089] Among them, based on the Pearson correlation coefficient, the feature x is obtained i The correlation coefficient ι i for:

[0090]

[0091] In the formula, cov(x i ,y) is the feature x i and the covariance between the target vector y, σ xi , σ y Respectively represent x i , the standard deviation of y.

[0092] Then, according to the correlation coefficient ι i The absolute value of is used to select the top 30% of the features in terms of feature importance, forming a nonlinear local view.

[0093] Similarity view acquisition step S3: clustering the features in the training set based on a predetermined clustering algorithm to obtain multiple clusters, and selecting the cluster with the most feature data as the local view for similarity measurement.

[0094] Figure 7 is a flowchart of the steps of obtaining similarity views in this embodiment.

[0095] like Figure 7 As shown, in this embodiment, step S3 specifically includes the following sub-steps:

[0096] Step S3-1: For the training set D = {x1, x2, ..., x n ,y} to obtain the transposed training set, where features are used as rows and samples are used as columns.

[0097] Step S3-2: Use the K-means algorithm to divide all features in the training set into four clusters.

[0098] Figure 8 This is a flow chart of using the K-means algorithm to perform feature segmentation in this embodiment.

[0099] like Figure 8 As shown, step S3-2 specifically includes the following sub-steps:

[0100] Step S3-2-1: Select the number of clusters k=4 and randomly initialize k cluster centers.

[0101] Step S3-2-2: For each sample in the training set, calculate the distance between the sample and each cluster center, and assign the sample to the cluster corresponding to the cluster center with the closest distance.

[0102] Step S3-2-3: Update the cluster center. For each cluster center, calculate the average value of the samples belonging to the corresponding cluster, and use the average value as the new cluster center of the cluster.

[0103] Step S3-2-4: Determine whether the predetermined clustering iteration stop condition is reached. In this embodiment, the clustering iteration stop condition is that the cluster center no longer changes significantly between two consecutive iterations, or reaches a predetermined number of iterations. If the determination is no, return to step S3-2-2 for the next iteration.

[0104] Step S3-2-5: When the judgment in step S3-2-4 is yes, stop the iteration and obtain the clustering result, that is, obtain multiple clusters.

[0105] Step S3-3: Enumerate the number of features in each cluster and select the cluster with the largest number of features as the local view of the similarity measure.

[0106] Global view acquisition step S4: Acquire a global view containing all features in the training set.

[0107] This global view feature space is used to provide the possibility of global optimization for the prediction model.

[0108] Prediction model construction step S5: construct a complex data prediction model, which includes multiple multi-layer perceptrons, a concatenation layer and a fully connected layer, wherein the concatenation layer is used to concatenate the outputs of multiple multi-layer perceptrons, and the fully connected layer is used to activate the output of the concatenation layer.

[0109] Fig. 9 It is a schematic diagram of the architecture of the complex data prediction model in this embodiment.

[0110] like Fig. 9As shown, the complex data prediction model includes a global multilayer perceptron (MLP), multiple local multilayer perceptrons, a concatenation layer, a first fully connected layer ((FC(15) in the figure), and a second fully connected layer ((FC(1) in the figure). The output of the global multilayer perceptron and the output of the multiple local multilayer perceptrons are respectively connected to the input of the concatenation layer, and the concatenation layer, the first fully connected layer, and the second fully connected layer are connected in sequence.

[0111] Among them, the global multi-layer perceptron is used to learn the knowledge of the above-mentioned global view feature space, and the multiple local multi-layer perceptrons are used to learn the knowledge of the above-mentioned multiple nonlinear local view feature spaces, linear local view feature spaces, and local view feature spaces of similarity measurement. The splicing layer is used to splice the output of the global multi-layer perceptron and the output of the multiple local multi-layer perceptrons. The first fully connected layer is used to activate the output of the splicing layer, and the second fully connected layer is used to activate the output of the first fully connected layer.

[0112] The global multilayer perceptron includes a global first fully connected layer (FC(20) in the figure), a global batch normalization layer (Batch Normalization, Norm in the figure), and a global second fully connected layer (FC(5) in the figure) which are connected in sequence. The global first fully connected layer and the global second fully connected layer are both fully connected layers based on the Relu activation function.

[0113] Each local multilayer perceptron includes a local first fully connected layer ((FC(20) in the figure), a local batch normalization layer (Norm), a local second fully connected layer ((FC(10) in the figure), and a local third fully connected layer ((FC(1) in the figure)) connected in sequence. Among them, the local first fully connected layer and the local second fully connected layer are both fully connected layers based on the Relu activation function, and the local third fully connected layer is a fully connected layer based on the Sigmoid activation function.

[0114] The first fully connected layer and the second fully connected layer are both fully connected layers based on the Sigmoid activation function.

[0115] Prediction model training step S6: Use the training set to train the complex data prediction model, wherein the global view and multiple local views are respectively input into different multi-layer perceptrons, so that the prediction model learns multi-view knowledge.

[0116] Among them, during the model training process, for the input local view, the corresponding local first fully connected layer is first used to learn its complex mapping relationship, and the output of the local first fully connected layer is batch normalized (regularized) based on the local batch normalization layer:

[0117]

[0118] In the formula, F i represents the features in the multi-view space, W1 and b1 represent the weight and bias of the first fully connected layer locally, respectively. Represents the output result of the first fully connected layer locally.

[0119] Then, the predictions are obtained by locally using the second connected layer and locally using the third fully connected layer (output layer):

[0120]

[0121] Where W2 and b2 represent the weight and bias term of the second fully connected layer used locally, respectively, and W3 and b3 represent the weight and bias term of the third fully connected layer used locally, respectively.

[0122] For the global view of the input, we first learn its complex mapping relationship through the global first fully connected layer, and then batch normalize the output of the global first fully connected layer based on the global batch normalization layer:

[0123]

[0124] In the formula, F G represents the global feature, W1 and b1 represent the weight and bias term of the first fully connected layer used globally, Represents the output result of the first fully connected layer used globally.

[0125] Then, the global feature space is extracted by using the second fully connected layer globally (whose number of neurons is less than the first fully connected layer globally):

[0126]

[0127] Where W2 and b2 represent the weight and bias of the second fully connected layer used globally, respectively.

[0128] Then, the outputs of the global multilayer perceptron and multiple local multilayer perceptrons are concatenated through the concatenation layer:

[0129]

[0130] The output of the concatenated layer is then processed by the first fully connected layer to learn the importance of different elements:

[0131] F 4 =Relu(W4*F * +b4). (12)

[0132] Where W4 and b4 represent the weight and bias term of the first fully connected layer, respectively.

[0133] Finally, the output of the first fully connected layer is processed by the second fully connected layer to learn the mapping relationship between features and true labels:

[0134] y predict =sigmoid(W5*F 4 +b5). (13)

[0135] Where W5 and b5 represent the weight and bias term of the second fully connected layer, respectively.

[0136] In this embodiment, the cross entropy loss function is used as the loss function to optimize the prediction model. At the same time, the prediction model is optimized based on the Adam optimizer. Since the classification task involved in this embodiment is a binary classification, in which the class labels are 1 and 0, the real label y and F are optimized during the model training process. i 3 The cross entropy between , to learn proprietary nonlinear or linear mapping knowledge from local views:

[0137]

[0138] Complex data prediction step S7: input the complex data to be predicted into the trained complex data prediction model to obtain the prediction result.

[0139] Among them, the complex data to be predicted, such as financial table data, needs to be preprocessed and multiple views generated before being input into the prediction model. The method of generating multiple views can refer to the method of steps S1 to S4 above, so it will not be repeated here.

[0140] Fig.10 is a block diagram of a complex data prediction device based on multi-view learning of key information in this embodiment.

[0141] like Fig.10 As shown, this embodiment also provides a complex data prediction device 100 based on multi-view learning of key information corresponding to the above method, and the device includes: a prediction model storage unit 101, a training data acquisition unit 102, a nonlinear view acquisition unit 103, a linear view acquisition unit 104, a similarity view acquisition unit 105, a global view acquisition unit 106, a prediction model training unit 107, a complex data prediction unit 108, a prediction result output unit 109, and a complex data prediction control unit 110.

[0142] Among them, the prediction model storage unit 101 stores the above-mentioned complex data prediction model. The training data acquisition unit 102 is used to obtain training set data from other devices. The nonlinear view acquisition unit 103 obtains a nonlinear local view according to the method of the above-mentioned step S1. The linear view acquisition unit 104 obtains a linear local view according to the method of the above-mentioned step S2. The similarity view acquisition unit 105 obtains a local view of similarity measurement according to the method of the above-mentioned step S3. The global view acquisition unit 106 obtains a global view according to the method of the above-mentioned step S4. The prediction model training unit 107 trains the stored complex data prediction model according to the method of the above-mentioned step S6, and stores the trained prediction model in the prediction model storage unit 101 again. The complex data prediction unit 108 predicts complex data using the trained prediction model according to the method of the above-mentioned step S7 to obtain a prediction result. The prediction result output unit 109 is used to output the prediction result to a display device or other devices. The complex data prediction control unit 110 is used to control the work of the above-mentioned functional units.

[0143] In this embodiment, parts not described in detail are well-known technologies in the art.

[0144] Example Function and Effect

[0145] According to the complex data prediction method and device based on multi-view learning of key information provided by this embodiment, the method includes a nonlinear view acquisition step, a linear view acquisition step, a similarity view acquisition step, a global view acquisition step, a prediction model training step, and a complex data prediction step. Since there is a low correlation between features and labels in complex data such as financial table data, feature selection plays an important role in filtering key information in the prediction method for these complex data. In order to focus on key information and increase information diversity, the present invention proposes a strategy based on feature importance to organize multi-view feature space, mainly including linear correlation, nonlinear correlation and similarity measurement. Further, the present invention obtains outputs in different views as local view knowledge through a multi-layer perceptron, thereby obtaining key local view knowledge and reducing the interference of less important information. In addition, for all features in the data, a global feature space is obtained through a multi-layer perceptron, and for all these feature spaces, an information fusion mechanism is learned through multi-layer perception, so that effective information in complex data can be fully utilized to obtain robust and highly accurate prediction results.

[0146] In the embodiment, the feature importance of each feature in the training set is calculated based on decision tree, gradient boosting decision tree and random forest, and the top 30% of the most important features are screened out to obtain a nonlinear local view; the feature importance of each feature in the training set is calculated based on logistic regression and Pearson correlation coefficient, and the top 30% of the most important features are screened out to obtain a linear local view; the similarity measure local view is obtained based on K-means clustering, that is, the method of the embodiment can automatically screen important features to reduce the interference of unimportant information on the prediction model, and automatically optimize the focus on important information based on multi-view learning, thereby realizing data-driven key information mining, thereby improving the prediction performance of complex data.

[0147] <Comparative Example>

[0148] This comparative example provides several existing models for credit default prediction, which are used as comparative models to compare the effects of the complex data prediction model of the embodiment.

[0149] In this comparative example, the comparison models include four ensemble learning-based algorithm models in the prior art: extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), gradient boosting decision tree (GBDT), random forest (RF), and two machine learning models: decision tree (DT) and logistic regression (LR), i.e., a total of 6 comparison models. The complex data prediction model of the embodiment and the above 6 comparison models are respectively tested and evaluated for performance.

[0150] For performance testing, the test set used is the Taiwan Personal Credit Dataset, which was donated by Chung Hua University and Tamkang University in 2016. It contains 30,000 samples and 20 features. Each sample includes user information, education level, repayment record, credit card bill (including whether there is credit default), etc. This dataset is an unbalanced dataset, in which the ratio of class 0 samples to class 1 samples is 1:3.52.

[0151] For performance evaluation, this comparative example conducts overall performance evaluation and category prediction performance evaluation on each model.

[0152] Specifically, the indicators for overall performance evaluation include accuracy, area under the ROC curve (AUC), and Kolmogorov-Smirnov (KS). Accuracy is used to measure the correctness of the model's prediction, that is, the ratio of the number of samples correctly classified by the model to the total number of samples. AUC calculates the area under the ROC curve, which is not affected by the sample imbalance problem, and can use class imbalanced data sets to comprehensively evaluate model performance. KS is an evaluation indicator widely used in the financial field. It measures the risk discrimination ability of the model by calculating the maximum difference between the cumulative positive and negative examples. The calculation formula is as follows:

[0153] KS=max(TPR-FPR), (15)

[0154] Among them, TPR = TP / (TP+FN) and FPR = FP / (FP+TN) are the cumulative results of the two types of samples respectively.

[0155] In the category prediction performance evaluation, the prediction performance of 1 category of samples is evaluated based on precision, recall, and f1 score. Among them, the f1 score is the harmonic mean of Precision and Recall, taking into account the prediction accuracy of the default sample. The calculation formulas of Precision, Recall, and f1 scores are as follows:

[0156] Precision=TP / (TP+FP), (16)

[0157] Recall=TP / (TP+FN), (17)

[0158] f1=2*(Precision*Recall) / (Precision+Recall), (18)

[0159] Where TP and FP are class 1 samples with predicted labels of 1 and 0, respectively, and FN is class 0 sample with predicted labels of 1.

[0160] Table 1 below shows the comparison results of the performance evaluation of the complex data prediction model of the embodiment and the above-mentioned various comparison models.

[0161] Table 1 Performance comparison of complex data prediction model and comparison model

[0162] Prediction Model XGBoost AdaBoost GBDT RF DT LR Acc 0.813 0.817 0.816 0.814 0.818 0.807 0.809 AUC 0.76 0.772 0.772 0.767 0.777 0.723 0.716 KS 0.406 0.414 0.417 0.403 0.428 0.373 0.369 Precision 0.613 0.657 0.678 0.639 0.668 0.606 0.712 Recall 0.423 0.361 0.322 0.362 0.356 0.368 0.229 f1 0.499 0.465 0.436 0.462 0.464 0.457 0.347

[0163] From the data comparison in Table 1, we can see that in terms of the prediction performance of Class 1 samples, the Recall of the complex data prediction model of the embodiment reached 0.432, which is more than 14.9% better than the comparison model. At the same time, the Precision and f1 scores of the prediction model of the embodiment also maintained a relatively good level. From the overall performance point of view, the Acc, AUC and KS of the prediction model of the embodiment all maintained the same effect as ensemble learning, and were significantly better than DT and LR.

[0164] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A complex data prediction method based on multi-view learning of key information, characterized in that: The following steps are involved: Non-linear view acquisition step: calculating the feature importance of each feature in the training set based on at least one non-linear algorithm, and extracting a predetermined proportion of the most important features from the training set according to the feature importance, to obtain a non-linear local view; A linear view acquisition step: calculating the feature importance of each feature in the training set based on at least one linear algorithm, and extracting a predetermined proportion of the most important features from the training set according to the feature importance, to obtain a linear local view; Similarity view acquisition step: clustering the features in the training set based on a clustering algorithm to obtain multiple clusters, and selecting the cluster with the largest number of features as a local view for similarity measurement; Global view acquisition step: acquiring a global view including all features in the training set; Prediction model training step: using the training set to train a complex data prediction model, wherein the complex data prediction model includes a global multilayer perceptron for learning knowledge of the global view, a plurality of local multilayer perceptrons for learning knowledge of a plurality of local views, a splicing layer for splicing outputs of the global multilayer perceptron and the local multilayer perceptron, and a fully connected layer for processing the output of the splicing layer; Complex data prediction step: input the complex data to be predicted into the trained complex data prediction model to obtain the prediction result.

2. The complex data prediction method based on multi-view learning of key information according to claim 1, characterized in that: in, In the nonlinear view acquisition step, the information gain of each feature in the training set is calculated based on the decision tree, the gradient boosting decision tree and the random forest, and then the feature importance of each feature is calculated based on the information gain, and the most important features of a predetermined proportion are screened out to form a plurality of nonlinear local views.

3. The complex data prediction method based on multi-view learning of key information according to claim 1 is characterized in that: in, In the linear view acquisition step, the optimal coefficient and Pearson correlation coefficient of each feature in the training set are calculated based on logistic regression and Pearson method respectively, and then the feature importance of each feature is calculated based on the corresponding optimal coefficient and the corresponding Pearson correlation coefficient respectively, and the most important features of a predetermined proportion are screened out to form multiple linear local views.

4. The complex data prediction method based on multi-view learning of key information according to claim 1, characterized in that: in, In the non-linear view acquisition step and the linear view acquisition step, the predetermined ratio is 30%.

5. The complex data prediction method based on multi-view learning of key information according to claim 1 is characterized by: in, In the similarity view acquisition step, the features in the training set are clustered based on the K-means algorithm, and all the features are divided into 4 clusters.

6. The complex data prediction method based on multi-view learning of key information according to claim 1 is characterized by: in, In the complex data prediction model, the fully connected layer includes a first fully connected layer and a second fully connected layer, both of which are fully connected layers based on a Sigmoid activation function, and the number of neurons in the second fully connected layer is smaller than that in the first fully connected layer. The splicing layer, the first fully connected layer, and the second fully connected layer are connected in sequence.

7. The complex data prediction method based on multi-view learning of key information according to claim 1 is characterized by: in, In the complex data prediction model, the global multilayer perceptron includes a global first fully connected layer, a global batch normalization layer, and a global second fully connected layer connected in sequence. The first global fully connected layer and the second global fully connected layer are both fully connected layers based on the Relu activation function. The global batch normalization layer is used to perform batch normalization on the output of the global first fully connected layer.

8. The complex data prediction method based on multi-view learning of key information according to claim 1 is characterized by: in, Each of the local multilayer perceptrons includes a local first fully connected layer, a local batch normalization layer, a local second fully connected layer, and a local third fully connected layer, which are connected in sequence. The first local fully connected layer and the second local fully connected layer are both fully connected layers based on the Relu activation function. The local third fully connected layer is a fully connected layer based on the Sigmoid activation function. The local batch normalization layer is used to perform batch normalization on the output of the local first fully connected layer.

9. The complex data prediction method based on multi-view learning of key information according to claim 1, characterized in that: in, In the prediction model training step, the cross entropy loss function is used as the loss function to optimize the complex data prediction model, and the complex data prediction model is optimized based on the Adam optimizer.

10. A complex data prediction device based on multi-view learning of key information, characterized in that: include: A nonlinear view acquisition unit calculates the feature importance of each feature in a training set based on at least one nonlinear algorithm, and extracts a predetermined proportion of the most important features from the training set according to the feature importance to obtain a nonlinear local view; A linear view acquisition unit, which calculates the feature importance of each feature in the training set based on at least one linear algorithm, and extracts a predetermined proportion of the most important features from the training set according to the feature importance to obtain a linear local view; A similarity view acquisition unit, which clusters the features in the training set based on a clustering algorithm to obtain multiple clusters, and selects the cluster with the largest number of features as a local view for similarity measurement; A global view acquisition unit, used to acquire a global view including all features in the training set; A prediction model training unit, used for training a complex data prediction model using the training set, wherein the complex data prediction model includes a global multilayer perceptron for learning knowledge of the global view, a plurality of local multilayer perceptrons for learning knowledge of a plurality of local views, a splicing layer for splicing outputs of the global multilayer perceptron and the local multilayer perceptrons, and a fully connected layer for processing outputs of the splicing layer; as well as The complex data prediction unit is used to input the complex data to be predicted into the trained complex data prediction model to obtain the prediction result.

Images