Parallel random forest optimization method based on gain rate and stacked autoencoders

Abstract

This invention proposes a parallel random forest optimization method based on gain rate and stacked autoencoders, including: S1, feature dimensionality reduction of the training dataset: first, calculating the feature dependency of each feature to obtain a candidate feature set, then using a redundancy filter function to filter out redundant and irrelevant features outside the candidate feature set, and using a stacked autoencoder to extract features from the dataset to obtain a reduced dimensionality dataset; S2, subspace selection: calculating the information content of the feature subspace generated by initialization of the reduced dimensionality dataset, determining whether it meets a set threshold, and reselecting feature subspaces that do not meet the set threshold; S3, parallel construction of random forests: calculating the amount of node data after each node is assigned to measure the node load, and then selecting nodes with less load to assign reduce tasks using a node allocation function; S4, inputting the test data into the random forest to obtain the final classification result. This invention significantly improves classification effect and parallel efficiency.

Description

Technical Field

[0001] The present invention relates to the field of big data mining, and in particular to a parallel random forest optimization method based on gain rate and stacked autoencoders. Background Art

[0002] The random forest algorithm is an ensemble learning method based on decision trees. It builds a classifier consisting of multiple independent decision trees by modeling them using bootstrap sampling of a dataset. The classification results from all these trees are then aggregated to produce the final result. Compared to other classification algorithms, it offers excellent classification performance, strong robustness, and high computational speed. Therefore, it has been widely used in recent years in fields such as environmental monitoring, landslide prediction, network defense, medical prognostics, fault detection, and bioinformatics.

[0003] With the rapid development of information technology, such as social media, the internet, and cloud computing, the pace of data accumulation across various industries has accelerated, leading to a rapid increase in both the volume and variety of data, resulting in the emergence of big data. Compared to traditional data, big data exhibits the four V characteristics of volume (large quantity), variety (high variety), velocity (high speed), and value (low value density). This results in higher time complexity and memory requirements for random forest algorithms for big data. Furthermore, progress in improving hardware capabilities to meet current big data processing requirements remains slow. Therefore, optimizing random forest algorithms for big data has become a key research topic.

[0004] In recent years, the MapReduce parallel processing model has garnered significant attention due to its ease of use, scalability, and low cost. By integrating it with the MapReduce framework, several optimized parallel random forest models have been successfully implemented in the field of big data processing. Among them, Lakshmanaprabu SK et al. proposed the IDARF algorithm, a random forest algorithm that combines the crossover rate with the dragonfly algorithm. This algorithm mathematically describes the five static and dynamic group behaviors of dragonflies. After evaluating individual features using an evaluation function, it uses the crossover rate to generate new individuals and iterates to approximate the optimal value to select appropriate features. IDARF outperforms traditional parallel random forests in terms of runtime and classification performance. However, this algorithm still suffers from three issues: a large number of redundant and irrelevant features in the dataset, an inability to effectively guarantee the information content of the feature subspace, and uneven node load when parallelized. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a parallel random forest optimization method based on gain rate and stacked autoencoders.

[0006] In order to achieve the above-mentioned object of the present invention, the present invention provides a parallel random forest optimization method based on gain rate and stacked autoencoder, comprising:

[0007] S1, perform feature dimensionality reduction on the training dataset: first calculate the feature dependency of each feature to obtain a candidate feature set, then use a redundant filtering function to filter out redundant and irrelevant features outside the candidate feature set, and use a stacked autoencoder to extract features from the dataset to obtain a reduced dimensionality dataset, effectively reducing the number of redundant and irrelevant features;

[0008] S2, subspace selection: Calculate the information content of the feature subspace generated by the initialization of the dimensionality reduction data set and determine whether it meets the set threshold. Reselect the feature subspace that does not meet the set threshold, effectively ensuring the information content of the feature subspace;

[0009] S3, parallel construction of random forests: After calculating the amount of node data after each node is assigned to measure the node load, the node allocation function selects the node with the lightest load to assign the Reduce task, effectively balancing the node load and improving parallelization efficiency;

[0010] S4, input the test data into the random forest to obtain the final classification result.

[0011] Furthermore, the S1 includes:

[0012] S1-1, feature selection: used to reduce the number of redundant and irrelevant features in the data set;

[0013] 1) Obtain average information gain: Calculate the information gain IG of each feature i , and then calculate the average information gain AIG based on the probability of each feature gain value;

[0014] 2) Filter irrelevant features: Obtain the gain evaluation coefficient g of each feature based on AIG i And calculate the gain rate Gr of each feature i , and then remove irrelevant features based on feature dependency FD;

[0015] 3) Filtering redundant features: The redundant filtering function RFF is used to filter out the features in the set F that have a greater impact on the dominant features in the classification, and then recombining them to obtain the optimized feature set;

[0016] S1-2, feature extraction: further extract and optimize the dataset after feature selection;

[0017] 1) Initial weight matrix and feature matrix reconstruction: Use stacked autoencoders to obtain the initial weight matrix and bias, adjust the weight matrix and bias through backpropagation to reconstruct the feature matrix, and use the SoftMax classifier for classification;

[0018] 2) Information loss and classification error estimation: The L2 norm is used to estimate the information loss and classification error;

[0019] 3) Parameter set optimization: In order to minimize the amount of information loss and the total classification error, a parameter optimization function is proposed to optimize the parameter set.

[0020] Furthermore, the average information gain AIG includes:

[0021]

[0022] Among them, IG i represents the information gain of the i-th feature;

[0023] p i is the reciprocal of the characteristic number;

[0024] q is the total number of features.

[0025] Furthermore, the feature dependency FD includes:

[0026] FD(d i ,L)=g i ×Gr(d i ,L)

[0027]

[0028] Where FD(d i ,L) is the feature d under the label set L i The feature dependency of

[0029] d i represents the i-th feature;

[0030] g i is the gain coefficient;

[0031] Gr(d i ,L) is the feature d under the label set L i The gain rate;

[0032] IG i is the information gain of the feature;

[0033] AIG is the average information gain;

[0034] k is the total number of features after removing irrelevant features.

[0035] Furthermore, the redundant filter function RFF includes:

[0036] RFF(d α ,d β )=FD(L,d α )-Gr(d α ,d β )

[0037] Where FD(L,d α ) is the dominant feature d under the label set L α The feature dependency of

[0038] Gr(d α ,d β ) represents feature d j About feature d k The gain rate;

[0039] d α represents the αth feature;

[0040] d β represents the βth feature.

[0041] Furthermore, the reconstruction of the initial weight matrix and the feature matrix includes:

[0042] First, set the initial stacked autoencoder to contain one input layer, one output layer, and two hidden layers, where the number of nodes in the two hidden layers is h and h' respectively. Then input the feature matrix D' to obtain the initial weight matrix, where the weight matrix from the input feature matrix to the first hidden layer is W1, the bias is b1, the weight matrix from the first hidden layer to the second hidden layer is W2, the bias is b2, the weight matrix from the second hidden layer to the output layer is W3, the bias is b3, and the encoding and decoding process is expressed as follows:

[0043] M1=σ(D′W1+b1)

[0044] M2=σ(M1W2+b2)

[0045] D″=σ(M2W3+b3)

[0046]

[0047] Where σ(·) is the activation function;

[0048] D″ is the reconstructed feature matrix,

[0049] M1 is the intermediate matrix of the first hidden layer,

[0050] M2 is the intermediate matrix of the second hidden layer;

[0051] Finally, after merging M2 with the label set L, we get the feature-extracted dataset DB″, and use the SoftMax classifier to classify the dataset DB″ to obtain the classification matrix C about M2. Then, we can get the predicted label f(M2) of M2 as:

[0052] f(M2)=M2C

[0053] f(M2)=σ(M1W2+b2)C

[0054] Where f(M2) is the predicted label set obtained by the reconstructed matrix D″ through SoftMax classification prediction.

[0055] Furthermore, the classification error includes:

[0056] L error =||L,f(M2)||2

[0057] Among them L error is the classification error;

[0058] L is the label set of the feature matrix D′;

[0059] f(M2) is the predicted label set obtained by SoftMax classification prediction of the reconstructed matrix D″;

[0060] ||·||2 denotes the L2 norm.

[0061] Furthermore, the parameter optimization function includes:

[0062] P(θ,C)=J(θ)+λL error

[0063]

[0064] Among them, P(θ,C) is the parameter optimization function;

[0065] J(θ) is the amount of information loss;

[0066] λ is the weight of classification error;

[0067] L error is the classification error;

[0068] s represents the feature dimension;

[0069] d i ′ is the i-th reconstructed feature;

[0070] d i is the i-th original feature;

[0071] L is the label set;

[0072] f(M2) is the predicted label set;

[0073] ||·||2 denotes the L2 norm.

[0074] Furthermore, the S2 includes the following steps:

[0075] S2-1, for feature set M * =[d1,d2,····,d h′ ] and perform sampling without replacement to obtain S feature subsets (Γ1, Γ2, ···, Γ S ), then form a feature subset set Γ containing S feature subsets; where d h′ represents the h′th feature, h′ represents the number of nodes in the second hidden layer, Γ S represents the Sth feature subset;

[0076] S2-2, the subspace information metric IC is used to measure the information content of the feature subset, and then the information content of each subset Γ is calculated. l The included characteristic variables are used to calculate IC;

[0077]

[0078] in,

[0079] Gr lj represents the jth feature in the lth feature subset;

[0080] SIC l Indicates the amount of feature information contained in the lth feature subset;

[0081] Gr i Denotes the i-th feature subset;

[0082] l=1,2,···,S is the index of the lth feature subset;

[0083] i=1,2,···,h′ is the i-th feature;

[0084] S2-3, sort the feature subsets in descending order according to the IC value, set the threshold τ to remove the feature subsets with lower information content ratio, and then re-obtain the subsets according to S2-1 and S2-2 until S feature subsets that meet the threshold τ are obtained.

[0085] S2-4, perform h′ bootstrap sampling with replacement on each of the S feature subsets that meet the threshold τ, and then obtain S feature subsets T for building the decision tree l (l=1,2,···,S).

[0086] Furthermore, the step S3 includes the following steps:

[0087] S3-1, Node Allocation: First, count the number of times each key appears in the Map process, then calculate the standard deviation of each key on each node to obtain the key standard deviation set SDK. Finally, a node distribution function (NDF) is proposed to calculate the NDF value according to the order of the keys in the set SDK and assign the key to the node with the smallest NDF value.

[0088] S3-2, build a random forest: First, call the Map function to use S feature subsets as training sets to build S sub-forests. Then, use the sub-forests to predict the dataset DB″, merge all the sub-forests to obtain a random forest model, and finally assign the prediction results of each sub-forest to the Reducer node for merging to obtain the global classification result. After comparing with the label set L, the accuracy of the model is obtained.

[0089] 11. The parallel random forest optimization method based on gain ratio and stacked autoencoders according to claim 1, wherein S3-2 comprises:

[0090] 1) First, the feature set after feature dimensionality reduction is measured using the subspace information measurement strategy to obtain S feature subsets as training sets; then the Map function is called to use key-value pairs <key T ,value T >Formally construct a decision tree (key T Number the decision tree, value T For decision tree model), after all Mapper nodes are executed, they are merged into S sub-forests; finally, the sub-forests in the Mapper node are used to predict the data set DB″ and form new key-value pairs <key′, value′> (key′ represents the sample number and the array of the corresponding category, and value′ represents the number of key-value pairs that appear), and then merge the key-value pairs with the same key′ value.

[0091] 2) First call the Map function with key-value pairs <Fkey F ,value F >Construct a global random forest (Fkey F is the sub-forest number, value F is the sub-forest model); then after all Mapper nodes are completed, all sub-forests are merged to obtain the random forest model; finally, the key-value pairs predicted in the Mapper node are assigned to the corresponding Reducer node for merging, and the global classification result is obtained and compared with the label L to obtain the accuracy of the model.

[0092] Furthermore, the node allocation function includes:

[0093]

[0094] Where NDF represents the node allocation function;

[0095] z is the total number of nodes;

[0096] nodedata j is the amount of data on node j;

[0097] key x The average amount of data on each node after being distributed to a certain node is

[0098] In summary, due to the adoption of the above technical solutions, the present invention has significantly improved the classification effect and parallel efficiency.

[0099] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0101] Figure 1 Schematic diagram of the parallel construction of random forests of the present invention.

[0102] Figure 2 This is a comparison chart of the acceleration ratios of the four algorithms.

[0103] Figure 3 This is a comparison chart of the acceleration ratios of the four algorithms. DETAILED DESCRIPTION

[0104] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0105] The present invention proposes a parallel random forest optimization method based on gain rate and stacked autoencoders. The specific embodiment is as follows, comprising the following steps:

[0106] S1, perform feature dimensionality reduction on the image training dataset: first calculate the feature dependency of each feature to obtain a candidate feature set, then use a redundant filter function to filter out redundant and irrelevant features outside the candidate feature set, and use a stacked autoencoder to extract features from the dataset to obtain a reduced-dimensional image dataset;

[0107] S2, subspace selection: Calculate the information content of the feature subspace generated by initialization of the dimensionality-reduced image dataset and determine whether it meets the set threshold, and reselect the feature subspace that does not meet the set threshold;

[0108] S3, parallel construction of random forests: After calculating the amount of node data after each node is assigned to measure the node load, the node allocation function is used to select nodes with smaller loads to assign Reduce tasks;

[0109] S4, input the image data to be tested into the random forest to obtain the final classification result.

[0110] Based on the advantages of the MapReduce programming model, this paper proposes a parallel random forest optimization method based on gain rate and stacked autoencoders, PRFGRSAE. Firstly, in the feature dimensionality reduction stage, the algorithm proposes a dimensionality reduction strategy DRGRSAE (Dimension Reduction based on Gain Ratioand Stacked Auto Encoders). After calculating the feature dependency of each feature, the candidate feature set is obtained. Then, the redundant filtering function is used to filter out the redundant and irrelevant features outside the candidate feature set. The stacked autoencoder is used to extract the features of the dataset to obtain a reduced dimensionality dataset, which effectively reduces the number of redundant and irrelevant features. Secondly, in the feature subspace selection stage, a subspace information evaluation strategy SIEBG (Subspace Information Evaluation based on Bootstrap of little bag and Gain ratio) based on BLB (Bootstrap of Little Bag) sampling method and gain ratio is proposed. After calculating the information content of the feature subspace generated by the dimensionality reduction dataset, it is judged whether it meets the set threshold. The feature subspace that does not meet the set threshold is reselected, which effectively ensures the information content of the feature subspace. Finally, a MapReduce model optimization strategy SND (Strategy of NodeDistribution calculates the amount of node data allocated to each node to measure node load. The node allocation function then selects nodes with smaller loads to assign Reduce tasks, effectively balancing node loads and improving parallelization efficiency. The algorithm proposed in this paper significantly improves both operational efficiency and classification accuracy. Furthermore, data mining using this method can provide significant benefits in biology, medicine, and target detection.

[0111] 1. Feature Dimensionality Reduction

[0112] Currently, in the parallel random forest algorithm under the big data environment, due to the increasing dimension of the data, there will be problems of feature subset redundancy and excessive number of irrelevant features when obtaining feature subsets. Therefore, a dimensionality reduction strategy DRGRSAE based on gain rate and stacked autoencoders is proposed to reduce the redundancy and number of irrelevant features in the big data environment. The main process of this strategy is as follows: (1) Feature selection process: First, the gain evaluation coefficient of each feature is obtained based on the average gain, and then the feature dependency is calculated to delete irrelevant features in parallel. Finally, redundant features are deleted according to the redundant feature function; (2) Feature extraction process: First, a feature reconstruction matrix is ​​constructed based on the stacked autoencoder, and then the classification matrix is ​​obtained using SoftMax and the reconstruction error and classification error are estimated using the L2 norm. Finally, the parameters of the reconstruction process are optimized to obtain the training data after feature extraction.

[0113] 1.1 Feature Selection

[0114] In the current big data environment, in order to reduce the number of redundant and irrelevant features in the data set, it is necessary to perform feature selection on the data set. The feature selection process is as follows: 1) Obtaining the average information gain: First, calculate the information gain IG of each feature i , and then calculate the average information gain AIG according to the probability of each feature gain value; 2) Filter irrelevant features: First, obtain the gain evaluation coefficient g of each feature according to AIG i And calculate the gain rate Gr of each feature i , and then proposed the feature dependency FD to remove irrelevant features; 3) Filtering redundant features: First, a redundant filtering function RFF is proposed to filter the features in the set F that have a greater impact on the dominant features in the classification, and then recombined to obtain the optimized feature set. Given a feature set where d q represents the qth group of features, represents a real number of p×q dimensions; category matrix Represents the category corresponding to the feature matrix D, then:

[0115] 1) Obtain average information gain

[0116] Given a feature set D, the information gain of each feature can be obtained through parallel calculation, and then the average information gain can be calculated. The process of obtaining the average information gain is as follows: first, the feature set D is divided into file blocks of uniform size according to Hadoop's default file block strategy; then Map is called to use the file blocks as input data to calculate the information gain IG of each feature. i (i=1,2,···,q); Finally, the average information gain AIG is proposed, by statistical IG iThe probability of each feature information gain value appearing can be obtained and the average information gain can be calculated to facilitate the subsequent calculation of feature dependencies and filter out irrelevant features.

[0117] Theorem 1 (Average Information Gain AIG): The information gain of each feature is known to be IG i , the probability of each feature information gain is p i , then the average information gain AIG can be calculated as follows:

[0118]

[0119] Where i=1, 2,···, q represents the number of features.

[0120] Proof: It is known that the feature set D and the information gain of each feature in the feature set D is IG i , and p i It can be seen that it is the reciprocal of the characteristic number. According to Definition 4, Hinchin's law of large numbers It can be seen that in high-dimensional data, the dimension q is high. When q is sufficiently large, the arithmetic mean and That is, the arithmetic mean With a probability close to 1, it falls within a very small range of the true mean μ. (ε is a very small value), and μ is the expectation of IG, which can be expressed as The average information gain can be obtained The proof is complete. Where P(·) represents probability, E(·) represents expectation, n is the number of elements, X i For each element value.

[0121] 2) Filter irrelevant features

[0122] After obtaining the average information gain, the gain rate can be improved to obtain feature dependency to filter out irrelevant features. The process of filtering irrelevant features is as follows: First, the gain evaluation coefficient g of each feature is obtained according to AIG i And calculate the gain rate Gr of each feature i ; Then the feature dependency FD is proposed as an indicator to filter irrelevant features, and the features with FD>0 are treated as key-value pairs. <f i ,FD i >() form to count the feature combinations and sort them in descending order, where f i is the feature name of the i-th feature; finally, the feature dependency set F = {<f1,FD1> ,<f2,FD2> ,······, <f k ,FD k >}(k is the feature dimension after filtering and 1<k≤q) and the first μ features are taken as the dominant features.

[0123] Theorem 2 (Feature Dependence FD): It is known that feature d under category L i The gain rate is Gr(d i ,L), and the corresponding gain coefficient is g i , the average information gain is AIG, then under category L, feature d i The characteristic dependence FD(d i ,L) can be calculated as follows:

[0124] FD(d i ,L)=g i ×Gr(d i ,L) (12)

[0125]

[0126] Among them, i=1,2,···,k is the number of features; j=1,2,···,q is the number of category values.

[0127] Proof: Since the gain rate tends to select features with fewer values, if you want to select appropriate features as dominant features, you need to add a gain coefficient g i To filter. And the information gain IG of the feature with fewer values i Below average AIG, ie IG i -AIG≤0, at this time, g=0, FD=0, then the features with fewer values ​​will be placed at the back of the set F during sorting and will not be selected as dominant features; while IG i When -AIG>0, let g=1 and FD=Gr, then the features with moderate values ​​can be selected as dominant features through sorting. Therefore, the feature dependency FD can be expressed as FD(d i ,L)=g i ×Gr(d i ,L), the proof is complete.

[0128] 3) Filtering redundant features

[0129] After reducing the number of irrelevant features, in order to reduce the number of redundant features in the feature set, it is necessary to filter the redundant features. The process of filtering redundant features is as follows: First, a redundant filtering function RFF is proposed to filter the set F, reducing the number of redundant features in the set that have a greater impact on the classification of the dominant features; then the optimized feature set is obtained by recombining (s is the feature dimension after feature selection and 1<s≤k), and finally the dataset DB′ obtained by column merging the feature matrix D′ and the label set (category) L is passed into the feature extraction stage.

[0130] Theorem 3 (Redundant Filter Function RFF): Given a feature pair (d α ,d β) is IG(d α ,d β ), where the dominant feature d α is indexed in set F, d β It is extracted from the set {D′-F}. Advantageous feature d α The characteristic dependence of FD(L,d α ), then feature d β The redundant filtering function can be calculated as follows:

[0131] RFF(d α ,d β )=FD(L,d α )-Gr(d α ,d β ) (14)

[0132] Among them, Gr(d α ,d β )=IG(d α ,d β ) / SI(d β ), c represents feature d β The number of values, P(·) represents the probability, Gr(d α ,d β ) represents feature d α About feature d β The gain ratio, IG(d α ,d β ) represents the feature pair (d α ,d β )’s information gain.

[0133] Proof: Feature dependence (L,d α ) is essentially the gain rate, which is used to measure the effect of the dominant feature on the classification of the category, and (d α ,d β ) is used to measure the feature d β With the advantage feature d α When Gr(d α ,d β )≤FD(L,d α ), RFF(d α ,d β )≥0, feature pair (d α ,d β ) does not affect the dominant feature d i For the classification, if it does not constitute redundancy, then retain the feature d α ; On the contrary, Gr(d α ,d β )>FD(L,dα ), RFF(d α ,d β )<0, characteristic pair (d α ,d β ) will affect the dominant feature d α Effect on classification, d β If it is a redundant feature, then feature d is removed. β , therefore, the available feature d β Redundant filter function RFF(d α ,d β )=FD(L,d α )-Gr(d α ,d β ) to filter out redundant features, the proof is complete.

[0134] 1.2 Feature Extraction

[0135] In the feature extraction stage, the dataset after feature selection is further extracted and optimized. The process is as follows: 1) Initial weight matrix and feature matrix reconstruction: Use stacked autoencoders to obtain the initial weight matrix and bias, adjust the weight matrix and bias through backpropagation to reconstruct the feature matrix, and use the SoftMax classifier for classification; 2) Information loss and classification error estimation: Use the L2 norm to estimate the information loss and classification error; 3) Parameter set optimization: In order to minimize the information loss and the sum of the classification error, a parameter optimization function P(θ, C) is proposed to optimize the parameter set, where θ = {W1, W2, W3, b1, b2, b3}, and C is the classification matrix. The specific process of feature extraction is as follows:

[0136] 1) Initial weight matrix and feature matrix reconstruction

[0137] For the feature matrix D′ and data set DB′ obtained after feature selection, the stacked autoencoder can be used to obtain the initial weight matrix and use the intermediate matrix in the reconstructed matrix for dimensionality reduction. The process is as follows: First, the initial stacked autoencoder is set to contain one input layer, one output layer and two hidden layers, where the number of nodes in the two hidden layers is h and h′ respectively. Then, the feature matrix D′ is input to obtain the initial weight matrix, where the weight matrix from the input feature matrix to the first hidden layer is in represents a real number of s×h dimensions; the bias is The weight matrix from the first hidden layer to the second hidden layer is Bias The weight matrix from the second hidden layer to the output layer is Bias The encoding and decoding process is expressed as follows:

[0138] M1=σ(D′W1+b1) (15)

[0139] M2=σ(M1W2+b2) (16)

[0140] D″=σ(M2W3+b3) (17)

[0141]

[0142] Where σ(·) is the activation function, is the reconstructed feature matrix, is the intermediate matrix of the first hidden layer, is the intermediate matrix of the second hidden layer. Finally, M2 is merged with the label set L to obtain the feature-extracted dataset DB″, and the SoftMax classifier is used to classify the dataset DB″ to obtain the classification matrix of M2 Then the predicted label f(M2) of M2 can be obtained as:

[0143] f(M2)=M2C (19)

[0144]

[0145] 2) Information loss and classification error estimation

[0146] After obtaining the classification matrix and the reconstruction matrix, in order to measure whether the reconstruction matrix is ​​as equal as possible to the original matrix, it is necessary to estimate the information loss and classification error after reconstruction. According to the classification matrix C and the reconstruction matrix D″, the information loss J(θ) and the classification error L are proposed using the L-2 norm. error Make an estimate.

[0147] Theorem 4 (Information loss J(θ)): Given the original feature d i ∈D′, reconstructed feature d i ′∈D″, the information loss J(θ) can be calculated as follows:

[0148]

[0149] Proof: Since the conversion process of the stacked autoencoder is lossy, the process matrix D′ is converted into D″ through the weight matrices W1, W2, W3, that is, D″=σ(σ 2 D′W1W2+σb1W2+σb2)W3+b3≈D′, so the characteristic d of the reconstructed matrix D″ is obtained i ′ and the characteristic d of the original characteristic matrix D′ iThere is information difference, and from Definition 3, we know that the L2 norm can calculate the Euclidean distance of each element in the two matrices to reflect the amount of information loss caused by the transformation. Therefore, the information loss of matrix D′ and reconstruction matrix D″ can be expressed as The proof is complete.

[0150] Theorem 5 (Classification Error L error ): The label set of the known feature matrix D′ is L, and the predicted label set obtained by the reconstructed matrix D″ through SoftMax classification prediction is f(M2), then the classification error L error It can be calculated as follows:

[0151] L error =||L,f(M2)||2 (22)

[0152] Proof: Since the process of converting the matrix D′ into the reconstructed feature matrix D″ through the weight matrices W1, W2, and W3 is lossy, when classifying the reconstructed feature matrix D″, the predicted label set obtained is f(M2)=σ(M1W2+b2)C, which has information difference with the label set L of the feature matrix D′. According to Definition 3, the L2 norm can calculate the Euclidean distance of each element in the two matrices to reflect the amount of information loss caused by the conversion. Therefore, the classification error L error It can be expressed as ||L,f(M2)||2. The proof is complete.

[0153] 3) Parameter set optimization

[0154] In order to make the reconstructed matrix as equal as possible to the original matrix, it is necessary to obtain the optimal weight matrix and bias, that is, it is necessary to adjust J(θ), L error To minimize, let the parameter set θ = {W1, W2, W3, b1, b2, b3}, then we can propose a parameter optimization function P(θ, C) about the parameters W1, W2, W3, b1, b2, b3, C to optimize the parameters.

[0155] Theorem 6 (Parameter Optimization Function P(θ, C)): Given the feature matrix D′ and the label set L, the reconstructed feature matrix D″ and the predicted label set f(M2) can be obtained through the θ parameter set and the classification matrix C. Then the parameter optimization function P(θ, C) can be calculated as follows:

[0156] P(θ,C)=J(θ)+λL error (twenty three)

[0157]

[0158] Among them, J(θ) is the amount of information loss, λ is the weight of classification error, and the original feature d i ∈D′, reconstructed feature d i′∈D″, L is the label set, f(M2) is the predicted label set, and s represents the feature dimension.

[0159] Proof: To find the optimal solution for P(θ, C), we essentially need to minimize the sum of the reconstruction error and the classification error, and the classification error is affected by the reconstruction error. First, we need to treat the classification matrix C as a constant, then use the gradient descent method to solve each element of the parameter set θ, and finally substitute the optimal solution of each element into the classification matrix C. For P(θ, C), we can get i The gradient of (i=1,2,3) is make The local optimal weight matrix ω can be obtained i , that is, for All have P(θ i ,C)≥P(θ ω ,C). Similarly, substitute the optimal solutions of each parameter into P(θ,C) to obtain P(θ ω ,C), then the gradient of the classification matrix C is make The local optimal classification matrix C′ can be obtained, that is, for All have P(θ ω ,C)≥P(θ ω ,C′), that is, the parameter set θ that minimizes P(θ,C) is obtained. ω And the classification matrix C′, that is, the feature extraction process loss is minimal, so the parameter optimization function can reach the optimal value. The proof is complete.

[0160] In order to obtain the global optimization parameters, it is necessary to transform the local optimal weight matrix ω obtained in Theorem 6 into i And the local optimal classification matrix C′ is substituted into the parameter optimization function P(θ,C) and iteratively solved until convergence, and the global optimal classification matrix C can be obtained. * and the global optimal weight matrix W i * , then substitute (15) (16) to obtain the feature matrix after feature extraction After column merging with the label L, the feature-extracted dataset DB″ is obtained.

[0161] 2. Subspace selection

[0162] Currently, in parallel random forest algorithms used in big data environments, subspace construction typically uses random selection or uniform selection. These methods do not adequately consider the information content of features, resulting in the constructed feature subspaces failing to fully represent the original data. Therefore, this paper proposes a subspace information measurement strategy (SIEBG) that combines the BLB sampling method with the gain rate to measure the subspace information content. This strategy selects feature subspaces by setting an information threshold to improve the subspace information content. The subspace selection process is as follows:

[0163] (1) First, the feature set Sampling without replacement is performed to obtain S feature subsets (Γ1, Γ2, ···, Γ S )(each feature subset contains b features (b<<h′, S×b<h′)), and then form a feature subset set Γ containing S feature subsets. represents a real number of p×h′ dimension, where h′ represents the number of nodes in the second hidden layer.

[0164] (2) Firstly, the subspace information metric IC is proposed to measure the information content of feature subsets, and then the subspace information metric IC is proposed according to the information content of each subset Γ. l IC is calculated based on the characteristic variables contained in (l=1,2,···,S).

[0165] Theorem 7 (Subspace Information Metric IC): Given a known feature subset Γ l The amount of information is SIC l , feature set M * The amount of information is Then the feature subset Γ l Subspace information IC l It can be calculated as follows:

[0166]

[0167] in, Gr lj represents the jth feature in the lth feature subset, l = 1, 2, ···, S is the index of the lth feature subset, and i = 1, 2, ···, h′ is the ith feature.

[0168] Proof: Let the feature set M * The total amount of information is Indicates M * The sum of the information contents of the h′ features in the equation; similarly, the information content SIC of each feature subset can be obtained, using To represent the feature subset Γ l The sum of the information content of b features in the feature set is H(M *) is unchanged, so by evaluating the information content SIC of each feature subset, we can represent the feature set M * The information content of can reflect the degree of expression of the information contained in each feature subset to the feature set. When the IC value is larger, the information contained in the feature subset has a higher degree of expression of the feature set, and the greater the effect on category classification; on the contrary, the lower the IC value is, the smaller the effect on category classification is. Therefore, the feature subset Γ l Subspace information IC l It can be expressed as The proof is complete.

[0169] (3) First, sort the feature subsets in descending order according to the IC value, and set the threshold τ to remove the feature subsets with lower information content ratio. Then, obtain the subsets again according to (1) (2) until S feature subsets that meet the threshold τ are obtained.

[0170] (4) First, perform h′ bootstrap sampling with replacement on each of the S feature subsets that meet the threshold τ, and then obtain S feature subsets T for building the decision tree. l (l=1,2,···,S).

[0171] 3. Construct random forests in parallel

[0172] Currently, in parallel random forest algorithms used in big data environments, the parallel construction of random forest algorithms often results in different predicted key-value pairs due to the different decision trees constructed in each compute node. Consequently, the number of key-value pairs on each Mapper node after merging can vary significantly, leading to uneven load on the Reducer nodes and affecting parallelization efficiency. Therefore, this paper proposes a node allocation function (NDF) to balance node load in the Reducer phase to optimize the MapReduce model. This optimized MapReduce model is then used to construct random forests in parallel, and then the predicted dataset is classified to achieve the highest accuracy. The specific process is as follows: (1) Node allocation: First, count the number of times each key appears in the Map process, then calculate the standard deviation of each key on each node to obtain the key standard deviation set SDK, and finally propose a node distribution function NDF to calculate the NDF value according to the order of the keys in the set SDK, and assign the key to the node with the smallest NDF value; (2) Construct a random forest: First, call the Map function to use S feature subsets as training sets to construct S sub-forests, then use the sub-forests to predict the data set DB″, and then merge all the sub-forests to obtain a random forest model. Finally, assign the prediction results of each sub-forest to the Reducer node for merging to obtain the global classification result, and compare it with the label set L to obtain the accuracy of the model.

[0173] (1) Node allocation

[0174] In the current parallel construction of random forests, the number of key-value pairs on each Mapper node varies greatly, resulting in uneven load on the Reducer nodes. Therefore, it is necessary to reasonably allocate the nodes in the Reducer stage to achieve balance. Assume that the key-value pair set obtained after the Mapper is merged is P1, P2, ···, P o , the node allocation process is as follows:

[0175] 1) First, all key-value pairs are saved as intermediate results in an intermediate file and sorted in descending order according to the key corresponding to the key-value pair. Then, the number of occurrences of each key is counted and saved in the form of a key-value data table (as shown in Table 1).

[0176] Table 1 Key value data table

[0177] Key Value Key1 20 Key2 18 … … Keyo 0

[0178] 2) First, analyze the key value data table and calculate the standard deviation of the data volume of each key on the node. Then, sort the key value data in descending order and generate the key standard deviation set SDK = {SDkey1, SDkey2, ···, SDkey o}

[0179] 3) First, the node allocation function NDF is proposed to calculate the NDF value of the key after it is assigned to the node according to the set SDK, and then each key is assigned to the node with the smallest NDF value to perform the Reduce operation.

[0180] Theorem 8 (Node Allocation Function NDF): Given a key x When mapping, it is assumed that the key x After being assigned to a certain node, the amount of data on node j is nodedata j , the number of nodes is z, and key x The average amount of data on each node after being distributed to a certain node is key x The node allocation function is calculated as follows:

[0181]

[0182] in

[0183] Proof: For the first key in the collection SDK x , because SDkey is large, so key x The distribution on the nodes is relatively uneven. To maintain the node load balance, the key is assigned x After reaching a certain node, the data distribution of each node should be the most even, that is, to measure the key xThe uniformity of the data distribution of each node after being assigned to a node, because the uniformity of the data of each node depends on the amount of data on node j after the key is assigned to a node. j The average amount of data on each node So only under the current allocation method each node has In order to make the node data distribution more even and make the node load balanced, that is, Therefore, we need to minimize the standard deviation of the node data volume distribution, and we can get The proof is complete.

[0184] (2) Constructing random forests in parallel

[0185] The optimized MapReduce model can be obtained through the node allocation function, and the random forest can be constructed in parallel under this model. Figure 1 The overall process is given in , and the construction process is as follows:

[0186] 1) First, the feature set after feature dimensionality reduction is measured using the subspace information measurement strategy to obtain S feature subsets as training sets; then the Map function is called to use key-value pairs <key T ,value T >Formally construct a decision tree (key T Number the decision tree, value T For decision tree model), after all Mapper nodes are executed, they are merged into S sub-forests; finally, the sub-forests in the Mapper node are used to predict the data set DB″ and form new key-value pairs <key′, value′> (key′ represents the sample number and the array of the corresponding category, and value′ represents the number of key-value pairs that appear), and then merge the key-value pairs with the same key′ value.

[0187] 2) First call the Map function with key-value pairs <Fkey F ,value F >Construct a global random forest (Fkey F is the sub-forest number, value F is the sub-forest model); then after all Mapper nodes are completed, all sub-forests are merged to obtain the random forest model; finally, the key-value pairs predicted in the Mapper node are assigned to the corresponding Reducer node for merging, and the global classification result is obtained and compared with the label L to obtain the accuracy of the model.

[0188] 4. Effectiveness of Parallel Random Forest Optimization Method Based on Gain Rate and Stacked Autoencoders

[0189] To verify the classification performance of PRFGRSAE, we applied the PRFGRSAE method to four datasets: Deepfakes, SUSY, URL, and LoT_attack. The dataset details are shown in the table below. We also compared the classification accuracy with the MRRF, IDARF, and DRF algorithms.

[0190] Table 2 Experimental dataset

[0191] Dataset name Data volume Number of features Features Deepfakes 20000 200000 Small sample size and large number of features SUSY 5000000 18 Large sample size and small number of features URL 2396130 3231961 Large sample size and many features LoT_attack 7062606 115 Large sample size and moderate number of features

[0192] 4.1 Parallel Performance Analysis of PRFGRSAE Method

[0193] In order to verify the speedup ratio of the PRFGRSAE algorithm, the PRFGRSAE algorithm, MRRF algorithm, IDARF algorithm and DRF algorithm were compared on four datasets: Deepfakes, SUSY, URL and LoT_attack. The speedup ratio was used as a measurement indicator to compare the speedup ratios of each algorithm under different numbers of nodes, and then the performance of each algorithm was compared and analyzed. The experimental results are as follows: Figure 2 As shown:

[0194] from Figure 2 As can be seen, as the number of nodes increases, the PRFGRSAE algorithm gradually achieves the highest speedup on each dataset. When the number of nodes is 5, on the Deepfakes dataset, which has a large number of features, the PRFGRSAE algorithm outperforms the MRRF, DRF, and IDARF algorithms by 0.53, 0.35, and 0.77, respectively. On the SUSY dataset, which has a large number of samples and a small number of features, the PRFGRSAE algorithm outperforms the MRRF, DRF, and IDARF algorithms by 0.5, 0.25, and 0.85, respectively. On the URL dataset, which has a large number of samples and a large number of features, the PRFGRSAE algorithm outperforms the MRRF, DRF, and IDARF algorithms by 0.64, 0.38, and 0.79, respectively. On the LoT_attack dataset, which has a large number of samples and a moderate number of features, the PRFGRSAE algorithm outperforms the MRRF, DRF, and IDARF algorithms by 0.51, 0.22, and 0.82, respectively. These results are primarily due to the use of the DRGRSAE strategy to reduce feature set redundancy and the number of irrelevant features. This significantly reduces the time required to process redundant and irrelevant features compared to the IDARF algorithm, which uses the improved dragonfly algorithm for feature selection. Furthermore, the SND strategy is used to optimize MapReduce in the parallel phase, allowing for the selection of nodes with less load to allocate Reduce tasks during merging. This effectively balances the load between nodes and improves parallelization efficiency, achieving significantly higher parallelization efficiency than the MRRF and DRF algorithms, which use only MapReduce. Therefore, compared to the aforementioned three algorithms, the PRFGRSAE algorithm exhibits superior parallel performance and robustness on large datasets.

[0195] 4.2 Analysis of classification effect of PRFGRSAE method

[0196] To verify the accuracy of the PRFGRSAE algorithm, the Kappa value was used as a measurement indicator. The PRFGRSAE algorithm, MRRF algorithm, IDARF algorithm and DRF algorithm were compared on four datasets: Deepfakes, SUSY, URL and LoT_attack. The classification results were obtained by running them 20 times respectively, and the mean value was used as the experimental result. Figure 3 Shown: From Figure 3 As can be seen in the figure, the PRFGRSAE algorithm achieved the highest Kappa value on all four datasets compared to the MRRF, DRF, and IDARF algorithms. The Kappa values ​​on the Deepfakes and URL datasets, which have a large number of features, are much higher than those of the other three algorithms. On the Deepfakes dataset, the Kappa values ​​increased by 15.1%, 5.2%, and 8.9% compared to the MRRF, DRF, and IDARF algorithms, respectively, and on the URL dataset, they increased by 19.2%, 8.8%, and 12.7%, respectively. On the LoT_attack and SUSY datasets, the Kappa values ​​did not improve significantly. On the LoT_attack dataset, which has a large number of samples and a moderate number of features, the Kappa values ​​increased by 5.5%, 3.6%, and 7.4% compared to the MRRF, DRF, and IDARF algorithms, respectively. On the SUSY dataset, which has a large number of samples and a small number of features, the Kappa values ​​increased by 7.3%, 3.5%, and 9.3%, respectively. The main reason for these results is that the PRFGRSAE algorithm effectively reduces the number of redundant and irrelevant features during the feature dimensionality reduction phase using the DRGRSAE strategy, and uses the SIEBG strategy when forming the feature subspace to ensure the information content of the feature subspace. However, the MRRF algorithm does not consider the impact of feature information content on accuracy when selecting features to construct the decision tree. IDARF uses the improved dragonfly algorithm for feature selection, which easily selects features with sufficient local information. The DRF algorithm uses uniform stratified sampling to extract features, which limits the number of features with sufficient information. Therefore, on the Deepfakes and URL datasets with a large number of features, the PRFGRSAE algorithm can achieve higher Kappa values ​​than the other three algorithms. Therefore, the PRFGRSAE algorithm has the best classification effect on datasets with large data sizes and a large number of features.

[0197] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A parallel random forest optimization method based on gain rate and stacked autoencoders, characterized in that: include: S1, performing feature dimensionality reduction on the image training dataset: first, calculating the feature dependency of each feature to obtain a candidate feature set, then using a redundant filtering function to filter out redundant and irrelevant features outside the candidate feature set, and using a stacked autoencoder to extract features from the dataset to obtain a reduced-dimensional image dataset; S1 includes: S1-1, feature selection: used to reduce the number of redundant and irrelevant features in the data set; 1) Obtain average information gain: Calculate the information gain IG of each feature i , and then calculate the average information gain AIG based on the probability of each feature gain value; 2) Filter irrelevant features: Obtain the gain evaluation coefficient g of each feature based on AIG i And calculate the gain rate Gr of each feature i , and then remove irrelevant features based on the feature dependency FD; the feature dependency FD includes: FD(d i ,L)=g i ×Gr(d i ,L) Where FD(d i ,L) is the feature d under the label set L i The feature dependency of d i represents the i-th feature; g i is the gain coefficient; Gr(d i ,L) is the feature d under the label set L i The gain rate; IG i is the information gain of the feature; AIG is the average information gain; k is the total number of features after removing irrelevant features; 3) Filtering redundant features: Using the redundant filtering function RFF, the features in the set F that have a greater impact on the dominant features in the classification are filtered out, and then recombined to obtain an optimized feature set; the redundant filtering function RFF includes: RFF(d α ,d β )=FD(L,d α )-Gr(d α ,d β ) Where FD(L,d α ) is the dominant feature d under the label set L α The feature dependency of Gr(d α ,d β ) represents feature d j About feature d k The gain rate; d α represents the αth feature; d β represents the βth feature; S1-2, feature extraction: further extract and optimize the dataset after feature selection; 1) Initial weight matrix and feature matrix reconstruction: Use stacked autoencoders to obtain the initial weight matrix and bias, adjust the weight matrix and bias through backpropagation to reconstruct the feature matrix, and use the SoftMax classifier for classification; the initial weight matrix and feature matrix reconstruction includes: First, set the initial stacked autoencoder to contain one input layer, one output layer, and two hidden layers, where the number of nodes in the two hidden layers is h and h' respectively. Then input the feature matrix D' to obtain the initial weight matrix, where the weight matrix from the input feature matrix to the first hidden layer is W1, the bias is b1, the weight matrix from the first hidden layer to the second hidden layer is W2, the bias is b2, the weight matrix from the second hidden layer to the output layer is W3, the bias is b3, and the encoding and decoding process is expressed as follows: M1=σ(D′W1+b1) M2=σ(M1W2+b2) D″=σ(M2W3+b3) Where σ(·) is the activation function; D″ is the reconstructed feature matrix, M1 is the intermediate matrix of the first hidden layer, M2 is the intermediate matrix of the second hidden layer; Finally, after merging M2 with the label set L, we get the feature-extracted dataset DB″, and use the SoftMax classifier to classify the dataset DB″ to obtain the classification matrix C about M2. Then, we can get the predicted label f(M2) of M2 as: f(M2)=M2C f(M2)=σ(M1W2+b2)C Where f(M2) is the predicted label set obtained by SoftMax classification prediction of the reconstructed matrix D″; 2) Information loss and classification error estimation: The L2 norm is used to estimate the information loss and classification error; 3) Parameter set optimization: In order to minimize the amount of information loss and the total classification error, a parameter optimization function is proposed to optimize the parameter set; S2, subspace selection: Calculate the information content of the feature subspace generated by initialization of the dimensionality-reduced image dataset and determine whether it meets the set threshold, and reselect the feature subspace that does not meet the set threshold; S3, parallel construction of random forests: After calculating the amount of node data after each node is assigned to measure the node load, the node allocation function is used to select nodes with smaller loads to assign Reduce tasks; S4, input the image data to be tested into the random forest to obtain the final classification result.

2. A parallel random forest optimization method based on gain rate and stacked autoencoders according to claim 1, characterized in that: The average information gain AIG includes: Among them, IG i represents the information gain of the i-th feature; p i is the reciprocal of the characteristic number; q is the total number of features.

3. The parallel random forest optimization method based on gain rate and stacked autoencoder according to claim 1, characterized in that: The classification errors include: <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> error <h2 style=";text-align:left;direction:ltr"> =||L,f(M2)||2 Among them L error is the classification error; L is the label set of the feature matrix D′; f(M2) is the predicted label set obtained by SoftMax classification prediction of the reconstructed matrix D″; ||·||2 denotes the L2 norm.

4. The parallel random forest optimization method based on gain ratio and stacked autoencoder according to claim 1, characterized in that: The parameter optimization function includes: P(θ,C)=J(θ)+λL error Among them, P(θ,C) is the parameter optimization function; J(θ) is the amount of information loss; λ is the weight of classification error; L error is the classification error; s represents the feature dimension; d i ′ is the i-th reconstructed feature; d i is the i-th original feature; L is the label set; f(M2) is the predicted label set; ||·||2 denotes the L2 norm.

5. The parallel random forest optimization method based on gain rate and stacked autoencoder according to claim 1, characterized in that: The S2 comprises the following steps: S2-1, for feature set M * =[d1,d2,····,d h′ ] and perform sampling without replacement to obtain S feature subsets (Γ1, Γ2, ···, Γ S ), then form a feature subset set Γ containing S feature subsets; where d h′ represents the h′th feature, h′ represents the number of nodes in the second hidden layer, Γ S represents the Sth feature subset; S2-2, the subspace information metric IC is used to measure the information content of the feature subset, and then the information content of each subset Γ is calculated. l The included characteristic variables are used to calculate IC; in, Gr lj represents the jth feature in the lth feature subset; SIC l Indicates the amount of feature information contained in the lth feature subset; Gr i Denotes the i-th feature subset; l=1,2,···,S is the index of the lth feature subset; i=1,2,···,h′ is the i-th feature; S2-3, sort the feature subsets in descending order according to the IC value, set a threshold τ to remove feature subsets with low information content ratio, and then re-obtain subsets according to S2-1 and S2-2 until S feature subsets that meet the threshold τ are obtained; S2-4, perform h′ bootstrap sampling with replacement on each of the S feature subsets that meet the threshold τ, and then obtain S feature subsets T for building the decision tree l (l=1,2,···,S).

6. The parallel random forest optimization method based on gain rate and stacked autoencoders according to claim 1, characterized in that: The S3 includes the following steps: S3-1, Node Allocation: First, count the number of times each key appears in the Map process, then calculate the standard deviation of each key on each node to obtain the key standard deviation set SDK. Finally, a node distribution function (NDF) is proposed to calculate the NDF value according to the order of the keys in the set SDK and assign the key to the node with the smallest NDF value. S3-2, build a random forest: First, call the Map function to use S feature subsets as training sets to build S sub-forests. Then, use the sub-forests to predict the dataset DB″, merge all the sub-forests to obtain a random forest model, and finally assign the prediction results of each sub-forest to the Reducer node for merging to obtain the global classification result. After comparing with the label set L, the accuracy of the model is obtained.

Images