A method, device, computer equipment and medium for acquiring feature sets

By adjusting the feature set in the deep random forest model, combining attention mechanism and Monte Carlo sampling, the problems of low training efficiency and limited accuracy of traditional models are solved, and faster convergence and higher prediction accuracy are achieved.

CN111126423BActive Publication Date: 2024-11-08BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201811296525.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-11-01
Publication Date
2024-11-08
Estimated Expiration
2038-11-01

AI Technical Summary

Technical Problem

The traditional deep random forest model has low training efficiency in the stacking stage, and its accuracy can only reach the local optimal solution, making it difficult to quickly converge to the final optimal solution.

Method used

By obtaining the input feature set of the current stack layer, comparing its classification result value with the historical maximum, if not less, increase the eigenvalue, if less, decrease the eigenvalue, adjust the feature set to speed up the convergence process, and adopting attention-based and memory-based The depth random forest model of the gate, combined with the Monte Carlo sampling algorithm, guides feature selection to avoid local optimal solutions.

Benefits of technology

The convergence speed and model training efficiency of the stacking stage in deep random forest model training are significantly improved, ensuring higher prediction accuracy and acquisition of global optimal solutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111126423B_ABST
    Figure CN111126423B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a feature set acquisition method, device, computer equipment and medium, the method comprising: acquiring the input feature set of the current stacking layer to obtain the classification result value of the current stacking layer; comparing the classification result value with the historical classification maximum value; if the classification result value is not less than the maximum value, increasing the feature value of the feature in the input feature set of the current stacking layer, and obtaining the input feature set of the next stacking layer according to the adjusted input feature set; if the classification result value is less than the maximum value, reducing the feature value of the feature in the difference set between the input feature set of the current stacking layer and the input feature set corresponding to the maximum value, and obtaining the input feature set of the next stacking layer according to the adjusted difference set, thereby accelerating the process of the stacking stage converging to the final optimal solution and improving the model training efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a feature set acquisition method, apparatus, computer equipment and medium. Background Technology

[0002] With the rise of artificial intelligence, machine learning and deep learning have been widely applied. Deep learning's strategies for deep models allow for further extraction of features and combinations of features, significantly improving the accuracy of classification and prediction. In 2006, Geoffrey Hinton, a professor at the University of Toronto and a leading figure in machine learning, and his student Ruslan Salakhutdinov published an article in *Science*, sparking a wave of deep learning adoption in academia and industry. The essence of deep learning is to learn more useful features by constructing machine learning models with many hidden layers and massive amounts of training data, ultimately improving the accuracy of classification or prediction. Therefore, "deep models" are the means, and "feature learning" is the goal.

[0003] Deep learning is a new field of machine learning research. Its motivation is to build and simulate neural networks that analyze and learn like the human brain. It mimics the mechanisms of the human brain to interpret data such as images, sounds, and text.

[0004] Existing deep model strategies mainly include the following four aspects:

[0005] 1) Deep Neural Networks (DNNs) originate from the concept of neurons. In the structure of a DNN, lower-layer neurons can form connections with all upper-layer neurons.

[0006] 2) Convolutional Neural Networks (CNNs) are mainly derived from the convolution operation of images. CNNs connect upper and lower layers through convolution kernels. The same convolution kernel is shared in all images, and the images retain their original positional relationships after the convolution operation.

[0007] 3) Recurrent Neural Networks (RNNs) are mainly used in applications involving time series. In RNNs, the output of a neuron can directly affect itself in the next time period.

[0008] 4) After fusing features, deep random forests stack multiple random forest units to finally complete the prediction and classification work.

[0009] Traditional deep model strategies such as DNN, CNN, and RNN require a large training dataset because their training process involves learning and updating many parameters. Traditional deep random forest strategies, during the stacking process, use the highest value of the stacking stage's mean as the classification or prediction result through iterative training. This process is inefficient, and the accuracy can only reach a local optimum. Summary of the Invention

[0010] This invention provides a feature set acquisition method, apparatus, computer device, and medium to accelerate the convergence to the final optimal solution during the stacking stage of deep random forest model training, thereby improving model training efficiency.

[0011] In a first aspect, embodiments of the present invention provide a method for obtaining feature sets in the stacking stage of a deep random forest model training, the method comprising:

[0012] Obtain the input feature set of the current stacking layer, and input the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer;

[0013] The classification result value of the current stacking layer is compared with the historical maximum value, where the historical maximum value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers before the current stacking layer.

[0014] If the classification result value of the current stacking layer is not less than the maximum value of the historical classification, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0015] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The input feature set of the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0016] Secondly, embodiments of the present invention also provide a feature set acquisition device for the stacking stage in the training of a deep random forest model, the device comprising:

[0017] The current result acquisition module is used to acquire the input feature set of the current stacking layer, and to obtain the classification result value of the current stacking layer by inputting the input feature set into the current stacking layer;

[0018] The classification value comparison module is used to compare the classification result value of the current stacking layer with the historical maximum classification value, wherein the historical maximum classification value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers before the current stacking layer.

[0019] The input feature set acquisition module is used to increase the feature value of the features in the input feature set of the current stacking layer if the classification result value of the current stacking layer is not less than the historical maximum value, and obtain the input feature set of the next stacking layer based on the input feature set after the feature value is increased.

[0020] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The input feature set of the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0021] Thirdly, embodiments of the present invention also provide a computer device, the device comprising:

[0022] One or more processors;

[0023] Storage device for storing one or more programs;

[0024] When the one or more programs are executed by the one or more processors, the one or more processors implement the feature set acquisition method in the stacking stage of deep random forest model training as provided in any embodiment of the present invention.

[0025] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the feature set acquisition method in the stacking stage of deep random forest model training as provided in any embodiment of the present invention.

[0026] This invention, through its embodiments, obtains the input feature set of the current stacking layer, inputs the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer, compares the classification result value of the current stacking layer with the historical maximum classification value (the maximum value among the classification result values ​​of the current stacking layer and all previous stacking layers), and if the classification result value of the current stacking layer is not less than the historical maximum classification value, then the feature values ​​of the features in the input feature set of the current stacking layer are increased, and the input feature set of the next stacking layer is obtained based on the input feature set with increased feature values. If the classification result value of the current stacking layer is less than the historical maximum classification value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum classification value is determined, and the feature values ​​of the features in the difference are decreased, and the input feature set of the next stacking layer is obtained based on the difference with decreased feature values. This accelerates the process of convergence to the final optimal solution in the stacking stage and improves the model training efficiency. Attached Figure Description

[0027] Figure 1a This is a flowchart of the feature set acquisition method in the stacking stage of deep random forest model training provided in Embodiment 1 of the present invention;

[0028] Figure 1b This is a schematic diagram of the memory gate structure in the deep random forest model provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is a flowchart of the feature set acquisition method in the stacking stage of deep random forest model training provided in Embodiment 2 of the present invention;

[0030] Figure 3a This is a flowchart of the feature set acquisition method in the stacking stage of deep random forest model training provided in Embodiment 3 of the present invention;

[0031] Figure 3b This is a schematic diagram of the training process of a deep random forest learning model in existing technology;

[0032] Figure 3c This is a schematic diagram of the training process of a deep random forest learning model provided in Embodiment 3 of the present invention;

[0033] Figure 4 This is a schematic diagram of the feature set acquisition device in the stacking stage of deep random forest model training provided in Embodiment 4 of the present invention;

[0034] Figure 5 This is a schematic diagram of the structure of the computer device provided in Embodiment 5 of the present invention. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0036] Example 1

[0037] Figure 1a This is a flowchart of a feature set acquisition method in the stacking stage of deep random forest model training, as provided in Embodiment 1 of the present invention. This method can be executed by a feature set acquisition device in the stacking stage of deep random forest model training. This device can be implemented in software and / or hardware; for example, the device can be configured in a computer device. Figure 1a As shown, the method specifically includes:

[0038] S110. Obtain the input feature set of the current stacking layer, input the input feature set into the current stacking layer, and obtain the classification result value of the current stacking layer.

[0039] In this embodiment, the concept of attention from deep learning is introduced into the training process of a random forest model, proposing an attention-based deep random forest model and a memory gate mechanism. During the training of the attention-based deep random forest model, initial features are incorporated into the feature acquisition process at each stacking layer, and a feature memory gate is added. Features are selected based on the prediction results of each stacking layer to determine which features are remembered and passed to the next layer, and which features need to be filtered. The results from the previous stacking layer guide the feature selection of the next layer, avoiding feature loss during feature fusion and accelerating the selection of the current optimal feature and convergence to the final optimal solution.

[0040] Optionally, the deep random forest model includes a feature fusion unit and a deep learning unit employing a stacking strategy, wherein the deep learning unit includes multiple cascaded stacking layers, each stacking layer containing a single random forest or a cascaded random forest.

[0041] Optionally, during the stacking phase of deep random forest model training, a single random forest model or a cascaded random forest model can be used as the prediction unit. After obtaining the input feature set of the current stacking layer, the obtained input feature set is input into the current stacking layer, and the obtained prediction result is used as the classification result value of the current stacking layer.

[0042] For example, if the input feature set F of the current stacking layer is obtained, the feature set F is input into the current stacking layer, the prediction result R output by the current stacking layer is obtained, and the obtained prediction result R is used as the classification result value of the current stacking layer.

[0043] It should be noted that the current stacking layer includes all stacking layers in the deep learning unit except for the last stacking layer. The penultimate stacking layer takes the input feature set of the last stacking layer and feeds it into the last stacking layer to obtain the final classification result value, i.e., the Final Prediction.

[0044] S120. Compare the current classification result value of the stacking layer with the historical maximum classification value.

[0045] In existing technologies, feature selection in each stacking step lacks guidance from the previous stage. Each step uses the highest value of the stacking stage's mean as the classification or prediction result through iterative training. This method is inefficient. Furthermore, if the prediction result in the current stage has some error, subsequent stages lack effective measures to correct it, making error correction difficult and leading to low accuracy in the trained model.

[0046] In this embodiment, the concept of reinforcement learning is proposed, which rewards and penalizes features based on the difference between the best predicted solution in the historical stage and the current solution. Specifically, the features in the feature set are adjusted based on the classification result value of the current stacking layer and the historical maximum classification value, and the adjusted feature set is used as the input feature set for the next stacking layer. This can avoid the error in the current stage caused by the error in the prediction result in the subsequent stage.

[0047] Optionally, the historical maximum classification value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers preceding the current stacking layer. When the current stacking layer is the first stacking layer, the historical maximum classification value refers to the classification result value of the first stacking layer.

[0048] For example, if the classification result value of the current stacking layer is R, then the maximum value among the classification result values ​​of the current stacking layer and the classification result values ​​of all stacking layers preceding the current stacking layer is R. max Then R and R max The features in the feature set are compared, and the features in the feature set are adjusted based on the comparison results.

[0049] S130. If the classification result value of the current stacking layer is not less than the historical maximum value, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0050] In this embodiment, if the classification result of the current stacking layer is not less than the historical maximum classification value, it indicates that the accuracy of the prediction result has improved during the stacked training process. This further indicates that the feature values ​​used for prediction in the current stacking layer are of high importance. Therefore, features in the input feature set of the current stacking layer are rewarded, increasing the memorization of each feature value in the input feature set of the current stacking layer, so as to gradually converge to the optimal prediction solution during training. Specifically, the feature values ​​in the input feature set of the current stacking layer are increased, and the input feature set of the next stacking layer is obtained based on the increased input feature set. The feature values ​​are used to characterize the importance of the corresponding features.

[0051] For example, if the classification result value R of the current stacking layer is not less than the maximum value R of the classification result values ​​of all stacking layers before the current stacking layer... max That is, R≥R max If the feature value is increased in the input feature set F of the current stacking layer, the input feature set of the next stacking layer is obtained based on the input feature set F after the feature value is increased.

[0052] S140. If the classification result value of the current stacking layer is less than the historical maximum value, then determine the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum value, and reduce the feature value of the feature in the difference set. Based on the difference set after the feature value reduction, obtain the input feature set of the next stacking layer.

[0053] In this embodiment, if the classification result of the current stacking layer is less than the historical maximum classification value, it indicates that the accuracy of the prediction result has decreased during the stacked training process. This suggests that the feature values ​​used for prediction in the current stacking layer are less important. Therefore, a penalty is applied to the features in the difference set between the current stacking layer's input feature set and the input feature set corresponding to the historical maximum classification value, reducing the memorization of each feature value in the difference set. Specifically, the difference set between the current stacking layer's input feature set and the input feature set corresponding to the historical maximum classification value is calculated. The feature values ​​of the features in the difference set are reduced, and the input feature set for the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0054] For example, if the classification result value R of the current stacking layer is less than the maximum value R of the classification result values ​​of all stacking layers before the current stacking layer... max That is, R < R max Then obtain the feature set F corresponding to the historical maximum value of the classification. max And calculate F and F max The difference set △F is used to reduce the eigenvalues ​​of the features in △F, and the input feature set of the next stacking layer is obtained based on the difference set △F after the eigenvalues ​​are reduced.

[0055] This invention, through its embodiments, obtains the input feature set of the current stacking layer, inputs the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer, compares the classification result value of the current stacking layer with the historical maximum classification value, and if the classification result value of the current stacking layer is not less than the historical maximum classification value, then the feature values ​​of the features in the input feature set of the current stacking layer are increased, and the input feature set of the next stacking layer is obtained based on the input feature set with increased feature values. If the classification result value of the current stacking layer is less than the historical maximum classification value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum classification value is determined, and the feature values ​​of the features in the difference set are decreased, and the input feature set of the next stacking layer is obtained based on the difference set with decreased feature values. This accelerates the process of convergence to the final optimal solution in the stacking stage and improves the model training efficiency.

[0056] Based on the above scheme, the step of increasing the feature values ​​of the features in the input feature set of the current stacking layer includes:

[0057] Multiply the feature values ​​of each feature in the input feature set of the current stacking layer by a first preset coefficient, which is a value greater than 1;

[0058] Accordingly, reducing the eigenvalues ​​of the features in the difference set includes:

[0059] The eigenvalues ​​of each feature in the difference set are multiplied by a second preset coefficient, which is a value less than 1.

[0060] In this embodiment, a first preset coefficient K1 can be preset to increase the feature value of a feature, and a second preset coefficient K2 can be preset to decrease the feature value of a feature. The first preset coefficient K1 is a value greater than 1, and the second preset coefficient K2 is a value less than 1. Optionally, the first preset coefficient K1 is 1.5, and the second preset coefficient K2 is 0.5. When it is necessary to increase the feature value of each feature in the input feature set of the current stacking layer, the feature value of each feature in the input feature set of the current stacking layer is multiplied by the first preset coefficient K1. When it is necessary to decrease the feature value of each feature in the difference set, the feature value of each feature in the difference set is multiplied by the second preset coefficient K2.

[0061] For example, if the classification result value R of the current stacking layer is not less than the maximum value R of the classification result values ​​of all stacking layers before the current stacking layer... max That is, R≥R max Then, the feature values ​​of each feature in the input feature set F of the current stacking layer are multiplied by K1 to increase the feature values ​​of each feature in the input feature set F of the current stacking layer. If the classification result R of the current stacking layer is less than the maximum value R of the classification results of all stacking layers before the current stacking layer... max That is, R < R max Then obtain the feature set F corresponding to the historical maximum value of the classification. max And calculate F and F max The difference set △F is obtained by multiplying the eigenvalues ​​of each feature in the difference set △F by K2 to reduce the eigenvalue set of each feature in △F.

[0062] Based on the above scheme, obtaining the input feature set of the next stacking layer from the input feature set after the feature values ​​are increased includes:

[0063] The Monte Carlo sampling algorithm is used to sample the input feature set after the feature values ​​are increased. The feature set formed by the sampled features is used as the input feature set of the next stacking layer.

[0064] Accordingly, obtaining the input feature set of the next stacking layer based on the difference set after the feature values ​​are reduced includes:

[0065] The Monte Carlo sampling algorithm is used to sample the difference set after the eigenvalues ​​are reduced. The feature set formed by the sampled features is used as the input feature set of the next stacking layer.

[0066] In this embodiment, a Monte Carlo sampling algorithm can be used to obtain the input feature set for the next stacking layer based on the adjusted feature values. Using the Monte Carlo algorithm to sample features according to the feature memory distribution of the current layer can prevent the training process from getting stuck in local optima, enabling the training to obtain the global optimum as much as possible.

[0067] For example, if the classification result value R of the current stacking layer is not less than the maximum value R of the classification result values ​​of all stacking layers before the current stacking layer... max That is, R≥R max Then, the feature values ​​of each feature in the input feature set F of the current stacking layer are multiplied by K1, and the Monte Carlo sampling algorithm is used to sample the input feature set F after the feature values ​​are increased. The sampled features form the feature set F. out This serves as the input feature set for the next stacking layer. If the classification result R of the current stacking layer is less than the maximum R among the classification results of all previous stacking layers... max That is, R < R max Then obtain the feature set F corresponding to the historical maximum value of the classification. max And calculate F and F max The difference set ΔF is obtained by multiplying the eigenvalues ​​of each feature in ΔF by K2, and then using the Monte Carlo sampling algorithm to sample the features of the reduced difference set ΔF. The sampled features form the feature set F. out This serves as the input feature set for the next stacking layer.

[0068] Optionally, the classification result of the current stacking layer can be compared with the historical maximum classification value, and the feature set can be adjusted according to the comparison result. The input feature set of the next stacking layer can be obtained based on the adjusted feature set. This can be achieved through a memory gate structure.

[0069] Figure 1b This is a schematic diagram of the memory gate structure in the deep random forest model provided in Embodiment 1 of the present invention. Figure 1b The diagram illustrates the process of training an attention-based deep random forest model using a memory gate structure. Figure 1b As shown, A is a judgment structure used to determine whether the current predicted value R of the stacking layer is not less than the historical maximum predicted value R of the stacking layer.max Specifically, the predicted maximum value R of the historical stacking layer is... max And the predicted value R of the current stacking layer is input to A, when R ≥ R max When the judgment result is "Yes", proceed to C, input the current feature set F, and multiply the feature values ​​of the current feature set F by the reward term γ>1; when R <R max When the judgment result is "No", proceed to B, and combine the feature set F corresponding to the current stacking layer with the current optimal state R. max The corresponding feature set F max Input B, and obtain the current feature set F and Fi from B. max After obtaining the difference set, the eigenvalues ​​in the difference set are multiplied by a penalty term γ<1. Finally, the sample is fed into the D Monte Carlo sampling structure, where the feature set F multiplied by the reward term γ>1 or the difference set multiplied by the penalty term γ<1 is input into D to obtain the feature set F output by D. out , will F out This serves as the feature set used for prediction in the next stacking layer.

[0070] Example 2

[0071] Figure 2 This is a flowchart of the feature set acquisition method in the stacking stage of deep random forest model training provided in Embodiment 2 of the present invention. This embodiment further optimizes the above embodiments. Figure 2 As shown, the method includes:

[0072] S210. Obtain the feature matrix output after feature fusion in the feature fusion stage before the stacking stage, and merge the feature matrix with the original feature matrix used during feature fusion to obtain the merged feature set.

[0073] Generally, deep random forest models typically use the predictions from cascaded random forests as the fused features during training. However, directly using the predictions from cascaded random forests as the fused features can lead to feature loss during training and may not achieve high prediction accuracy during the stacking phase of deep learning.

[0074] In this embodiment, the original features are added during the feature fusion stage before the features are input to the stacking stage. The fused features are then merged with the original features, and the merged features are then input together into the stacking stage. Merging the fused features with the original features prevents feature loss during the feature fusion process and increases the feature dimensionality.

[0075] For example, if the feature matrix output after feature fusion in the feature fusion stage between stacking stages is F mr The original feature matrix is ​​F raw Then the characteristic matrix F mr With the original feature matrix F raw Merge.

[0076] S220. Obtain the feature values ​​of the features in the merged feature set through cascaded random forest. Filter the features in the merged feature set according to the feature values ​​to obtain the input feature set of the first stacking layer. Use the first stacking layer as the current stacking layer.

[0077] Optionally, after merging the fused feature matrix with the original feature matrix, all features in the merged feature set are filtered through a cascaded random forest. The feature set composed of the filtered features is used as the input feature set of the first stacking layer, and the first stacking layer is used as the current stacking layer. Training is then performed based on the input feature set of the first stacking layer.

[0078] Specifically, the cascaded random forest is used to obtain the feature values ​​of all features in the merged feature set. Then, based on these feature values, each feature in the merged feature set is filtered, and the filtered features form a feature set, which serves as the input feature set for the first stacking layer. Optionally, all features in the merged feature set are input into the cascaded random forest to obtain the features output by the cascaded random forest. This feature set is then used as the input feature set for the first stacking layer.

[0079] S230. Input the input feature set of the current stacking layer into the current stacking layer to obtain the classification result value of the current stacking layer.

[0080] S240. Compare the current classification result value of the stacking layer with the historical maximum classification value.

[0081] S250. If the classification result value of the current stacking layer is not less than the historical maximum value, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0082] S260. If the classification result value of the current stacking layer is less than the historical maximum value, then determine the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum value, and reduce the feature value of the feature in the difference set. Based on the difference set after the feature value reduction, obtain the input feature set of the next stacking layer.

[0083] S270. Set the next stacking layer as the current stacking layer and return to execute S230.

[0084] Optionally, after obtaining the input feature set of the next stacking layer, the next stacking layer is used as the current stacking layer, and S230 is executed to select the feature set and train the model.

[0085] The technical solution of this invention, based on the above embodiments, adds an operation to merge the original feature matrix with the feature matrix output after feature fusion in the feature fusion stage before the stacking stage. By merging the feature fused features with the original features, feature loss is prevented and the feature dimension is increased.

[0086] Example 3

[0087] Figure 3a This is a flowchart of the feature set acquisition method in the stacking stage of deep random forest model training provided in Embodiment 3 of the present invention. This embodiment further optimizes the above embodiments. Figure 3a As shown, the method includes:

[0088] S310. Obtain the original feature matrix, split the original feature matrix to obtain at least two sub-feature matrices, and input the at least two sub-feature matrices into a single random forest or a cascaded random forest respectively to obtain the fused feature matrix output by the single random forest or the cascaded random forest.

[0089] In this embodiment, feature fusion is specifically implemented during the feature fusion stage. Optionally, during feature fusion, the original feature matrix is ​​first split into multiple sub-feature matrices, and then each sub-feature matrix is ​​input into a single random forest or a cascaded random forest to obtain the fused feature matrix output by the single random forest or the cascaded random forest. Optionally, the original feature matrix can be split into at least two sub-feature matrices by sliding a window.

[0090] For example, the original feature matrix is ​​split into sub-feature matrices M1, M2, and M3. Sub-feature matrices M1, M2, and M3 are then input into a single random forest or a cascaded random forest, respectively, to obtain fused feature matrices F1, F2, and F3 output by the single random forest or the cascaded random forest, which correspond to the sub-feature matrices M1, M2, and M3, respectively.

[0091] Optionally, the step of splitting the original feature matrix to obtain at least two sub-feature matrices, and inputting the at least two sub-feature matrices into a single random forest or a cascaded random forest respectively to obtain the fused feature matrix output by the single random forest or the cascaded random forest includes:

[0092] The original feature vector of dimension m is passed through P windows to obtain P sub-feature matrices, where the dimension of the i-th sub-feature matrix is ​​(mm / n). i +1)*m / n i , where P is an integer greater than 1, and i takes values ​​in [1, P];

[0093] The P sub-feature matrices are each input into a single random forest or a cascaded random forest to obtain P fused feature matrices output by each single random forest or cascaded random forest, where the dimension of the i-th fused feature matrix is ​​(mm / n). i +1)*n class ;

[0094] Where, n i Used to control the size of the corresponding window, n i Takes values ​​between (1, m), n class This indicates the number of categories the target is divided into.

[0095] In this embodiment, the original feature vector is split into sub-feature matrices by sliding windows. Each sub-feature matrix is ​​then input into a single random forest or a cascaded random forest to obtain a fused feature matrix output by each single or cascaded random forest. Optionally, P sub-feature matrices are obtained by sliding P windows, and these P sub-feature matrices are input into a single or cascaded random forest to obtain P fused feature matrices output by each single or cascaded random forest.

[0096] For example, the original feature dimension is m. After sliding through windows of m / n1, m / n2, and m / n3 respectively, sub-feature matrices with feature dimensions of (mm / n1+1)*m / n1, (mm / n2+1)*m / n2, and (mm / n3+1)*m / n3 are obtained. Each sub-feature matrix is ​​then input into a single random forest or a cascaded random forest to obtain the feature dimension of (mm / n1+1)*m / n output by the single random forest or the cascaded random forest. class (mm / n2+1)*m / n class and (mm / n3+1)*m / n class The fusion feature matrix.

[0097] S320. Convert the obtained fusion feature matrices into one-dimensional feature matrices, which are used as the feature matrices output by the feature fusion unit.

[0098] In this embodiment, the multidimensional fusion feature matrix is ​​converted into a one-dimensional feature matrix, and the converted one-dimensional feature matrix is ​​used as the feature matrix output by the feature fusion unit.

[0099] For example, if the obtained fusion feature matrices are fusion feature matrix F1, fusion feature matrix F2, and fusion feature matrix F3, then fusion feature matrix F1, fusion feature matrix F2, and fusion feature matrix F3 are converted into one-dimensional feature matrices respectively and then merged to obtain a one-dimensional fusion feature matrix F. mr The one-dimensional fusion feature matrix F mr The feature matrix is ​​output as a feature fusion unit.

[0100] For example, if F1 = [1, 2], F2 = [3, 4; 5, 6; 7, 8; 9, 10], and F3 = [11, 12; 13, 14; 15, 16; 17, 18; 19, 20], then converting F1, F2, and F3 into one-dimensional matrices and merging them yields a one-dimensional matrix F. mr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18; 19, 20], and set F mr The feature matrix is ​​output as a feature fusion unit.

[0101] S330. The feature matrix is ​​merged with the original feature matrix used in the feature fusion unit to obtain the merged feature set.

[0102] S340. Obtain the feature values ​​of all features in the merged feature set through a cascaded random forest. Filter the features in the merged feature set based on the feature values ​​to obtain the input feature set of the first stacking layer. Use the first stacking layer as the current stacking layer.

[0103] S350. Input the input feature set of the current stacking layer into the current stacking layer to obtain the classification result value of the current stacking layer.

[0104] S360. Compare the current classification result value of the stacking layer with the historical maximum classification value.

[0105] S370. If the classification result value of the current stacking layer is not less than the historical maximum value, then increase the feature value of the feature in the input feature set of the current stacking layer, and obtain the input feature set of the next stacking layer based on the input feature set after the feature value is increased.

[0106] S380. If the classification result value of the current stacking layer is less than the historical maximum value, then determine the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum value, and reduce the feature value of the feature in the difference set. Based on the difference set after the feature value reduction, obtain the input feature set of the next stacking layer.

[0107] S390: Set the next stacking layer as the current stacking layer and return to execute S350.

[0108] Optionally, after obtaining the input feature set of the next stacking layer, the next stacking layer is used as the current stacking layer, and S230 is executed to select the feature set and train the model.

[0109] Figure 3b This is a schematic diagram of the training process of a deep random forest learning model in existing technology. Figure 3b In the deep random forest learning model, the cascaded random forest (105b, 106b, 107b) is used as the prediction unit. After obtaining the feature matrix through a sliding window, the obtained feature matrix is ​​input into the prediction unit. The obtained prediction result 111b is used as the feature after feature fusion. Then, the stacking strategy is combined to perform deep learning on the fused feature.

[0110] like Figure 3b As shown, the training process of a deep random forest learning model in the prior art involves feature fusion (10 bits), feature selection, and stacking (30 bits), resulting in a final prediction result (40 bits). Figure 3bAs can be seen, the original feature 101b has a dimension of m. In the input model, it first passes through windows m / n1, m / n2, and m / n3, resulting in sub-feature matrices 102b ((mm / n1+1)*m / n1), 103b ((mm / n2+1)*m / n2), and 104b ((mm / n3+1)*m / n3) with feature dimensions of (mm / n1+1)*m / n1, respectively. Sub-feature matrix 102b is then input into the cascaded random forest 105b (including the random forest Fo). The sub-feature matrix 103b is input into the cascaded random forest 106b (including random forests ForestB1 and ForestB2), and the sub-feature matrix 104b is input into the cascaded random forest 107b (including random forests ForestC1 and ForestC2). After feature fusion in each cascaded random forest, the feature dimension output by the cascaded random forest 105b is (mm / n1+1)*m / n. class The feature dimension output by the fused feature matrix 108b and the cascaded random forest 106b is (mm / n²+1)*m / n class The feature dimension output by the fused feature matrix 109b and the cascaded random forest 107b is (mm / n³+1)*m / n class The fused feature matrix 110b is then transformed into a one-dimensional matrix form, resulting in a one-dimensional fused feature matrix 111b, completing the feature fusion stage 10b of the entire preliminary process. Here, n1, n2, and n3 control the window size, and the sliding step size can be adjusted to increase the randomness of feature selection. class This indicates the number of categories the target is divided into.

[0111] Optionally, after completing the initial feature fusion, the feature matrix obtained after feature fusion is input into the feature filtering stage. Optionally, the features in the one-dimensional fused feature matrix 111b are filtered using a cascaded random forest 20b to obtain the filtered features, and the feature set composed of the filtered features is output. The feature set output by the cascaded random forest 20b is used as the input feature set of the first stacking layer. After obtaining the input feature set of the first stacking layer 301b, the input feature set of the first stacking layer is input into the first stacking layer 301b to obtain the classification result value of the first stacking layer. Based on the classification result value of the first stacking layer, the input feature set of the second stacking layer 302b is obtained. This process of deep stacking training with stacked layers is continuously performed until the final prediction result 40b output by the last stacking layer 303b is obtained. The idea of ​​convolutional window, feature selection strategy of random forest, and stacking model fusion are used to realize the prediction of events. Optionally, the cascaded random forest 20b includes multiple random forests.

[0112] Figure 3c This is a schematic diagram of the training process of a deep random forest learning model provided in Embodiment 3 of the present invention. Figure 3c In this process, the cascaded random forest (105c, 106c, 107c) is used as the prediction unit. After obtaining the feature matrix through a sliding window, the obtained feature matrix is ​​input into the prediction unit to obtain the prediction result. The original feature 112c and the feature fusion feature 111c are merged. Then, the fused feature is learned in depth by combining the stacking strategy. While continuously stacking, the prediction results of each stacking layer are combined with the memory gate M to obtain appropriate features. The prediction of the next stacking layer is guided by combining the historical prediction and the prediction of the current stacking layer.

[0113] like Figure 3c As shown, the deep random forest learning model based on attention provided in this embodiment of the invention undergoes feature fusion (10c), feature selection, and stacking (30c) during training to obtain the final prediction result (40c). Figure 3cAs can be seen, the original feature 101c has a dimension of m. In the input model, it first passes through windows m / n1, m / n2, and m / n3, resulting in sub-feature matrices 102c (mm / n1+1)*m / n1, 103c (mm / n2+1)*m / n2, and 104c (mm / n3+1)*m / n3, respectively. Sub-feature matrix 102c is then input into the cascaded random forest 105c (including the random forest Fo). The sub-feature matrix 103c is input into the cascaded random forest 106c (including random forests ForestB1 and ForestB2), and the sub-feature matrix 104c is input into the cascaded random forest 107c (including random forests ForestC1 and ForestC2). After feature fusion in each cascaded random forest, the feature dimension output by the cascaded random forest 105c is (mm / n1+1)*m / n. class The feature dimension output by the fused feature matrix 108c and the cascaded random forest 106c is (mm / n²+1)*m / n class The feature dimension output by the fused feature matrix 109c and the cascaded random forest 107c is (mm / n³+1)*m / n class The fused feature matrix 110c is obtained by converting fused feature matrices 108c, 109c, and 110c into one-dimensional matrix form, resulting in a one-dimensional fused feature matrix 111c. The one-dimensional feature matrix 111c is F. mr And obtain the original feature matrix F raw The one-dimensional fused feature matrix 111c is merged with the original feature matrix 112c, i.e., the original feature 101c, to obtain the merged feature matrix, completing the feature fusion stage of the entire preliminary process. Here, n1, n2, and n3 control the window size, and the sliding step size can be adjusted to increase the randomness of feature selection. class This indicates the number of categories the target is divided into.

[0114] Optionally, after completing the initial feature fusion, the merged feature matrix is ​​input into the feature filtering stage. Optionally, the features of the merged feature matrix are filtered using a cascaded random forest 20c to obtain filtered features, and the feature set composed of the filtered features is output. The feature set output by the cascaded random forest 20c is used as the input feature set of the first stacking layer. The input feature set of the first stacking layer 301c is input into the first stacking layer 301c to obtain the classification result value of the first stacking layer 301c. Based on the classification result value of the first stacking layer 301c and the historical maximum classification value, the input feature set of the second stacking layer 302c is determined. The input feature set of the second stacking layer 302c is then input into the second stacking layer 302c to obtain the classification result value of the second stacking layer 302c. This deep stacking training process with stacked layers is continuously performed until the final prediction result 40c output by the last stacking layer 303c is obtained. Optionally, the cascaded random forest 20c includes multiple random forests.

[0115] For each stacking layer, appropriate features are obtained by combining the prediction results of each stacking layer with the memory gate M. These features, along with historical predictions and the current stacking layer's predictions, guide the prediction of the next stacking layer. Taking the first stacking layer 301c as an example, the input feature set of the first stacking layer 301c is input into it to obtain the classification result value. This classification result value, along with the historical maximum classification value, is then input into the memory gate 304c to obtain the feature set output by the memory gate 304c. This feature set is then used as the input feature set for the second stacking layer 302c. Specifically, within the memory gate 304c, the classification result value is compared with the historical maximum classification value. Based on the comparison result, features are rewarded or penalized to obtain an adjusted feature set. A Monte Carlo sampling algorithm is then used to sample this adjusted feature set, and the sampled features form the input feature set for the next stacking layer. Details regarding the use of the memory gate to determine the input feature set for the next stacking layer can be found in the above embodiment and will not be repeated here.

[0116] The technical solution of this invention, based on the above embodiments, further specifies the process of dividing the original feature matrix into sub-feature matrices and obtaining a fused feature matrix based on the sub-feature matrices. By sliding the original feature vector of dimension m through P windows, P sub-feature matrices are obtained. The P sub-feature matrices are then input into a single random forest or a cascaded random forest, resulting in P fused feature matrices output by each single random forest or cascaded random forest. This allows for adjusting the dimension of the sub-feature matrices by adjusting the window sliding size, and also allows for adjusting the sliding step size to increase the randomness of feature selection.

[0117] Example 4

[0118] Figure 4 This is a schematic diagram of the feature set acquisition device for the stacking stage in deep random forest model training provided in Embodiment 4 of the present invention. This feature set acquisition device for the stacking stage in deep random forest model training can be implemented in software and / or hardware; for example, it can be configured in a computer device. Figure 4 As shown, the device includes a current result acquisition module 410, an input feature set acquisition module 420, and an input feature set acquisition module 430, wherein:

[0119] The current result acquisition module 410 is used to acquire the input feature set of the current stacking layer, and to obtain the classification result value of the current stacking layer by inputting the input feature set into the current stacking layer;

[0120] The classification value comparison module 420 is used to compare the classification result value of the current stacking layer with the historical maximum classification value, wherein the historical maximum classification value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers before the current stacking layer.

[0121] The input feature set acquisition module 430 is used to increase the feature value of the feature in the input feature set of the current stacking layer if the classification result value of the current stacking layer is not less than the historical classification maximum value, and obtain the input feature set of the next stacking layer based on the input feature set after the feature value is increased.

[0122] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The input feature set of the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0123] This invention, through its embodiments, obtains the input feature set of the current stacking layer, inputs the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer, compares the classification result value of the current stacking layer with the historical maximum classification value, and if the classification result value of the current stacking layer is not less than the historical maximum classification value, then the feature values ​​of the features in the input feature set of the current stacking layer are increased, and the input feature set of the next stacking layer is obtained based on the input feature set with increased feature values. If the classification result value of the current stacking layer is less than the historical maximum classification value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical maximum classification value is determined, and the feature values ​​of the features in the difference set are decreased, and the input feature set of the next stacking layer is obtained based on the difference set with decreased feature values. This accelerates the process of convergence to the final optimal solution in the stacking stage and improves the model training efficiency.

[0124] Based on the above scheme, the input feature set acquisition module 430 is specifically used for:

[0125] If the classification result value of the current stacking layer is not less than the historical maximum value, then the feature value of each feature in the input feature set of the current stacking layer is multiplied by a first preset coefficient, which is a value greater than 1, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0126] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature value of each feature in the difference set is multiplied by a second preset coefficient, which is a value less than 1. The input feature set of the next stacking layer is obtained based on the difference set after the feature value is reduced.

[0127] Based on the above scheme, the first preset coefficient is 1.5 and the second preset coefficient is 0.5.

[0128] Based on the above scheme, the input feature set acquisition module 430 is specifically used for:

[0129] If the classification result value of the current stacking layer is not less than the historical maximum value, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the Monte Carlo sampling algorithm is used to sample the input feature set after the feature value is increased. The feature set formed by the sampled features is used as the input feature set of the next stacking layer.

[0130] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference set between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The Monte Carlo sampling algorithm is used to sample the features of the difference set after the feature values ​​are reduced, and the feature set formed by the sampled features is used as the input feature set of the next stacking layer.

[0131] Based on the above scheme, the current result acquisition module 410 includes:

[0132] The feature set merging unit is used to obtain the feature matrix output after feature fusion in the feature fusion stage before the stacking stage when the current stacking layer is the first stacking layer, and merge the feature matrix with the original feature matrix used when performing feature fusion to obtain the merged feature set.

[0133] The feature filtering unit is used to obtain the feature values ​​of all features in the merged feature set through the cascaded random forest, and to filter the features in the merged feature set according to the feature values ​​to obtain the input feature set of the first stacking layer.

[0134] Based on the above solution, the device further includes:

[0135] The feature fusion module is used to obtain the original feature matrix, split the original feature matrix to obtain at least two sub-feature matrices, and input the at least two sub-feature matrices into a single random forest or a cascaded random forest respectively to obtain the fused feature matrix output by the single random forest or the cascaded random forest.

[0136] The feature matrix acquisition module is used to convert the obtained fused feature matrices into one-dimensional feature matrices, which are then used as the feature matrices output by the feature fusion unit.

[0137] Based on the above scheme, the feature fusion module is specifically used for:

[0138] Obtain the original feature matrix. Then, slide the original feature vector of dimension m through P windows to obtain P sub-feature matrices, where the dimension of the i-th sub-feature matrix is ​​(mm / n). i+1)*m / n i , where P is an integer greater than 1, and i takes values ​​in [1, P];

[0139] The P feature matrices are each input into a single random forest or a cascaded random forest to obtain P fused feature matrices output by each single random forest or cascaded random forest, where the dimension of the i-th fused feature matrix is ​​(mm / n). i +1)*n class ;

[0140] Where, n i Used to control the size of the corresponding window, n i Takes values ​​between (1, m), n class This indicates the number of categories the target is divided into.

[0141] The feature set acquisition device for the stacking stage in deep random forest model training provided in this embodiment of the invention can execute the feature set acquisition method for the stacking stage in deep random forest model training provided in any embodiment, and has the corresponding functional modules and beneficial effects of the execution method.

[0142] Example 5

[0143] Figure 5 This is a schematic diagram of the structure of the computer device provided in Embodiment 5 of the present invention. Figure 5 A block diagram of an exemplary computer device 512 suitable for implementing embodiments of the present invention is shown. Figure 5 The computer device 512 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0144] like Figure 5 As shown, the computer device 512 is represented in the form of a general-purpose computing device. The components of the computer device 512 may include, but are not limited to: one or more processors 516, system memory 528, and bus 518 connecting different system components (including system memory 528 and processor 516).

[0145] Bus 518 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor 516, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0146] Computer device 512 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 512, including volatile and non-volatile media, removable and non-removable media.

[0147] System memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. Computer device 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage device 534 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 518 via one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0148] A program / utility 540 having a set (at least one) of program modules 542 may be stored, for example, in memory 528. Such program modules 542 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 542 typically perform the functions and / or methods described in the embodiments of the present invention.

[0149] Computer device 512 can also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), and with one or more devices that enable a user to interact with the computer device 512, and / or with any device that enables the computer device 512 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 522. Furthermore, computer device 512 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 520. As shown, network adapter 520 communicates with other modules of computer device 512 via bus 518. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0150] The processor 516 executes various functional applications and data processing by running programs stored in the system memory 528, such as implementing the feature set acquisition method in the stacking stage of deep random forest model training provided in this embodiment of the invention, the method including:

[0151] Obtain the input feature set of the current stacking layer, and input the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer;

[0152] The classification result value of the current stacking layer is compared with the historical maximum value, where the historical maximum value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers before the current stacking layer.

[0153] If the classification result value of the current stacking layer is not less than the maximum value of the historical classification, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0154] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The input feature set of the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0155] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the feature set acquisition method in the stacking stage of deep random forest model training provided in any embodiment of the present invention.

[0156] Example 6

[0157] Embodiment 6 of the present invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the feature set acquisition method in the stacking stage of deep random forest model training as provided in the embodiments of the present invention. The method includes:

[0158] Obtain the input feature set of the current stacking layer, and input the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer;

[0159] The classification result value of the current stacking layer is compared with the historical maximum value, where the historical maximum value refers to the maximum value among the classification result value of the current stacking layer and the classification result values ​​of all stacking layers before the current stacking layer.

[0160] If the classification result value of the current stacking layer is not less than the maximum value of the historical classification, then the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained based on the input feature set after the feature value is increased.

[0161] If the classification result value of the current stacking layer is less than the historical classification maximum value, then the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature values ​​of the features in the difference set are reduced. The input feature set of the next stacking layer is obtained based on the difference set after the feature values ​​are reduced.

[0162] Of course, the computer program stored on the computer-readable storage medium provided in the embodiments of the present invention is not limited to the method operation described above, but can also execute related operations in the feature set acquisition method of the stacking stage in the deep random forest model training provided in any embodiment of the present invention.

[0163] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0164] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0165] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0166] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0167] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A feature set acquisition method for the stacking stage in deep random forest model training, which is applied to object classification and prediction. in, The object includes image data, sound data or text data, characterized in that the deep random forest model includes a feature fusion unit and a deep learning unit using a stacking strategy, the deep learning unit includes a plurality of stacking layers connected in series, each stacking layer includes a single random forest or a cascaded random forest, and the method includes: Obtain the input feature set of the current stacking layer, input the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer, so as to avoid feature loss caused by the feature fusion process and speed up the selection of the current optimal feature and the process of converging to the final optimal solution; Compare the classification result value of the current stacking layer with the historical classification maximum value, where the historical classification maximum value refers to the maximum value of the classification result value of the current stacking layer and the classification result values ​​of each stacking layer before the current stacking layer, so as to avoid errors in subsequent stages caused by errors in the prediction results of the current stage; If the classification result value of the current stacking layer is not less than the historical classification maximum value, the feature value of the feature in the input feature set of the current stacking layer is increased, and the input feature set of the next stacking layer is obtained according to the input feature set after the feature value is increased, so as to improve the accuracy of the prediction result in the stacking training process; If the classification result value of the current stacking layer is less than the historical classification maximum value, the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the feature value of the feature in the difference is reduced. The input feature set of the next stacking layer is obtained according to the difference after the feature value is reduced, so as to reduce the memory of each feature value in the difference and improve the accuracy of classification or prediction; The step of obtaining the input feature set of the next stacking layer according to the input feature set after the feature value is increased includes: The Monte Carlo sampling algorithm is used to perform feature sampling on the input feature set after the eigenvalue is increased, and the feature set composed of the sampled features is used as the input feature set of the next stacking layer; Accordingly, the step of obtaining the input feature set of the next stacking layer according to the difference set after the eigenvalue reduction includes: The Monte Carlo sampling algorithm is used to perform feature sampling on the difference set after the eigenvalue is reduced, and the feature set composed of the sampled features is used as the input feature set of the next stacking layer.

2. The method according to claim 1, It is characterized in that The step of increasing the feature value of the feature in the input feature set of the current stacking layer includes: Multiply the feature value of each feature in the input feature set of the current stacking layer by a first preset coefficient, where the first preset coefficient is a value greater than 1; Accordingly, reducing the eigenvalue of the feature in the difference set includes: The feature value of each feature in the difference set is multiplied by a second preset coefficient, where the second preset coefficient is a value less than 1.

3. The method according to claim 2, It is characterized in that The first preset coefficient is 1.5, and the second preset coefficient is 0.

5.

4. The method according to any one of claims 1 to 3, It is characterized in that When the current stacking layer is the first stacking layer, obtain the input feature set of the first stacking layer, including: Obtain a feature matrix output after feature fusion in a feature fusion stage before the stacking stage, and merge the feature matrix with the original feature matrix used in the feature fusion to obtain a merged feature set; The feature values ​​of all features in the merged feature set are obtained by cascading random forests, and the features in the merged feature set are screened according to the feature values ​​to obtain the input feature set of the first stacking layer.

5. The method according to claim 4, It is characterized in that Feature fusion is performed in the feature fusion stage, including: Obtain an original feature matrix, split the original feature matrix to obtain at least two sub-feature matrices, input the at least two sub-feature matrices into a single random forest or a cascade random forest respectively, and obtain a fusion feature matrix output by the single random forest or the cascade random forest; Each obtained fused feature matrix is ​​converted into a one-dimensional feature matrix as the feature matrix output by the feature fusion unit.

6. The method according to claim 5, It is characterized in that The method of splitting the original feature matrix to obtain at least two sub-feature matrices, and inputting the at least two sub-feature matrices into a single random forest or a cascade random forest respectively to obtain a fusion feature matrix output by the single random forest or the cascade random forest includes: The original feature vector with dimension m is slid through the window of P windows to obtain P sub-feature matrices, where the dimension of the i-th sub-feature matrix is ​​(m / n i +1)*m / n i , where P is an integer greater than 1, and i takes values ​​in [1, P]; The P sub-feature matrices are respectively input into a single random forest or a cascade random forest to obtain P fused feature matrices output by each single random forest or cascade random forest, where the dimension of the i-th fused feature matrix is ​​(mm / n i +1)*n class ; Among them, n i Used to control the size of the corresponding window, n i Take values ​​between (1, m), n class Indicates the number of target classification categories.

7. A feature set acquisition device for the stacking stage in deep random forest model training, which is applied to object classification and prediction. in, The object includes image data, sound data or text data, characterized in that the deep random forest model includes a feature fusion unit and a deep learning unit using a stacking strategy, the deep learning unit includes a plurality of stacking layers connected in series, each stacking layer includes a single random forest or a cascaded random forest; the device includes: The current result acquisition module is used to obtain the input feature set of the current stacking layer, and input the input feature set into the current stacking layer to obtain the classification result value of the current stacking layer, so as to avoid feature loss caused by the feature fusion process and speed up the selection of the current optimal feature and the process of converging to the final optimal solution; A classification value comparison module is used to compare the classification result value of the current stacking layer with the historical classification maximum value, where the historical classification maximum value refers to the maximum value of the classification result value of the current stacking layer and the classification result values ​​of each stacking layer before the current stacking layer, so as to avoid errors in subsequent stages caused by errors in the prediction results of the current stage; An input feature set acquisition module is used to increase the feature value of the feature in the input feature set of the current stacking layer if the classification result value of the current stacking layer is not less than the historical classification maximum value, and adopt a Monte Carlo sampling algorithm to perform feature sampling on the input feature set after the feature value is increased, and use the feature set composed of the sampled features as the input feature set of the next stacking layer; obtain the input feature set of the next stacking layer according to the input feature set after the feature value is increased, so as to improve the accuracy of the prediction results in the stacking training process; If the classification result value of the current stacking layer is less than the historical classification maximum value, the difference between the input feature set of the current stacking layer and the input feature set corresponding to the historical classification maximum value is determined, and the eigenvalues ​​of the features in the difference are reduced. The Monte Carlo sampling algorithm is used to perform feature sampling on the difference after the eigenvalue is reduced, and the feature set composed of the sampled features is used as the input feature set of the next stacking layer; the input feature set of the next stacking layer is obtained according to the difference after the eigenvalue is reduced, so as to reduce the memory of each eigenvalue in the difference and improve the accuracy of classification or prediction.

8. A computer device, It is characterized in that The device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the feature set acquisition method in the stacking stage in the deep random forest model training as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method for acquiring a feature set in the stacking stage in the deep random forest model training as described in any one of claims 1 to 6 is implemented.