Gradient descent tree generation method, device, electronic device and storage medium

By generating and evaluating gradient descent trees in the gradient descent tree search space, the problem of poor combination of weak classifiers is solved, the automatic generation and performance improvement of gradient descent trees are achieved, and the robustness of gradient descent trees is improved.

CN111914884BInactive Publication Date: 2024-12-06BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202010561789.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-18
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing gradient descent tree generation method, the combination of weak classifiers fails to fully consider the relationship between each weak classifier, resulting in the generated gradient descent tree performance and insufficient robustness.

Method used

By generating the gradient descent tree search space, using K training subsets to train the gradient descent tree, and using K verification subsets to evaluate it, iteratively update the gradient descent tree to be trained according to the score value until the scoring requirements are met or the pre-pre-result is achieved. Set the number of iterations to achieve automatic generation and performance improvement of gradient descent trees.

Benefits of technology

By automatically searching in the gradient descent tree search space, the generated gradient descent tree can obtain better performance and robustness, overcoming the problem of poor combination of weak classifiers in traditional methods, and improving the overall performance of the gradient descent tree model .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111914884B_ABST
    Figure CN111914884B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, electronic device and storage medium for generating a gradient descent tree, which relates to the fields of artificial intelligence technology and deep learning, and can be applied to image processing. The specific implementation scheme is: first generate a gradient descent tree search space, and then generate a gradient descent tree to be trained according to the gradient descent tree search space; then, train the gradient descent tree to be trained and perform performance evaluation, and when the evaluation result does not meet the scoring requirements, iteratively update the gradient descent tree to be trained until the evaluation result meets the scoring requirements or the number of iterative updates of the gradient descent tree to be trained reaches a preset number of iterations. The present application realizes the automatic generation of a gradient descent tree, and performs an automatic search in the gradient descent tree search space to generate a gradient descent tree, so that the gradient descent tree finally generated can obtain better performance and improve the robustness of the gradient descent tree.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present application generally relate to the field of computer technology, and more specifically, to the field of artificial intelligence technology and deep learning, and can be applied to image processing. Background Art

[0002] Deep learning is a new field in machine learning research, which aims to build a neural network that simulates the human brain for analysis and learning. In recent years, deep learning technology has achieved success in many aspects of artificial intelligence technology research, such as computer vision, speech recognition, and natural language processing.

[0003] In deep learning technology, the quality of artificial neural network (ANN) structure has a very important impact on the effect of the final model. Manual design of network topology requires designers to have rich experience and multiple attempts, and when there are many parameters, explosive combinations will occur. The conventional random search algorithm is less feasible to generate network structure. Therefore, Neural Architecture Search (NAS) technology has gradually become a research hotspot in the field of deep learning. Summary of the invention

[0004] The present application provides a method, device, electronic device and storage medium for generating a gradient descent tree.

[0005] According to a first aspect, a method for generating a gradient descent tree is provided, comprising:

[0006] Obtain a training set and a validation set, and divide the training set and the validation set into K training subsets and K validation subsets, where K is a positive integer;

[0007] Generate a gradient descent tree search space, and generate a gradient descent tree to be trained according to the gradient descent tree search space;

[0008] Training the to-be-trained gradient descent tree according to the K training subsets to generate K gradient descent tree models;

[0009] Using the K validation subsets respectively to evaluate the K gradient descent tree models to generate score values ​​for the K gradient descent tree models; and

[0010] The gradient descent tree to be trained is iteratively updated N times according to the score values ​​of the K gradient descent tree models until the score values ​​of the K gradient descent tree models meet the score requirements or N reaches a preset number of iterations, where N is a positive integer.

[0011] According to a second aspect, a device for generating a gradient descent tree is provided, comprising:

[0012] An acquisition module, used to acquire a training set and a validation set, and divide the training set and the validation set into K training subsets and K validation subsets, where K is a positive integer;

[0013] A first generation module, for generating a gradient descent tree search space;

[0014] A second generating module, used for generating a gradient descent tree to be trained according to the gradient descent tree search space;

[0015] A training module, used for training the to-be-trained gradient descent tree according to the K training subsets to generate K gradient descent tree models;

[0016] a validation module, configured to respectively use the K validation subsets to evaluate the K gradient descent tree models to generate scoring values ​​for the K gradient descent tree models; and

[0017] An updating module is used to perform N iterative updates on the gradient descent tree to be trained according to the scoring values ​​of the K gradient descent tree models, until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or N reaches a preset number of iterations, where N is a positive integer.

[0018] According to a third aspect, there is provided an electronic device, comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a gradient descent tree as described in the first aspect.

[0022] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method for generating a gradient descent tree as described in the first aspect.

[0023] According to a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for generating a gradient descent tree as described in the first aspect.

[0024] The method, device, electronic device and storage medium for generating a gradient descent tree provided by the present application have the following beneficial effects:

[0025] By generating a gradient descent tree search space and automatically searching in the gradient descent tree search space to generate a gradient descent tree, not only the automatic generation of the gradient descent tree is achieved, but also the final generated gradient descent tree can obtain better performance and improve the robustness of the gradient descent tree model.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0028] Figure 1 is a flow chart of a method for generating a gradient descent tree according to the first embodiment of the present application;

[0029] Figure 2 is a flow chart of a method for generating a gradient descent tree according to the second embodiment of the present application;

[0030] Figure 3 is a flow chart of a method for generating a gradient descent tree according to the third embodiment of the present application;

[0031] Figure 4 is a flow chart of a method for generating a gradient descent tree according to a fourth embodiment of the present application;

[0032] Figure 5 is a schematic structural diagram of a device for generating a gradient descent tree according to a fifth embodiment of the present application;

[0033] Figure 6 is a structural schematic diagram of a device for generating a gradient descent tree according to a sixth embodiment of the present application;

[0034] Figure 7 is a schematic structural diagram of a device for generating a gradient descent tree according to a seventh embodiment of the present application;

[0035] Figure 8 is a schematic structural diagram of a device for generating a gradient descent tree according to an eighth embodiment of the present application;

[0036] Fig. 9 It is a block diagram of an electronic device used to implement the method for generating a gradient descent tree according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] The following describes the gradient descent tree generation method, device, electronic device and storage medium of the present application with reference to the accompanying drawings.

[0039] Gradient Boosting Decision Tree (GBDT) is an iterative decision tree algorithm that obtains the final gradient decision tree through multiple rounds of iterations. A weak classifier is generated in each round of iteration, and each weak classifier is learned based on the residual of the weak classifier obtained in the previous round. Since the learning process is to continuously improve the accuracy of the final classifier by reducing the deviation, this requires the weak classifier to be simple enough and have low variance and high deviation. Based on this requirement, the depth of each weak classifier will not be very deep. In the traditional gradient descent tree generation method, the final total classifier (i.e., gradient descent tree) is obtained by weighted summing the weak classifiers obtained in each round of learning. The relationship between each weak classifier is not considered globally, so that the combination of each weak classifier is not optimal, which will limit the performance of the gradient descent tree and lead to poor performance of the final generated gradient descent tree.

[0040] In response to the above problems, the present application discloses a method for generating a gradient descent tree, which generates a gradient descent tree search space, generates a gradient descent tree to be trained according to the gradient descent tree search space, and then uses the obtained K training subsets to train the generated gradient descent tree to be trained to obtain K gradient descent tree models, and uses the obtained K validation sets to perform performance evaluation on the K gradient descent tree models to obtain scoring values ​​of the K gradient descent tree models. When the scoring values ​​of the K gradient descent tree models do not meet the scoring requirements and do not reach the preset number of iterations, the gradient descent tree to be trained is iteratively updated until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or the number of iterations N reaches the preset number of iterations, thereby realizing the automatic generation of the gradient descent tree. The solution of the present application generates a gradient descent tree by automatically searching in a gradient descent tree search space. Since the gradient descent tree search space contains the possibility of all gradient descent trees that can be searched, the combination of weak classifiers contained in each gradient descent tree in the gradient descent tree search space is arbitrary, rather than simply weighted summing the weak classifiers obtained in each round of learning. This fully considers the relationship between different weak classifiers, so that the final generated gradient descent tree is a better combination of weak classifiers, thereby ensuring the performance of the gradient descent tree and improving the robustness of the gradient descent tree.

[0041] Figure 1 It is a flow chart of a method for generating a gradient descent tree according to the first embodiment of the present application. The method can be executed by the device for generating a gradient descent tree provided by the present application, or by the electronic device provided by the present application, wherein the electronic device can include but is not limited to terminal devices such as desktop computers and tablet computers, or can be a server. The present application is explained below by taking the method for generating a gradient descent tree provided by the device for generating a gradient descent tree provided by the present application as an example, and it cannot be used as a limitation on the present application.

[0042] like Figure 1 As shown, the method for generating the gradient descent tree may include the following steps:

[0043] Step 101: obtain a training set and a validation set, and divide the training set and the validation set into K training subsets and K validation subsets, where K is a positive integer.

[0044] For different tasks, in the embodiments of the present application, training sets and validation sets can be obtained from different types of sample sets. The training set is used for model training, and the validation set is used to evaluate the performance of the trained model, that is, to test the performance of the trained model.

[0045] For example, for image processing tasks such as classification tasks, object detection tasks, and face detection tasks, a large number of image samples can be obtained from public image datasets as training sets and validation sets, where the public image datasets can be, for example, the ImageNet dataset, the PASCAL VOC dataset, the Labelme dataset, etc., where the image samples obtained in the validation set are different from the image samples in the training set to ensure the performance and robustness of the trained gradient descent tree.

[0046] For example, for speech recognition tasks, a large number of speech samples can be obtained from public open source speech datasets as training sets and validation sets. The open source speech datasets can be, for example, Chinese datasets, English datasets, etc. The speech data obtained in the validation set is different from the speech data in the training set to ensure the performance and robustness of the trained gradient descent tree.

[0047] For example, taking image samples from the ImageNet dataset as training sets and validation sets, we can first obtain a sample set containing a large number of image samples from the ImageNet dataset, and then divide the sample set into validation sets and training sets according to the preset allocation ratio. For example, the preset allocation ratio of the training set to the validation set in the sample set is 8:2, that is, 80% of the image samples in the sample set are used as the training set, and the remaining 20% ​​of the image samples in the sample set are used as the validation set, then the obtained sample set is divided into the training set and the validation set according to the ratio of 8:2.

[0048] In this embodiment, after obtaining the training set and the validation set, the training set and the validation set can be divided into K training subsets and the validation set can be divided into K validation subsets, where K is a positive integer.

[0049] For example, when dividing the training set and the validation set, the training set can be randomly divided into K parts to obtain K training subsets, and the number of sample images contained in each training subset can be the same or different; for the division of the validation set, the validation set can also be randomly divided into K parts to obtain K validation subsets, and the number of sample images contained in each validation subset can be the same or different.

[0050] It should be noted that, in this embodiment, the number of training subsets and validation subsets obtained by division is the same. For example, the training set is divided into 5 training subsets, and similarly, the validation set is divided into 5 validation subsets. Each training subset corresponds to a validation subset, so that the validation subset can be used to perform performance testing on the gradient descent tree model trained according to the training subset.

[0051] In a possible implementation method of the embodiment of the present application, in order to obtain K training subsets and K verification subsets, the obtained sample set can be first divided into K parts to obtain K sample subsets, and then for each verification subset, according to a preset distribution ratio of the training set to the verification set (for example, 8:2), the sample subset is divided into a training subset and a verification subset, and finally K training subsets and K verification subsets are obtained, and each training subset corresponds to a verification subset.

[0052] Step 102, generating a gradient descent tree search space, and generating a gradient descent tree to be trained according to the gradient descent tree search space.

[0053] In this embodiment, the design rules of the gradient descent tree search space can be pre-designed, and then the gradient descent tree search space is generated according to the design rules.

[0054] For example, the design rules of the gradient descent tree search space can stipulate the number of weak classifiers required for the gradient descent tree, the type of weak classifiers, and the properties of the weak classifiers corresponding to the type of weak classifiers. Among them, the type of weak classifier can be, for example, a decision tree. When the weak classifier is a decision tree, the properties of the weak classifier can include the maximum number of nodes (i.e., features) of the decision tree, the maximum depth of the decision tree, and the maximum number of child nodes that each node can contain.

[0055] According to the above design rules, a gradient descent tree search space that meets the conditions stipulated in the above design rules can be generated. The gradient descent tree search space contains all possibilities of gradient descent trees that can be generated, that is, the gradient descent tree search space is a set of all possible gradient descent trees. Moreover, when the weak classifier is a decision tree, in the generated gradient descent tree search space, in each decision tree contained in each gradient descent tree, the combination of each node of the decision tree is arbitrary, the total number of nodes contained in each decision tree is not greater than the maximum number of nodes stipulated in the design rules, the depth of each decision tree is not greater than the maximum depth stipulated in the design rules, and in each decision tree, the number of sub-nodes contained in each node is not greater than the maximum number of sub-nodes stipulated in the design rules.

[0056] Taking the weak classifier as a decision tree as an example, the maximum number of nodes agreed in the design rules is 100, the maximum depth of the tree is 10, and the maximum number of child nodes of the node is 5, then all possible decision trees that meet the design rules can be generated, and the total number of nodes contained in each decision tree is not more than 100, and the depth of each decision tree is not more than 10, and the number of child nodes contained in each node in each decision tree is not more than 5. After all possible decision trees are generated, the decision trees are arbitrarily combined according to the number of decision trees agreed in the design rules to obtain all possible combinations of decision trees, that is, all possible gradient descent trees are generated, and the set of all possible gradient descent trees constitutes the gradient descent tree search space. The scheme of the present application, by searching in the gradient descent tree search space, can find the better combination of each decision tree from all possible gradient descent trees, so that the performance of the gradient descent tree is better, and the shortcomings of the prior art that the weak classifiers obtained in each round of learning are simply weighted summed to generate the gradient descent tree, resulting in the performance of the gradient descent tree being limited, so that the searched gradient descent tree has better performance and robustness.

[0057] In this embodiment, after the gradient descent tree search space is generated, the gradient descent tree to be trained can be generated according to the gradient descent tree search space. As described above, the gradient descent tree search space contains all the possibilities of the gradient descent trees that can be generated, so a gradient descent tree can be randomly generated from the gradient descent tree search space, and the randomly generated gradient descent tree is used as the gradient descent tree to be trained.

[0058] In order to obtain a gradient descent tree with a simple network architecture and satisfactory performance, when generating a gradient descent tree to be trained according to the gradient descent tree search space, a gradient descent tree with a relatively simple network architecture can also be randomly generated as the gradient descent tree to be trained. For example, taking the weak classifier as a decision tree as an example, the network architecture of the gradient descent tree to be trained obtained for the first time can be that the gradient descent tree contains a combination of two decision trees, and each node in each decision tree contains only 2 child nodes, the maximum depth of the decision tree is 3, and the total number of nodes of the decision tree is 7.

[0059] It should be noted that there is no particular order in which steps 101 and 102 are executed. The two can be executed one after the other or simultaneously. This application only uses the execution of step 102 after step 101 as an example to explain this application, and it cannot be used as a limitation on this application.

[0060] Step 103: train the to-be-trained gradient descent trees according to the K training subsets to generate K gradient descent tree models.

[0061] In this embodiment, after the gradient descent tree to be trained is generated, the gradient descent tree to be trained can be trained according to the obtained K training subsets to generate K gradient descent tree models.

[0062] When using each training subset to train the gradient descent tree to be trained, the sample data (such as image samples, speech data) contained in the training subset is used as the input of the gradient descent tree to be trained, and the annotation results of the sample data are used as the output of the gradient descent tree to be trained. By continuously iteratively updating the parameters of the gradient descent tree to be trained, a set of model parameters that minimize the value of the loss function on the training subset is finally found. After the training is completed, a gradient descent tree model corresponding to the training subset is obtained.

[0063] It can be understood that the K gradient descent tree models are obtained by training the same model structure, i.e., the gradient descent tree to be trained, using K different training subsets. Since different training subsets are used, the parameters contained in the K gradient descent tree models obtained by training are also different.

[0064] In a possible implementation of the embodiment of the present application, before training the gradient descent tree to be trained, the gradient descent tree to be trained can be initialized first, such as initializing the parameters of the gradient descent tree to be trained. After the initialization is completed, the gradient descent tree to be trained is trained using K training subsets.

[0065] Step 104 , respectively using K validation subsets to evaluate the K gradient descent tree models to generate score values ​​for the K gradient descent tree models.

[0066] In this embodiment, after using K training subsets to train the to-be-trained gradient descent tree to obtain the corresponding K gradient descent tree models, for each gradient descent tree model, a validation subset corresponding to the training subset used when training the gradient descent tree model can be used to perform a performance test on the gradient descent tree model to generate a score value for the gradient descent tree model, and each gradient descent tree model is tested in turn to finally obtain score values ​​for K gradient descent tree models, obtaining a total of K score values.

[0067] Since in the gradient descent tree search space of the present application, the gradient descent tree is generated by any combination of different weak classifiers, the combination of each weak classifier in the gradient descent tree to be trained generated according to the gradient descent tree search space is arbitrary, so that the training of the gradient descent tree to be trained is to train each weak classifier in the gradient descent tree to be trained as a whole, rather than continuing to train to obtain a weak classifier based on the residual of the weak classifier trained in the previous round. During the training process, the relationship between the weak classifiers is considered globally, and the score value obtained by evaluating the gradient descent tree model using the validation subset can reflect the overall performance of the gradient descent tree model.

[0068] Step 105, performing N iterative updates on the gradient descent tree to be trained according to the score values ​​of the K gradient descent tree models, until the score values ​​of the K gradient descent tree models meet the score requirements or N reaches a preset number of iterations, where N is a positive integer.

[0069] The scoring requirement and the preset number of iterations can be preset. For example, the scoring requirement can be set to a minimum value of K scoring values ​​of not less than 90%, or the scoring requirement can be set to a mean value of K scoring values ​​of not less than 90%, etc.; the preset number of iterations can be set to 30 times, 40 times, etc.

[0070] In this embodiment, after obtaining the scoring values ​​of K gradient descent tree models, it can be determined whether the gradient descent tree to be trained needs to be iteratively updated based on the obtained K scoring values, and when further iterative updating is required, the gradient descent tree to be trained is iteratively updated.

[0071] In order to prevent unlimited searching in the gradient descent tree search space, a preset number of iterations can be set in advance. When the iterative update number of the gradient descent tree to be trained reaches the preset number of iterations, even if the score values ​​of the currently obtained K gradient descent tree models do not meet the score requirements, the gradient descent tree to be trained will no longer be iteratively updated.

[0072] That is to say, in this embodiment, after obtaining the score values ​​of K gradient descent tree models, it can be determined whether the obtained score values ​​meet the score requirements. For example, the score requirement is that the minimum value of the K score values ​​is not less than 90%. When the minimum value of the K score values ​​is less than 90%, it is determined that the score requirements are not met, that is, the performance of the current gradient descent tree still does not meet the requirements. It is necessary to perform the N+1th iterative update on the gradient descent tree to be trained after N iterative updates on the basis of the gradient descent tree to be trained that has been iteratively updated N times. At this time, before iteratively updating the gradient descent tree to be trained, it is first determined whether the current number of iterations N reaches the preset number of iterations. If N is less than the preset number of iterations, the operation of iteratively updating the gradient descent tree to be trained is performed.

[0073] Wherein, N is a positive integer.

[0074] It should be noted that N refers to the number of iterative updates of the gradient descent tree to be trained. For the first generated gradient descent tree to be trained, after K training subsets are used to train to obtain K gradient descent tree models, and K validation subsets are used to evaluate to obtain K scoring values, if the K scoring values ​​do not meet the scoring requirements, the gradient descent tree to be trained needs to be iteratively updated for the first time, that is, N is equal to 1 at this time; if the gradient descent tree to be trained after the iterative update still does not meet the scoring requirements, the gradient descent tree to be trained after the iterative update needs to be iteratively updated again, and N is equal to 2 at this time. By analogy, the number of iterations N when the gradient descent tree to be trained needs to be iteratively updated each time.

[0075] In this embodiment, the iterative update of the gradient descent tree to be trained may be performed by adjusting the weak classifiers, the positions and the number of weak classifiers in the gradient descent tree to be trained. When the weak classifier is a decision tree, the positions of the nodes in the decision tree may also be adjusted. This application does not limit the method of iteratively updating the gradient descent tree to be trained.

[0076] Furthermore, for the iteratively updated gradient descent tree to be trained, it is trained again according to K training subsets to generate K gradient descent tree models, and the K gradient descent tree models are evaluated using K validation subsets to generate scoring values ​​of the K gradient descent tree models, and the iteratively updated gradient descent tree to be trained is iteratively updated again according to the scoring values. That is, for the iteratively updated gradient descent tree to be trained, the above steps 103 to 105 are repeatedly performed until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or the number of iterative updates N reaches the preset number of iterations, the search ends, and the final generated gradient descent tree is obtained.

[0077] The method for generating a gradient descent tree in this embodiment obtains a training set and a validation set, and divides the training set and the validation set into K training subsets and K validation subsets, and generates a gradient descent tree search space, and generates a gradient descent tree to be trained according to the gradient descent tree search space, then trains the gradient descent tree to be trained according to the K training subsets to generate K gradient descent tree models, and uses K validation subsets to evaluate the K gradient descent tree models respectively to generate scoring values ​​of the K gradient descent tree models, and performs N iterative updates on the gradient descent tree to be trained according to the scoring values ​​of the K gradient descent tree models, until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or N reaches a preset number of iterations, thereby realizing the automatic generation of the gradient descent tree. Moreover, a gradient descent tree is generated by automatically searching in a gradient descent tree search space. Since the gradient descent tree search space contains the possibility of all searchable gradient descent trees, the combination of weak classifiers contained in each gradient descent tree in the gradient descent tree search space is arbitrary, rather than simply weighted summing the weak classifiers obtained in each round of learning. This comprehensively considers the relationship between different weak classifiers, so that the final generated gradient descent tree is a better combination of weak classifiers, thereby ensuring the performance of the gradient descent tree and improving the robustness of the gradient descent tree.

[0078] In a possible implementation of the embodiment of the present application, when the training set and the validation set are divided into K training subsets and K validation subsets, the training set and the validation set can be divided into K training subsets and K validation subsets by a K-fold cross partitioning algorithm.

[0079] The K-fold cross partitioning algorithm divides all data sets into K parts, takes one of them as the validation subset without duplication each time, and uses the other K-1 parts as the training subsets for training the model, resulting in a total of K combinations of training subsets and validation subsets.

[0080] For example, assuming K=5, in this embodiment, all data contained in the training set and the validation set are randomly divided into K parts, and each piece of data is numbered 1, 2, 3, 4, and 5 respectively. Then, the combination of the training subset and the validation subset obtained by the division is as follows:

[0081] (1) 1, 2, 3, 4 are used as training subsets, and 5 is used as validation subset;

[0082] (2) 1, 2, 3, 5 are used as training subsets, and 4 is used as validation subset;

[0083] (3) 1, 2, 4, 5 are used as training subsets, and 3 is used as validation subset;

[0084] (4) 1, 3, 4, 5 are used as training subsets, and 2 is used as validation subset;

[0085] (5) 2, 3, 4, 5 are used as training subsets, and 1 is used as validation subset.

[0086] It can be seen that after the 5-fold cross partitioning algorithm is used, 5 training subsets and 5 validation subsets corresponding to each training subset are obtained.

[0087] Since the K-fold cross partition algorithm randomly divides the data into K parts, one part is selected as the validation subset without duplication each time, and the remaining K-1 parts are used as the validation subset. Therefore, the training set and the validation set are divided into K training subsets and K validation subsets according to the K-fold cross partition algorithm, which ensures the randomness of the training subsets and the validation subsets. The gradient descent tree model obtained by training the training subset can be evaluated by the validation subset to improve the accuracy of the evaluation results, which is conducive to improving the performance and robustness of the gradient descent tree finally obtained.

[0088] In order to more clearly describe the specific implementation process of generating a gradient descent tree to be trained according to the gradient descent tree search space in the above embodiment, the following is combined with the attached Figure 2 Provide detailed explanation.

[0089] Figure 2 FIG. 1 is a flow chart of a method for generating a gradient descent tree according to the second embodiment of the present application. Figure 2 As shown, in Figure 1 Based on the illustrated embodiment, in step 102, generating a gradient descent tree to be trained according to the gradient descent tree search space may include the following steps:

[0090] Step 201, generating a gradient descent tree sequence generator according to the gradient descent tree search space.

[0091] Step 202, generating a gradient descent tree sequence according to a gradient descent tree sequence generator.

[0092] Step 203, generating a gradient descent tree to be trained according to the gradient descent tree sequence and the gradient descent tree search space.

[0093] As mentioned above, the gradient descent tree search space is generated according to a preset design rule, which defines all possibilities, and the gradient descent tree search space is a set of all possibilities. Therefore, in this embodiment, the gradient descent tree sequence generator can be initialized according to the gradient descent tree search space to generate a gradient descent tree sequence generator.

[0094] Among them, the gradient descent tree sequence generator can be randomly initialized, and each possibility in the gradient descent tree search space corresponds to an initialization result. According to the random initialization of the gradient descent tree search space, a gradient descent tree sequence generator can be randomly obtained.

[0095] The gradient descent tree sequence generator can generate a gradient descent tree sequence. The gradient descent tree sequence is a modeling that uses a set of sequences to uniquely correspond to a possibility, that is, the gradient descent tree sequence generator generates a unique corresponding gradient descent tree sequence.

[0096] The gradient descent tree sequence can represent each weak classifier selected for gradient descent tree modeling. According to the gradient descent tree sequence and the gradient descent tree search space, a gradient descent tree to be trained can be generated.

[0097] For example, assume that the gradient descent tree sequence generator can directly generate a gradient descent tree sequence such as [33, 142, 321, 222]. This sequence indicates that four weak classifiers numbered 33, 142, 321, and 222 are selected and combined in sequence to generate a gradient descent tree. Afterwards, when generating the gradient descent tree to be trained, a combination matching the weak classifiers contained in the above sequence is selected from the gradient descent tree search space as the gradient descent tree to be trained.

[0098] It should be noted that when the weak classifier is a decision tree, for each decision tree constituting the gradient descent tree, the combination of nodes in the decision tree is random, and using the same multiple nodes, the generated decision trees are different according to the different positions of the nodes, and the decision trees are also numbered differently, that is, in the embodiment of the present application, one number uniquely corresponds to one decision tree. In addition, using the same multiple decision trees, the generated gradient descent trees are also different according to the different positional relationships of the decision trees, for example, the gradient descent tree sequence [33,142,321,222] and the gradient descent tree sequence [321,33,142,222], due to the different positional relationships of the decision trees, these two sequences correspond to two different gradient descent trees, so that the gradient descent trees generated in sequence according to the respective decision trees corresponding to the known gradient descent tree sequence are unique.

[0099] In order to optimize the gradient descent tree sequence generated by the gradient descent tree sequence generator, the possible combination of each weak classifier contained in the gradient descent tree search space can be encoded to facilitate the expression of the gradient descent tree sequence. After the gradient descent tree sequence generator generates the gradient descent tree sequence, the gradient descent tree sequence is decoded according to the gradient descent tree search space to obtain the corresponding gradient descent tree to be trained. For example, for each possibility in the gradient descent tree search space, the encoding can be started from 1. The gradient descent tree sequence generated by the gradient descent tree sequence generator is the encoded data, such as 1, which is much simpler than the sequence [33,142,321,222] and other data. Then, according to the encoding of each possibility in the gradient descent tree search space, the gradient descent tree sequence is decoded, that is, the positional relationship between the weak classifiers and the weak classifiers corresponding to the gradient descent tree sequence is obtained, and then the positional relationship between the weak classifiers and the weak classifiers is used to generate the gradient descent tree to be trained in sequence. For example, a possible modeling [33, 142, 321, 222] in the gradient descent tree search space corresponds to the code 5. After obtaining the gradient descent tree sequence 5, by decoding according to the gradient descent tree search space, the modeling combination scheme can be determined to be [33, 142, 321, 222], and then the weak classifiers numbered 33, 142, 321 and 222] and their positional relationships are selected, and the gradient descent tree to be trained is generated in sequence.

[0100] The method for generating a gradient descent tree in this embodiment generates a gradient descent tree sequence generator according to a gradient descent tree search space, then generates a gradient descent tree sequence according to the gradient descent tree sequence generator, and then generates a gradient descent tree to be trained according to the gradient descent tree sequence and the gradient descent tree search space. Thus, the automatic search for the gradient descent tree to be trained from the gradient descent tree search space is realized, and the randomness of the gradient descent tree to be trained is ensured.

[0101] Figure 3 FIG. 1 is a flow chart of a method for generating a gradient descent tree according to the third embodiment of the present application. Figure 2 Based on the embodiment shown, Figure 3 As shown, step 105 may include the following steps:

[0102] Step 301, respectively obtain K score values ​​of K gradient descent tree models.

[0103] Step 302: Generate an average score value according to the K score values ​​of the K gradient descent tree models.

[0104] In this embodiment, for any one of the K gradient descent tree models, the validation subset corresponding to the training subset is used for evaluation, that is, the corresponding score value is obtained. For the K gradient descent tree models, K score values ​​can be obtained. Then, the average score value of the K score values ​​can be calculated based on the K score values.

[0105] Step 303: If the average score value is less than the score requirement and the current number of iterations N is less than the preset number of iterations, the gradient descent tree sequence generator is further updated.

[0106] Step 304: Update the gradient descent tree to be trained by using the updated gradient descent tree sequence generator.

[0107] As an example, the scoring requirement may be a preset performance standard threshold, such as setting the performance standard threshold to 90%. In this example, when the average score of the K score values ​​is less than the preset performance standard threshold, it is determined that the average score does not meet the scoring requirement, and then it is further determined whether the current number of iterations N is less than the preset number of iterations. If the current number of iterations N is less than the preset number of iterations, the gradient descent tree sequence generator is further updated.

[0108] In the embodiment of the present application, the gradient descent tree sequence generator may be a neural network module, or may be an evolutionary algorithm module. For different modules, the gradient descent tree sequence generator may be updated in different ways.

[0109] As a possible implementation method, when the gradient descent tree sequence generator is a neural network module, the gradient descent tree sequence generator can be updated by a back-propagation algorithm.

[0110] As a possible implementation method, when the gradient descent tree sequence generator is an evolutionary algorithm module, the gradient descent tree sequence generator can be updated by a population update algorithm.

[0111] In this embodiment, when the gradient descent tree sequence generator is a neural network module, the gradient descent tree sequence generator is updated by a back-propagation algorithm; when the gradient descent tree sequence generator is an evolutionary algorithm module, the gradient descent tree sequence generator is updated by a population update algorithm. Thus, different update methods are provided for updating the gradient descent tree sequence generator for different gradient descent tree sequence generators, thereby improving the flexibility of the method.

[0112] Next, after the gradient descent tree sequence generator is updated, the gradient descent tree to be trained can be updated through the updated gradient descent tree sequence generator. As mentioned above, the gradient descent tree sequence generator generates a unique corresponding gradient descent tree sequence. After the gradient descent tree sequence generator is updated, the gradient descent tree sequence generated by the gradient descent tree sequence generator also changes accordingly. Therefore, the gradient descent tree to be trained generated according to the gradient descent tree sequence and the gradient descent tree search space is also updated accordingly.

[0113] The method for generating a gradient descent tree in this embodiment obtains K scoring values ​​of K gradient descent tree models respectively, generates an average scoring value according to the K scoring values ​​of the K gradient descent tree models, and further updates the gradient descent tree sequence generator when the average scoring value is less than the scoring requirement and the current number of iterations N is less than the preset number of iterations, and then updates the gradient descent tree to be trained through the updated gradient descent tree sequence generator. Therefore, it is achieved that when the number of iterations does not reach the preset number of iterations, it is determined whether to update the gradient descent tree to be trained according to the average scoring value of the gradient descent tree model, and it is achieved that the gradient descent tree models that do not meet the scoring requirements are iteratively updated to ensure that the gradient descent trees that meet the scoring requirements are obtained as much as possible, which provides conditions for generating gradient descent trees with good performance and good robustness.

[0114] In order to more clearly describe the specific implementation process of generating the gradient descent tree search space in the above embodiment, Figure 4 Provide detailed explanation.

[0115] Figure 4 is a flow chart of a method for generating a gradient descent tree according to the fourth embodiment of the present application, such as Figure 4 As shown, in Figure 1 Based on the embodiment shown, in step 102, the generation of the gradient descent tree search space can be achieved by the following steps:

[0116] Step 401, obtaining the number of classifiers required for the gradient descent tree search space.

[0117] Among them, according to the strength of the classification ability of the classifier, the classifier can be divided into strong classifiers and weak classifiers.

[0118] In this embodiment, the number of classifiers can be set according to actual needs. The number of classifiers refers to the maximum number of classifiers that can be included in each gradient descent tree, that is, the threshold number of classifiers that the gradient descent tree can contain. The number of classifiers contained in any gradient descent tree in the gradient descent tree search space is not greater than the threshold number of classifiers.

[0119] Step 402, obtaining the type of classifier required by the gradient descent tree search space and the attributes corresponding to the type.

[0120] The type of classifier may be, for example, a decision tree, an association rule algorithm, etc. Different types of classifiers have different corresponding attributes. Taking a decision tree as an example, when the weak classifier is a decision tree, the acquired attributes may include the maximum number of nodes (i.e., features) of the decision tree, the maximum depth of the decision tree, and the maximum number of child nodes that each node can contain.

[0121] Step 403: construct a gradient descent tree search space according to the number of classifiers, the types of classifiers, and the attributes corresponding to the types.

[0122] In this embodiment, after obtaining the number of classifiers, the types of classifiers, and the attributes corresponding to the types, a gradient descent tree search space can be constructed based on the number of classifiers, the types of classifiers, and the attributes corresponding to the types. The gradient descent tree search space can generate any gradient descent tree that meets the above conditions.

[0123] Taking the type of classifier obtained as decision tree as an example, after obtaining the decision tree and the corresponding maximum number of nodes of the decision tree, the maximum depth of the decision tree and the maximum number of child nodes that each node can contain, the features in the training data can be arbitrarily combined to generate all possible decision trees based on the maximum number of nodes of the decision tree, the maximum depth of the decision tree and the maximum number of child nodes that each node can contain. Then, according to the number of decision trees obtained, all possible decision trees can be arbitrarily combined to obtain all possible gradient descent trees. The set of all possible gradient descent trees constitutes the gradient descent tree search space.

[0124] When a gradient descent tree search space is generated using a decision tree as a classifier, in the generated gradient descent tree search space, the number of decision trees contained in each gradient descent tree is not greater than the number of decision trees obtained, and the total number of nodes contained in each decision tree contained in any gradient descent tree is not greater than the maximum number of nodes, the depth of each decision tree is not greater than the maximum depth, and in each decision tree, the number of child nodes contained in each node is not greater than the maximum number of child nodes. The gradient descent tree search space can generate any gradient descent tree that meets the above conditions. The decision trees contained in the gradient descent tree are randomly combined, rather than simply weighted summing the decision trees obtained in each round of learning, thereby comprehensively considering the relationship between different decision trees, so that the final generated gradient descent tree is a better combination of decision trees, which can ensure the performance of the gradient descent tree.

[0125] The method for generating a gradient descent tree of this embodiment, by constructing a gradient descent tree search space according to the number of classifiers required for the acquired gradient descent tree search space, the type of classifier, and the attributes corresponding to the type of classifier, lays the foundation for automatically searching and generating a gradient descent tree, and the classifiers constituting the gradient descent tree are arbitrarily combined, and the relationship between the classifiers is comprehensively considered, which is conducive to improving the performance of the generated gradient descent tree. In addition, the early NAS did not limit the search space, and even required seven or eight hundred graphics processors during the search, and trained for a month to obtain a converged model, requiring more hardware equipment, and the search speed was slow; and in this application, by generating a gradient descent tree search space, searching in the gradient descent tree search space to generate a gradient descent tree, the gradient descent tree search space defines the variables of the optimization problem, and the variable scale determines the difficulty and search time of the search algorithm, so by defining a reasonable gradient descent tree search space for searching, it is possible to speed up the search speed and efficiency, reduce the use of hardware equipment, and thus save hardware costs.

[0126] The solution provided by the present application can be applied to image processing. For example, when the input of the gradient descent tree is a feature extracted by a deep learning network, the solution provided by the present application can be applied to tasks such as classification tasks, target detection tasks, face detection tasks, and face and body key point detection tasks in image processing. Since the gradient descent tree generated by the solution provided by the present application has good performance and robustness, when the gradient descent tree is used to complete tasks such as classification tasks, target detection tasks, and face detection tasks, the accuracy of task processing can be improved. For classification tasks, the accuracy of classification can be improved, and for target detection tasks, the accuracy of target detection results can be improved.

[0127] According to an embodiment of the present application, the present application also provides a device for generating a gradient descent tree.

[0128] Figure 5 is a schematic diagram of the structure of a device for generating a gradient descent tree according to the fifth embodiment of the present application. Figure 5 As shown, the gradient descent tree generating device 50 includes: an acquisition module 510, a first generating module 520, a second generating module 530, a training module 540, a verification module 550 and an updating module 560.

[0129] The acquisition module 510 is used to acquire a training set and a validation set, and divide the training set and the validation set into K training subsets and K validation subsets, where K is a positive integer.

[0130] In a possible implementation of the embodiment of the present application, the acquisition module 510 divides the training set and the validation set into K training subsets and K validation subsets by using a K-fold cross partitioning algorithm.

[0131] The first generating module 520 is used to generate a gradient descent tree search space.

[0132] The second generating module 530 is used to generate a gradient descent tree to be trained according to the gradient descent tree search space.

[0133] The training module 540 is used to train the to-be-trained gradient descent tree according to the K training subsets to generate K gradient descent tree models.

[0134] The verification module 550 is used to evaluate the K gradient descent tree models using the K verification subsets respectively to generate score values ​​of the K gradient descent tree models.

[0135] The updating module 560 is used to perform N iterative updates on the gradient descent tree to be trained according to the scoring values ​​of the K gradient descent tree models until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or N reaches a preset number of iterations, where N is a positive integer.

[0136] In a possible implementation of the embodiment of the present application, as Figure 6 As shown, in Figure 5 Based on the illustrated embodiment, the second generation module 530 includes:

[0137] A first generating unit 531 is used to generate a gradient descent tree sequence generator according to the gradient descent tree search space;

[0138] A second generating unit 532 is used to generate a gradient descent tree sequence according to the gradient descent tree sequence generator; and

[0139] The third generating unit 533 is used to generate the gradient descent tree to be trained according to the gradient descent tree sequence and the gradient descent tree search space.

[0140] In a possible implementation of the embodiment of the present application, as Figure 7 As shown, in Figure 6 Based on the illustrated embodiment, the update module 560 includes:

[0141] A score value acquisition unit 561 is used to respectively acquire K score values ​​of the K gradient descent tree models;

[0142] A calculation unit 562, configured to generate an average score value according to the K score values ​​of the K gradient descent tree models;

[0143] A first updating unit 563, configured to further update the gradient descent tree sequence generator if the average score value is less than the score requirement, or the current number of iterations N is less than the preset number of iterations;

[0144] In a possible implementation of the embodiment of the present application, the gradient descent tree sequence generator is a neural network module or an evolutionary algorithm module, and the first update unit 563 updates the gradient descent tree sequence generator through a back propagation algorithm when the gradient descent tree sequence generator is the neural network module; and updates the gradient descent tree sequence generator through a population update algorithm when the gradient descent tree sequence generator is the evolutionary algorithm module.

[0145] The second updating unit 564 is used to update the gradient descent tree to be trained by using the updated gradient descent tree sequence generator.

[0146] In a possible implementation of the embodiment of the present application, as Figure 8 As shown, in Figure 5 Based on the illustrated embodiment, the first generation module 520 includes:

[0147] A first acquisition unit 521 is used to acquire the number of classifiers required for the gradient descent tree search space;

[0148] A second acquisition unit 522 is used to acquire the type of classifier required by the gradient descent tree search space and the attribute corresponding to the type; and

[0149] The construction unit 523 is used to construct the gradient descent tree search space according to the number of classifiers, the types of classifiers and the attributes corresponding to the types.

[0150] It should be noted that the above explanation of the embodiment of the method for generating a gradient descent tree is also applicable to the device for generating a gradient descent tree of this embodiment, and the implementation principle is similar, which will not be repeated here.

[0151] The device for generating a gradient descent tree in an embodiment of the present application obtains a training set and a validation set, and divides the training set and the validation set into K training subsets and K validation subsets, and generates a gradient descent tree search space, and generates a gradient descent tree to be trained according to the gradient descent tree search space, then trains the gradient descent tree to be trained according to the K training subsets to generate K gradient descent tree models, and respectively evaluates the K gradient descent tree models using the K validation subsets to generate scoring values ​​of the K gradient descent tree models, and iterates and updates the gradient descent tree to be trained N times according to the scoring values ​​of the K gradient descent tree models until the scoring values ​​of the K gradient descent tree models meet the scoring requirements or N reaches a preset number of iterations, thereby realizing automatic generation of the gradient descent tree. Moreover, a gradient descent tree is generated by automatically searching in a gradient descent tree search space. Since the gradient descent tree search space contains the possibility of all searchable gradient descent trees, the combination of weak classifiers contained in each gradient descent tree in the gradient descent tree search space is arbitrary, rather than simply weighted summing the weak classifiers obtained in each round of learning. This comprehensively considers the relationship between different weak classifiers, so that the final generated gradient descent tree is a better combination of weak classifiers, thereby ensuring the performance of the gradient descent tree and improving the robustness of the gradient descent tree.

[0152] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0153] like Fig. 9 As shown, it is a block diagram of an electronic device for realizing the generation method of the gradient descent tree of the embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0154] like Fig. 9As shown, the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 701 is taken as an example.

[0155] The memory 702 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the method for generating a gradient descent tree provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method for generating a gradient descent tree provided in the present application.

[0156] The memory 702 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the method for generating a gradient descent tree in the embodiment of the present application (for example, the attached Figure 5 The processor 701 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 702, that is, the method for generating the gradient descent tree in the above method embodiment is implemented.

[0157] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of an electronic device that performs a method for generating a gradient descent tree, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device that performs the method for generating a gradient descent tree via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The electronic device for executing the method for generating a gradient descent tree may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0159] The input device 703 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device that executes the method for generating the gradient descent tree, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0160] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0163] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0164] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0165] According to the technical solution of the embodiment of the present application, a gradient descent tree search space is generated, and a gradient descent tree to be trained is generated according to the gradient descent tree search space, and then the generated gradient descent tree to be trained is trained using the obtained K training subsets to obtain K gradient descent tree models, and the performance of the K gradient descent tree models is evaluated using the obtained K validation sets respectively to obtain score values ​​of the K gradient descent tree models. When the score values ​​of the K gradient descent tree models do not meet the score requirements and do not reach the preset number of iterations, the gradient descent tree to be trained is iteratively updated until the score values ​​of the K gradient descent tree models meet the score requirements or the number of iterations N reaches the preset number of iterations, thereby realizing the automatic generation of the gradient descent tree. Moreover, a gradient descent tree is generated by automatically searching in a gradient descent tree search space. Since the gradient descent tree search space contains the possibility of all searchable gradient descent trees, the combination of weak classifiers contained in each gradient descent tree in the gradient descent tree search space is arbitrary, rather than simply weighted summing the weak classifiers obtained in each round of learning. This comprehensively considers the relationship between different weak classifiers, so that the final generated gradient descent tree is a better combination of weak classifiers, thereby ensuring the performance of the gradient descent tree and improving the robustness of the gradient descent tree.

[0166] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0167] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for generating a gradient descent tree, comprising: Obtaining a training set and a validation set, and dividing the training set and the validation set into K training subsets and K validation subsets, wherein K is a positive integer, and the training set and the validation set include image or speech samples; Generate a gradient descent tree search space, and generate a gradient descent tree to be trained according to the gradient descent tree search space; Training the to-be-trained gradient descent tree according to the K training subsets to generate K gradient descent tree models; Using the K validation subsets respectively to evaluate the K gradient descent tree models to generate score values ​​for the K gradient descent tree models; and Performing N iterative updates on the to-be-trained gradient descent tree according to the score values ​​of the K gradient descent tree models until the score values ​​of the K gradient descent tree models meet the score requirements or N reaches a preset number of iterations, where N is a positive integer; Wherein, generating a gradient descent tree search space includes: Obtaining the number of classifiers required for the gradient descent tree search space; Obtaining the type of classifier required for the gradient descent tree search space and the attributes corresponding to the type; and The gradient descent tree search space is constructed according to the number of classifiers, the types of the classifiers, and attributes corresponding to the types.

2. The method for generating a gradient descent tree according to claim 1, wherein: The step of generating a gradient descent tree to be trained according to the gradient descent tree search space comprises: Generate a gradient descent tree sequence generator according to the gradient descent tree search space; Generate a gradient descent tree sequence according to the gradient descent tree sequence generator; and The gradient descent tree to be trained is generated according to the gradient descent tree sequence and the gradient descent tree search space.

3. The method for generating a gradient descent tree according to claim 2, wherein: The step of performing N iterative updates on the gradient descent tree to be trained according to the score values ​​of the K gradient descent tree models comprises: Obtaining K score values ​​of the K gradient descent tree models respectively; Generate an average score value according to the K score values ​​of the K gradient descent tree models; If the average score value is less than the score requirement, and the current number of iterations N is less than the preset number of iterations, further updating the gradient descent tree sequence generator; and The gradient descent tree to be trained is updated by the updated gradient descent tree sequence generator.

4. The method for generating a gradient descent tree according to claim 3, wherein: The gradient descent tree sequence generator is a neural network module or an evolutionary algorithm module, wherein further updating the gradient descent tree sequence generator comprises: When the gradient descent tree sequence generator is the neural network module, updating the gradient descent tree sequence generator by a back propagation algorithm; When the gradient descent tree sequence generator is the evolutionary algorithm module, the gradient descent tree sequence generator is updated by a population update algorithm.

5. The method for generating a gradient descent tree according to claim 1, wherein: The step of dividing the training set and the validation set into K training subsets and K validation subsets comprises: The training set and the validation set are divided into K training subsets and K validation subsets by a K-fold cross partitioning algorithm.

6. A device for generating a gradient descent tree, comprising: An acquisition module, used to acquire a training set and a validation set, and divide the training set and the validation set into K training subsets and K validation subsets, wherein K is a positive integer, and the training set and the validation set include image or voice samples; A first generation module, for generating a gradient descent tree search space; A second generating module, used for generating a gradient descent tree to be trained according to the gradient descent tree search space; A training module, used for training the to-be-trained gradient descent tree according to the K training subsets to generate K gradient descent tree models; a validation module, configured to respectively use the K validation subsets to evaluate the K gradient descent tree models to generate scoring values ​​for the K gradient descent tree models; and An updating module, configured to perform N iterative updates on the to-be-trained gradient descent tree according to the score values ​​of the K gradient descent tree models, until the score values ​​of the K gradient descent tree models meet the score requirements or N reaches a preset number of iterations, where N is a positive integer; The first generating module comprises: A first acquisition unit, used to acquire the number of classifiers required for the gradient descent tree search space; A second acquisition unit is used to acquire the type of classifier required by the gradient descent tree search space and an attribute corresponding to the type; and A construction unit is used to construct the gradient descent tree search space according to the number of classifiers, the types of classifiers and attributes corresponding to the types.

7. The device for generating a gradient descent tree according to claim 6, wherein: The second generating module comprises: A first generating unit, configured to generate a gradient descent tree sequence generator according to the gradient descent tree search space; A second generating unit, configured to generate a gradient descent tree sequence according to the gradient descent tree sequence generator; and The third generating unit is used to generate the gradient descent tree to be trained according to the gradient descent tree sequence and the gradient descent tree search space.

8. The device for generating a gradient descent tree according to claim 7, wherein: The update module comprises: A scoring value acquisition unit, used to respectively acquire K scoring values ​​of the K gradient descent tree models; A computing unit, configured to generate an average score value according to the K score values ​​of the K gradient descent tree models; a first updating unit, configured to further update the gradient descent tree sequence generator if the average score value is less than the score requirement and the current number of iterations N is less than the preset number of iterations; and The second updating unit is used to update the gradient descent tree to be trained by using the updated gradient descent tree sequence generator.

9. The device for generating a gradient descent tree according to claim 8, wherein: The gradient descent tree sequence generator is a neural network module or an evolutionary algorithm module. When the gradient descent tree sequence generator is the neural network module, the first update unit updates the gradient descent tree sequence generator through a back propagation algorithm; when the gradient descent tree sequence generator is the evolutionary algorithm module, the first update unit updates the gradient descent tree sequence generator through a population update algorithm.

10. The device for generating a gradient descent tree according to claim 6, wherein: The acquisition module divides the training set and the validation set into K training subsets and K validation subsets by using a K-fold cross partitioning algorithm.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a gradient descent tree according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method for generating a gradient descent tree according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Method and device for establishing image detection model, equipment and computer storage medium

    CN110363810A

  • Intention recognition method, electronic equipment and storage medium

    CN110414005A