Tree model structure evaluation method, device and equipment

Abstract

The invention relates to the technical field of modeling process and model stability exploration, in particular to a tree model structure assessment method, device and equipment, and the method comprises the steps: firstly obtaining a to-be-assessed sample, generating an alternative combination according to the to-be-assessed sample, obtaining a base model, training the base model through the alternative combination, and obtaining a tree model structure; similarity punishment is added, a model KS mean value of the base model after training is completed is calculated, an excellent combination is determined according to the model KS mean value, iteration is carried out according to the excellent combination to generate a new alternative combination again, and after a preset iteration round is reached, a final excellent combination is determined, and a model effect is saved. Therefore, the order combination with a poor effect is selectively eliminated, and the problems that the training time is long and the training difficulty is large due to the fact that the training combination cannot be optimized during iterative training at present are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of modeling process and model stability exploration, and in particular to a tree model structure evaluation method, device and equipment. Background Art

[0002] In tree models, subsample parameters are generally provided to reduce the variance of model results by limiting the number of samples used for each tree. When exploring tree models, a large number of training combinations will be generated during iterative training, and the training combinations cannot be optimized, resulting in long training time and high training difficulty. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a tree model structure evaluation method, device and equipment to overcome the current problem that the training combination cannot be optimized during iterative training, resulting in long training time and high training difficulty.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present application provides a tree model structure evaluation method, comprising:

[0006] Obtain samples to be evaluated;

[0007] generating an alternative combination according to the sample to be evaluated;

[0008] Obtain a base model, train the base model through the candidate combination, add a similarity penalty and calculate the model KS mean of the base model after training, and determine an excellent combination according to the model KS mean;

[0009] Iterate the excellent combination to generate new candidate combinations again, until the preset iteration rounds are reached, the final excellent combination is determined, and the model effect is saved.

[0010] Furthermore, the method described above, before generating the candidate combination according to the sample to be evaluated, further includes:

[0011] The samples to be evaluated are grouped and numbered according to the channel characteristics of the samples to be evaluated.

[0012] Furthermore, the method described above, wherein the base model is obtained, the base model is trained by the candidate combination, a similarity penalty is added, and the model KS mean of the base model after training is calculated, and an excellent combination is determined according to the model KS mean, comprises:

[0013] Acquire a base model, and train the base model using the candidate combination;

[0014] Calculate the model KS mean of the base model after training, and determine the final model KS mean according to the similarity penalty;

[0015] According to the final KS mean of the model, a preset number of screenings are performed to determine a preset number of excellent combinations.

[0016] Furthermore, the method described above, iterating based on the excellent combination to generate new candidate combinations again until reaching a preset iteration round, determining the final excellent combination, and saving the model effect, includes:

[0017] Cache the base model training results;

[0018] Iterate and generate new candidate combinations again according to the excellent combinations;

[0019] Training the base model according to the new candidate combination, and generating a new base model training result according to the cached base model training result;

[0020] Calculate the model KS mean of the new candidate combination, determine a new excellent combination, and complete the second round of iteration;

[0021] Determine whether the current number of iterations reaches the preset rounds;

[0022] If yes, save the model effect.

[0023] In a second aspect, the present application provides a tree model result evaluation model, including:

[0024] A sample acquisition module is used to obtain samples to be evaluated;

[0025] An alternative combination generating module, used for generating an alternative combination according to the sample to be evaluated;

[0026] An excellent combination determination module is used to obtain a base model, train the base model through the candidate combination, add a similarity penalty and calculate the model KS mean of the base model after training, determine an excellent combination according to the model KS mean, and iterate through the candidate combination generation module according to the excellent combination to generate a new candidate combination again;

[0027] The model effect saving module is used to determine the final excellent combination and save the model effect after reaching the preset iteration round.

[0028] In a third aspect, the present application provides a tree model structure evaluation device, including a processor and a memory, wherein the processor is connected to the memory:

[0029] Wherein, the processor is used to call and execute the program stored in the memory;

[0030] The memory is used to store the program, and the program is at least used to execute any of the tree model structure evaluation methods described above.

[0031] The beneficial effects of the present invention are:

[0032] This application first obtains the sample to be evaluated, generates an alternative combination based on the sample to be evaluated, obtains the base model, trains the base model through the alternative combination, adds a similarity penalty and calculates the model KS mean of the base model after training, determines the excellent combination based on the model KS mean, iterates and generates a new alternative combination again according to the excellent combination, until the preset iteration round is reached, determines the final excellent combination, and saves the model effect. In this application, by adding a similarity penalty and calculating the model KS mean of the base model after training, determines the excellent combination based on the model KS mean, iterates and generates a new alternative combination again according to the excellent combination, thereby selectively eliminating the order combination with poor effect, solving the problem that the training combination cannot be optimized during iterative training, resulting in long training time and high training difficulty. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a flowchart provided by an embodiment of a tree model structure evaluation method of the present invention;

[0035] Figure 2 It is a diagram of the number of samples of the first input sample of the excellent combination provided by an embodiment of a tree model structure evaluation method of the present invention;

[0036] Figure 3 It is a structural schematic diagram provided by an embodiment of a tree model structure evaluation device of the present invention;

[0037] Figure 4 It is a structural schematic diagram provided by an embodiment of a tree model structure evaluation device of the present invention. DETAILED DESCRIPTION

[0038] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0039] Figure 1 This is a flowchart provided by an embodiment of a tree model structure evaluation method of the present invention. Figure 1 , this embodiment may include the following steps:

[0040] S1. Obtain samples to be evaluated.

[0041] S2. Generate alternative combinations based on the samples to be evaluated.

[0042] S3. Obtain the base model, train the base model through the alternative combination, add a similarity penalty and calculate the model KS mean of the base model after training, and determine the excellent combination based on the model KS mean.

[0043] S4. Iterate and generate new candidate combinations again according to the excellent combinations until the preset iteration rounds are reached, then determine the final excellent combination and save the model effect.

[0044] It should be noted that the base model is a new model obtained by iterating new training samples based on the model generated by other samples, wherein the beginning part of the new model is consistent with the base model.

[0045] The alternative combination is the model result obtained using different sample combinations, which can take into account the effects on multiple different types of validation sets.

[0046] The similarity penalty is that the trained model structures are consistent for the same sample combination order. If the sample orders before two candidate combinations are similar, the corresponding model structures must also be similar. In this case, excessive search may occur for a certain type of fixed-order candidate combination, resulting in the model being unable to search for the global optimal sample order combination.

[0047] When calculating different sample combinations, assuming there are 5 candidate samples and 10 rounds of training are required, the final number of different combinations is: 5^10 = 9765625. To ensure efficiency, we will only explore the combinations of samples with the highest effect evaluation. Assuming that the top 8 combinations are selected for expansion, the final total number of combinations is: 8*5*10 = 500 (including the basic model).

[0048] It can be understood that the present embodiment first obtains the sample to be evaluated, generates the candidate combination according to the sample to be evaluated, obtains the base model, trains the base model through the candidate combination, adds the similarity penalty and calculates the model KS mean of the base model after the training is completed, determines the excellent combination according to the model KS mean, iterates and generates a new candidate combination again according to the excellent combination, until reaching the preset iteration round, determines the final excellent combination, and saves the model effect. In the present embodiment, by adding the similarity penalty and calculating the model KS mean of the base model after the training is completed, determines the excellent combination according to the model KS mean, iterates and generates a new candidate combination again according to the excellent combination, thereby selectively eliminating the order combination with poor effect, and solves the problem that the training combination cannot be optimized during iterative training, resulting in long training time and high training difficulty.

[0049] In some optional embodiments, the actual construction method of the candidate combination is as follows: Assume that there are 31 candidate samples in total, corresponding to numbers 0 to 30. Among them, the model iterates an excellent combination every 5 rounds, so the candidate combination 0-1-2 means that the 1st to 5th trees of the model are trained using samples numbered 0, the 6th to 10th trees are trained using samples numbered 1, and the 11th to 15th trees are trained using samples numbered 2.

[0050] Preferably, before step S2, the method further comprises:

[0051] The samples to be evaluated are grouped and numbered according to the channel characteristics of the samples to be evaluated.

[0052] It is understandable that for the sake of record, we will group and number the samples participating in the training according to the channel characteristics. Different combination information corresponds to the order of using sample numbers to train the model. For example, 0-1-2 means that we first use the sample numbered 0 for model training, then use the sample numbered 1 for model training on this basis, and finally use the sample numbered 3 for model training. The corresponding model training is carried out in a serial line.

[0053] Preferably, step S3 includes:

[0054] Acquire a base model, and train the base model using the candidate combination;

[0055] Calculate the model KS mean of the base model after training, and determine the final model KS mean according to the similarity penalty;

[0056] According to the final KS mean of the model, a preset number of screenings are performed to determine a preset number of excellent combinations.

[0057] It is understandable that for different candidate combination models, we will also prepare different evaluation samples to evaluate the actual effect of the model. Here, we generally select the candidate samples as evaluation samples, and the corresponding evaluation sample KS mean as the combination ranking value. Assuming that the excellent candidate combinations are known to be 14-4-1, 14-4-2, and 14-4-3, the corresponding evaluation results are shown in Table 1. We believe that the ranking of the candidate combinations is: 14-4-3>14-4-1>14-4-2.

[0058] Alternative combination number 14-4-1 14-4-2 14-4-3 KS statistic mean 0.127 0.123 0.134

[0059] Table 1 KS mean statistics

[0060] Because the same excellent combination will be recombined with all sample number combinations, the final excellent candidate combination may come from the same excellent combination. In order to eliminate the impact of such similar combinations on the final results and expand the possible coverage of the model combination, the evaluation values ​​of similar combinations will be punished accordingly.

[0061] For the excellent combinations 0-1-2 and 0-1-1, the meaning of the above alternative combinations shows that the corresponding first ten tree structures are the same, and the only difference is the structure of the 11th to 15th tree. Therefore, the similarity between different sample combinations can be determined by determining the number of consistent sample numbers in the sample combination sequence.

[0062] The relevant experiments are as follows. We prepare 31 candidate samples for model training. We reselect excellent samples for model iteration every 5 rounds. The iteration combination is the 30 best combination sequences evaluated in the previous round of iteration. Then we count: in each round of iteration, the number of the first input sample numbers corresponding to the top 30 combination sequences can be obtained. Figure 2 As shown, the vertical axis corresponds to the number of stacking and the horizontal axis corresponds to the number of iterations.

[0063] It can be seen that in the case of no similarity penalty, after ten rounds of iterations, the first to fifth trees of the 30 candidate excellent combinations that were finally selected were all trained by sample number 2, that is, the structure of the first five model trees was the same. However, we believe that putting too many similar structures into the candidate combinations may cause the final result to fall into the local optimum.

[0064] In the case of similar triggers, after ten rounds of iterations, the 30 candidate excellent combinations that were finally selected, the number of models trained by sample 2 from the 1st to the 5th number is 6, that is, the difference in the final excellent model structure becomes larger.

[0065] For similarity penalty, according to Table 2, assuming that there are currently 6 candidate combinations, one optimal combination will be selected into the candidate combination each time. After each round of screening, the statistical value of samples with similar rankings to the excellent candidate combination will be penalized. Here, the corresponding statistical value will be reduced by 0.001 for each similar sequence number. In the fourth round of screening, 4-14-2 will be selected into the candidate combination. If there is no similarity penalty, the fourth round selected combination is 14-4-2.

[0066]

[0067] Table 2 Combination selection table

[0068] Preferably, step S4 comprises:

[0069] Cache the base model training results;

[0070] Iterate and generate new candidate combinations again according to the excellent combinations;

[0071] Training the base model according to the new candidate combination, and generating a new base model training result according to the cached base model training result;

[0072] Calculate the model KS mean of the new candidate combination, determine a new excellent combination, and complete the second round of iteration;

[0073] Determine whether the current number of iterations reaches the preset rounds;

[0074] If yes, save the model effect.

[0075] It is understandable that the important cached results here are the model results obtained under different sorting combinations. Assuming that we do not cache the model results, it takes about 0.5s to train a model of 50 trees. If we read the structure of the first 45 trees by caching, it takes about 0.05s to train the final 50 trees. The model parameters are fixed in each round of iteration. We generally use several fixed model parameters for training without making too many adjustments. We mostly change the structure of the final model by adjusting the training time of different samples.

[0076] The present invention also provides a tree model structure evaluation device for implementing the above method embodiment. Figure 3 FIG. 1 is a schematic diagram of a structure of a tree model structure evaluation device according to an embodiment of the present invention. Figure 3 As shown, this embodiment includes:

[0077] The sample acquisition module 1 is used to obtain samples to be evaluated;

[0078] The candidate combination generation module 2 is used to generate candidate combinations according to the samples to be evaluated;

[0079] The excellent combination determination module 3 is used to obtain the base model, train the base model through the candidate combination, add a similarity penalty and calculate the model KS mean of the base model after training, determine the excellent combination according to the model KS mean, and iterate through the candidate combination generation module 2 according to the excellent combination to generate a new candidate combination again;

[0080] The model effect saving module 4 is used to determine the final excellent combination and save the model effect after reaching the preset iteration round.

[0081] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0082] The present invention also provides a tree model structure evaluation device for implementing the above method embodiment. Figure 4 FIG. 1 is a schematic diagram of a structure of a tree model structure evaluation device according to an embodiment of the present invention. Figure 4 As shown, the tree model structure evaluation device of this embodiment includes a processor 21 and a memory 22, and the processor 21 is connected to the memory 22. The processor 21 is used to call and execute the program stored in the memory 22; the memory 22 is used to store the program, and the program is at least used to execute the tree model structure evaluation method in the above embodiment.

[0083] The specific implementation scheme of the tree model structure evaluation device provided in the embodiment of the present application can refer to the implementation scheme of the tree model structure evaluation method in any of the above embodiments, which will not be repeated here.

[0084] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0085] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0086] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0087] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0088] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0089] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0090] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A tree model structure evaluation method, characterized in that: include: Obtain samples to be evaluated; generating an alternative combination according to the sample to be evaluated; Obtain a base model, train the base model through the candidate combination, add a similarity penalty and calculate the model KS mean of the base model after training, and determine an excellent combination according to the model KS mean; Iterate the excellent combination to generate new candidate combinations again, until the preset iteration rounds are reached, the final excellent combination is determined, and the model effect is saved.

2. The method according to claim 1, characterized in that Before generating the candidate combination according to the sample to be evaluated, the method further includes: The samples to be evaluated are grouped and numbered according to the channel characteristics of the samples to be evaluated.

3. The method according to claim 2, characterized in that The obtaining of the base model, training the base model through the candidate combination, adding a similarity penalty and calculating the model KS mean of the base model after training, and determining the excellent combination according to the model KS mean, includes: Acquire a base model, and train the base model using the candidate combination; Calculate the model KS mean of the base model after training, and determine the final model KS mean according to the similarity penalty; According to the final KS mean of the model, a preset number of screenings are performed to determine a preset number of excellent combinations.

4. The method according to claim 2, characterized in that: The iterative generation of new candidate combinations based on the excellent combinations until a preset number of iterations is reached, the final excellent combination is determined, and the model effect is saved, including: Cache the base model training results; Iterate and generate new candidate combinations again according to the excellent combinations; Training the base model according to the new candidate combination, and generating a new base model training result according to the cached base model training result; Calculate the model KS mean of the new candidate combination, determine a new excellent combination, and complete the second round of iteration; Determine whether the current number of iterations reaches the preset rounds; If yes, save the model effect.

5. A tree model result evaluation model, characterized in that: include: A sample acquisition module is used to obtain samples to be evaluated; An alternative combination generating module, used for generating an alternative combination according to the sample to be evaluated; An excellent combination determination module is used to obtain a base model, train the base model through the candidate combination, add a similarity penalty and calculate the model KS mean of the base model after training, determine an excellent combination according to the model KS mean, and iterate through the candidate combination generation module according to the excellent combination to generate a new candidate combination again; The model effect saving module is used to determine the final excellent combination and save the model effect after reaching the preset iteration round.

6. A tree model structure evaluation device, characterized in that: The invention comprises a processor and a memory, wherein the processor is connected to the memory: Wherein, the processor is used to call and execute the program stored in the memory; The memory is used to store the program, and the program is at least used to execute the tree model structure evaluation method described in any one of claims 1-4.

Images