Model evaluation method and related device

Abstract

The invention discloses a model evaluation method and a related device. The evaluation method comprises the steps of acquiring prediction result values output after multiple sample data are input intoa to-be-evaluated model; sorting the label values according to the prediction result values of the sample data and a set sorting rule, and dividing the sorted sample data into a plurality of segments; determining the total quantity of the sample data with the label value being the first value in the sample data and the actual quantity and the average quantity of the sample data with the label value being the first value in each segment by utilizing the label value of the sample data; determining a maximum promotion index in all the segments based on the actual number and the average number; determining the sorting non-monotonous rate of all the segments according to the actual quantity of the sample data of which the label values of the two adjacent segments are the first values; and determining the sorting capability of the to-be-evaluated model according to the ratio of the maximum lifting index to the sorting non-monotony rate. The sorting capacity of the model can be quantified, and deviation caused by manual judgment is avoided.

Description

technical field [0001] The invention relates to the technical field of model evaluation, in particular to a model evaluation method and related devices. Background technique [0002] With the development of the big data era, models are used more and more frequently, and it also involves all walks of life. For the model itself, it will be endowed with different functions according to requirements, such as classification function, image restoration function, etc. In many models, computational ordering power is a fundamental element of the model. However, there is no mature method for evaluating the computational ranking ability of models. [0003] For example, in the existing field of financial model evaluation, when evaluating the ranking ability of financial models, it is usually to observe the sample ranking of financial models through artificial observation, and then compare the ranking ability of financial models. [0004] However, due to the subjectivity of human obse...

AI Technical Summary

Problems solved by technology

[0005] The invention provides a model evaluation method, an evaluation device, an intelligent system and a storage device, which can effectively solve the problem of inaccurate evaluation of the sorting ability of financial models in the prior art

Images