A method and device for evaluating a classification model

A classification model and classification rate technology, applied in the computer field, can solve the problems of high cost of manual labeling, time-consuming, and inability to evaluate the effect of the model in real time, so as to improve the reliability of evaluation, speed up evaluation, and improve development efficiency.

Inactive Publication Date: 2019-05-21
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Manual labeling is costly and time-consuming, and it is impossible to make an immediate evaluation of the model effect, which reduces the speed of model iteration;
[0006] Manual labeling relies on human subjective experience, and the labeled samples are not reliable enough to truly reflect the effect of the model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and device for evaluating a classification model
  • A method and device for evaluating a classification model
  • A method and device for evaluating a classification model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0034] In an industrial setting, developing machine learning models typically works in a mode where a model already exists and a better model is expected. Therefore, the embodiment of the present invention evaluates multiple classification models from the perspective of comparing two classification models, and proposes an indirect method for evaluating models. Instead of p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and device for evaluating a classification model, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of obtaininga plurality of first category data sets obtained by classifying an input data set by a plurality of classification models, wherein each first category data set corresponds to one classification model; Determining a false positive example data set in each first type of data set according to the pre-selected reference data set; And evaluating the plurality of classification models according to thefirst category data sets and the false positive example data sets in the first category data sets. The implementation mode can completely get rid of dependence on manual marking, the operation processcan be fully automatically operated, the evaluation cost is reduced, the evaluation speed is increased, the evaluation range is expanded, the evaluation reliability is improved, and the model development efficiency is improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and device for evaluating a classification model. Background technique [0002] Classification is widely used in industry. In the development process of classification models, evaluating the effect of the model is a crucial part, which directly determines whether a model can finally be launched. At present, the industry mainly uses the method of calculating the confusion matrix on the test set to evaluate the effect of the model. After using the classification model to predict the samples in the test set, each sample will have two labels, one is the predicted label of the model, and the other is the true value label of the sample, thereby constructing a confusion matrix, and each row in the confusion matrix Represents a model prediction result, each column represents a true value result, and the content of each unit in the confusion matrix is ​​the number of samples t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62
Inventor 王晨阳陈福豪寿如阳林睿
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap