Active learning based data automatic marking method

Abstract

The invention discloses an active learning based data automatic marking method, and belongs to the field of active learning. The data automatic marking method comprises the steps of 101, processing marked data and unmarked data; 102, classifying the unmarked data by using a plurality of different classifiers; 103, selecting data with low difference entropy; 104, performing manual marking on the data with low difference entropy; and 105, performing self-checking on a manual marking result. The invention provides a data automatic marking system with a self-checking function through combining an active learning method in allusion a problem of how to ensure the accuracy of manually marked data as far as possible while reducing the volume of the manually marked data, thereby achieving the purposes of reducing the workload and improving the accuracy of the manually marked data.

Description

technical field [0001] The invention relates to the field of active learning, in particular to an automatic data labeling method based on active learning. Background technique [0002] With the advent of the era of big data, a new type of profession has emerged on the Internet—data labeller. The job of a data labeler is to use automated tools to crawl and collect data from the Internet, including text, pictures, voice, etc., and then organize and label the captured data. Specific workflow: First, the labelers are trained to determine the sample data to be labeled and labeling rules; then, label the sample data according to the pre-arranged rules; finally, merge the labeled results. However, this marking process has the following problems: 1. The entire marking process is very complicated, and when the amount of data is particularly large, it requires a large amount of manual labor; 2. During the data labeling process, due to the limited energy of the labeling personnel or t...

AI Technical Summary

Problems solved by technology

However, this marking process has the following problems: 1. The entire marking process is very complicated, and when the amount of data is particularly large, it requires a large amount of manual labor; 2. During the data labeling process, due to the limited energy of the labeling personnel or the Subjectivity and other factors, resulting in the inability to guarantee the 100% accuracy of the labeled data, that is, the quality of the label cannot be judged

Since the manually labeled samples are limited, and there is no guarantee that all manually labeled data samples are correct

Moreover, in practical problems, some sample data, such as genetic composition data used in genetic analysis, are expensive to label, so generally speaking, the number of unlabeled samples will far exceed the data of labeled samples

Images