Sample labeling method based on crowdsourcing mode

Abstract

The invention discloses a sample labeling method based on a crowdsourcing mode, which comprises the following steps of: 1) obtaining an unannotated multivariate time sequence data set, and extractingsamples required to be annotated in a selected data set; 2) for all annotators, based on the annotation accuracy and annotation cost of each annotator, selecting an annotator set which reaches a confidence threshold and has the lowest cost as a cost benefit crowdsourcing annotation model; 3) for the extracted samples needing to be labeled, obtaining a labeling result based on a cost benefit crowdsourcing labeling model, adding the labeled samples into a labeling data set, classifying reverse nearest neighbor samples of the labeled samples into the same class as the unlabeled samples, and adding the labeled data set to obtain an updated labeling data set; and 4) calculating a stop standard, and obtaining a final mark data set result corresponding to the multivariate time sequence data set after a stop condition is reached. The method provided by the invention is used for realizing reliable and low-cost labeling of unlabeled samples.

Description

technical field [0001] The invention relates to data mining technology, in particular to a sample labeling method based on crowdsourcing mode. Background technique [0002] A high-quality dataset is crucial for model training. However, in real life, the amount of labeled data is often small, and efficient and accurate labeling of data is time-consuming and expensive. In response to this problem, active learning has gradually become a research hotspot, and one of the keys is the effective labeling of unlabeled samples. Traditional machine learning algorithms often do not take into account the different accuracy of labelers. In order to improve the accuracy of labeling, researchers have proposed some crowdsourcing labeling algorithms, the most important problem of which is the labeling error of the labeler. [0003] In order to improve the performance of classifiers trained based on crowdsourced labeled data, Zhang et al. proposed a meta-learning integration method for group...

AI Technical Summary

Problems solved by technology

However, the existing traditional crowdsourcing labeling methods do not consider group intelligence. In fact, each individual has different labeling accuracy for different types of instances.

Images