Sample screening method, system and device and medium

Abstract

The invention provides a sample screening method, system and device and a medium. The method comprises the steps of: obtaining part of unlabeled samples from a target sample set to form a candidate sample set; utilizing the classification model to predict each unlabeled sample in the candidate sample set for multiple times, and calculating an uncertainty value of each unlabeled sample in the candidate sample set according to multiple prediction results; and screening the first K unlabeled samples with the highest uncertainty values from the candidate sample set to serve as training samples ofthe classification model. According to the invention, a small number of most representative samples can be selected from mass data for manual annotation, and the problem that in the prior art, when full-amount samples need to be annotated, the cost is huge is solved. In addition, the number of data required to be manually annotated when the classification model is trained can be reduced, the laborcost of manual annotation is reduced, and the annotation efficiency is effectively improved. And meanwhile, the model can be quickly iterated and optimized at the minimum data annotation cost, so that an optimal model can be trained by using less data.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a method, system, equipment and medium for screening samples. Background technique [0002] In recent years, the rapid improvement of computer software and hardware capabilities has brought vitality to artificial intelligence technology, enabling it to achieve fruitful results in industry and academia, and at the same time usher in new development opportunities for many industries. Artificial intelligence technology represented by machine learning is often applied in fields such as computer vision, natural language processing, and speech recognition. There are massive amounts of data available for modeling in these fields. However, in actual application scenarios, obtaining sufficient The samples used for model training often encounter many limitations. [0003] Building and training an ideal machine learning model requires a large amount of labeled data....

AI Technical Summary

Problems solved by technology

Artificial intelligence technology represented by machine learning is often applied in fields such as computer vision, natural language processing, and speech recognition. There are massive amounts of data available for modeling in these fields. However, in actual application scenarios, obtaining sufficient The samples used for model training often encounter many limitations

[0003] Building and training an ideal machine learning model requires a large amount of labeled data. The current data labeling work is mainly done by manpower, which undoubtedly consumes a lot of manpower, material resources and time resources, especially for deep learning. The demand for data volume is even greater

In addition, the training process of machine learning or deep learning models also requires a high cost, and this cost will increase with the increase in the number of samples entered into the model

Images