Evaluation system and method applied to field of data labeling

Abstract

The invention discloses an evaluation system applied to the field of data labeling. The evaluation system comprises a cloud server, and a client, a labeling end and an auditing end in communication connection with the cloud server, wherein the client receives and releases labeling tasks from customers and evaluation feedbacks of the customers; the labeling end is used for labeling personnel to perform labeling; the auditing end is used for auditing personnel to perform auditing; the cloud server comprises a registration module and a statistical module; the registration module is used for storing information of the labeling personnel and the auditing personnel participating in the labeling tasks in attribute information of the corresponding labeling tasks; and the statistical module is usedfor evaluating the labeling personnel and the auditing personnel participating in the labeling tasks according to the evaluation feedbacks from the customers and the labeling personnel information and the auditing personnel information contained in the attribute information of the labeling tasks.

Description

Technical field [0001] The present invention relates to the field of data labeling review, and more specifically, to an evaluation system applied in the field of data labeling. Background technique [0002] As we all know, the process in the field of data labeling generally includes customers issuing data labeling tasks, labeling personnel receiving tasks and labeling data, reviewers reviewing the labeling results, and feeding back qualified data labeling results to customers. The quality of the results of data labeling is the top priority of customers, and only qualified and effective data labeling can produce value for customers. Therefore, the review after data annotation is particularly important. However, in the prior art, the review mechanism for data annotation results is still relatively single. Generally, after the marked results are reviewed, there will be no subsequent corresponding procedures to analyze and process the review results. [0003] On the one hand, since ...

AI Technical Summary

Problems solved by technology

[0003] On the one hand, because the review work is done manually, the existing review mechanism cannot supervise the reviewers and cannot guarantee the quality of the review, which further affects the quality of labeling

On the other hand, the existing technology has a very low utilization rate for the data related to the audit results, and there is no evaluation and analysis of the audit results, and it is impossible to select excellent reviewers and labelers to further improve the quality of data labeling

Images