Crowdsourcing worker reliability model establishing method and device in crowdsourcing knowledge verification environment

Abstract

The invention discloses a crowdsourcing worker reliability model establishing method and device in a crowdsourcing knowledge verification environment. The method comprises the following steps of: matching a knowledge domain set for crowdsourcing users according to a pre-stored strategy, and allocating knowledge in the knowledge domain set to the crowdsourcing users, so that the crowdsourcing usersverify the knowledge to obtain knowledge tags; establishing a crowdsourcing worker reliability model based on a reinforcement learning algorithm, and calculating a reward value of the knowledge labelthrough the crowdsourcing worker reliability model so as to update the pre-stored strategy according to the reward value; repeatedly executing the above operation until the updating frequency of thepre-stored strategy reaches the preset frequency, and carrying out qualification screening on the crowdsourcing users according to the latest pre-stored strategy; and adding the knowledge which is verified to be correct into the corresponding knowledge graph after knowledge verification is completed. According to the method, the crowdsourcing worker reliability model can be established in the crowdsourcing verification environment based on reinforcement learning, qualification screening of crowdsourcing workers is achieved, and therefore the crowdsourcing knowledge verification efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of knowledge graphs, in particular to a method and device for establishing a reliability model of crowdsourcing workers in a crowdsourcing knowledge verification environment. Background technique [0002] The knowledge graph was first proposed by Google. In order to improve the retrieval efficiency of users and enable computers to understand massive information like humans, its research and application have attracted extensive attention from academia and industry. Since the knowledge map is a dynamic construction process, it is necessary to continuously verify the newly added knowledge. At present, knowledge verification is mainly carried out through online crowdsourcing services, that is, knowledge is delivered to a group of crowdsourcing workers, and multiple crowdsourcing workers verify the knowledge to obtain knowledge labels. Among them, the accuracy of knowledge verification largely depends on the est...

AI Technical Summary

Problems solved by technology

However, when applied to large-scale knowledge graphs, the Opt-KG method cannot freely select instance-worker pairs, and can only evaluate the reliability of crowdsourcing workers, and cannot screen and update the qualifications of crowdsourcing workers, so that crowdsourcing workers can control The proportion of true labels on knowledge to maintain its reliability, and the introduction of false knowledge labels in the knowledge verification process

In addition, different crowdsourcing workers have different levels of mastery of different knowledge fields and different levels of work seriousness. The Opt-KG method only relies on labeling knowledge difficulty is not enough to determine the reliability of crowdsourcing workers.

Images