Crowdsourcing quality assessment method based on slide task window

Abstract

The invention discloses a crowdsourcing quality assessment method based on a slide task window. Along with development of Internet, crowdsourcing is widely applied in fields of commerce, scientific research and the like, but the crowdsourcing faces a severe challenge of uneven completion quality of tasks. In the invention, aiming at a quality assessment problem of the crowdsourcing, through introducing a professional accuracy concept, a worker is assessed so that the crowdsourcing quality assessment method based on the slide task window is realized. The method is characterized by firstly, combining an idea of a gold standard data method and testing the worker; then, referring a principle of a slide window, carrying out slide window splitting on a crowdsourcing task, wherein all the workersvote independently for all tasks; and then, improving an EM algorithm and estimating a task result in each task window; and finally, integrating estimation results so as to carry out high-efficient,accurate, real-time and dynamical assessment on crowdsourcing task quality.

Description

technical field [0001] The invention relates to a crowdsourcing quality evaluation method based on a sliding task window, which belongs to the field of crowdsourcing and Internet applications. Background technique [0002] Crowdsourcing is a distributed problem-solving mechanism that is open to the public on the Internet. It integrates computers and the unknown public on the Internet to complete tasks that are difficult for computers to complete alone. At present, crowdsourcing has already shined in the commercial field. For example, the French online media Aike.com uses the concept of user-generated content to inject strong impetus into brand innovation. Amazon’s Mturk platform allows the public to solve practical problems based on economic incentives. , Baidu Encyclopedia, Wikipedia, Sogou Pinyin input method, etc. are all excellent examples of free collaboration. At the same time, crowdsourcing can solve many problems that are difficult for computers to handle by pooling...

AI Technical Summary

Problems solved by technology

[0003] During the execution of crowdsourcing tasks, workers adopt a free and loose organizational form, and their work attitudes and abilities vary. Some workers have a loose attitude and answer questions randomly or even maliciously. For certain professional task requirements, many workers Insufficient professional knowledge and ability lead to uneven quality of work and greater uncertainty in execution results

Images