Collaborative crowd-sourcing quality control method based on user inconsistency information

A quality control method and consistent technology, applied in data processing applications, instruments, calculations, etc., can solve problems such as undiscoverable ambiguity tasks, low-quality crowdsourcing results, and inability to filter out data ambiguity, etc., to improve accuracy and overcome The effect of cost increase and labor reduction

Inactive Publication Date: 2018-10-12
EAST CHINA NORMAL UNIV
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For example, in labeling tasks such as image labeling and entity recognition, if the labeling data itself has ambiguity or polysemy, and the publisher pre-frames alternative answers (choose "yes" or "no"), workers In the case of uncertainty, a certain answer will still be selected, which will eventually lead to low-quality crowdsourcing results
Such a process cannot filter out the ambiguity present in the data, nor can it discover the ambiguity tasks in it and revise it

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Collaborative crowd-sourcing quality control method based on user inconsistency information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] refer to figure 1 , the method of the present invention mainly includes two stages: an ambiguity discovery stage and an ambiguity correction stage. In the stage of ambiguity discovery, potential ambiguity tasks in crowdsourcing are discovered through inconsistent information generated by users and feedback from users with "negative answers", and then redundant candidate categories are obtained through data analysis, which can be used for reference by workers in the correction stage ; In the revision stage, the present invention requires workers to iteratively revise those uncertain tasks generated in the first stage. The details are as follows:

[0023] (1) Ambiguity discovery stage:

[0024] During the execution of crowdsourcing tasks, different workers may produce different answers to the same task, and these different answers may contain knowledge that even the requester has not discovered. In order to obtain this part of knowledge more effectively, when workers d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a collaborative crown-sourcing quality control method based on user inconsistency information. The method comprises an ambiguity task discovery stage and an ambiguity task correction stage, and comprises the following steps: firstly analyzing and utilizing inconsistency information produced by a worker in the crowd-sourcing task, and discovering the possibly existent ambiguity task in the task through the feedback information provided by the worker, and optimizing the answer of the ambiguity by using an iterative modification mode. Meanwhile, a motivation mechanism forencouraging the brave is integrated into the ambiguity answer correction stage to motivate the subjective initiative of the user and encourage the user to provide the valuable explanation information,thereby effectively preventing the interference of the worker.

Description

technical field [0001] The invention relates to the technical fields of collaborative computing and crowdsourcing, and relates to a crowdsourcing quality control method for improving the quality of crowdsourcing tasks based on user inconsistency. Background technique [0002] The concept of "crowdsourcing" (crowdsourcing) was first proposed by Jeff Howe (Jeff Howe), a reporter from the American "Wired" magazine, in June 2006. The wisdom of crowds is used to complete tasks that are difficult for computers to complete. As an emerging business and work model, crowdsourcing has quickly attracted the attention of experts and scholars in various fields. A large number of researchers have carried out a series of applied research on crowdsourcing from different perspectives. Among them, the use of crowdsourcing technology for data annotation is a typical application of crowdsourcing. [0003] In fields such as machine learning, researchers often need large amounts of labeled data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/06
CPCG06Q10/06395
Inventor 孙玉灵王美红贺樑
Owner EAST CHINA NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap