Domain-specific human model collaborative annotation tool

Abstract

A human model collaborative annotation system for training a human annotator, comprising: a database storing images previously annotated by an expert human annotator and/or a machine learning annotator; a display that displays an image selected from the database; an annotation system that causes the human annotator to annotate an image presented on the display; an annotation training system. The annotation training system: selects image samples from the database for annotation by the human annotator; receiving one or more suggested annotations from the annotation system; comparing the one or more suggested annotations of the human annotator with previous annotations of the image sample by the expert human annotator or the machine learning annotator; presenting an attention map on the display to cause the human annotator to notice any annotation errors identified by the comparison; a next training image sample is selected from the data blocks as a function of any error identified in the comparing step.

Description

technical field [0001] This application relates to annotation tools, and in particular to domain-specific human-model collaborative annotation tools that train human annotators and machine learning systems to improve the efficiency of domain-specific image labeling processes. Background technique [0002] In the field of medical imaging, deep learning is widely used to solve classification, detection and segmentation problems. Labeled (annotated) data is critical for training deep learning models. However, medical image data types vary depending on the type of imaging equipment used and the anatomy / tissue being examined, adding to the difficulty of labeling such domain-specific data. [0003] Domain-specific image annotation requires professionally trained and domain-knowledgeable annotators. The level of experience of an annotator greatly affects annotation quality. Unfortunately, the lack of experienced annotators to label disparate biomedical data creates problems in p...

AI Technical Summary

Problems solved by technology

[0005] Unfortunately, such state-of-the-art systems are often not helpful in addressing the shortage of domain experts for annotation

Images