Spline Control Points for Line Plot Annotation

Solution Overview

Problem

The annotation of digital images of line plots for machine learning applications is laborious and costly due to the need for manual labeling by experts, with existing tools not efficiently facilitating this process.

Innovation Solution

A computer-implemented method that uses a machine-learning model, such as a recurrent neural network, to generate and adjust control points for a spline superimposed on the image, allowing users to dynamically manipulate these points for conformity with the line plot, thereby generating ground truth labels efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual annotation by experts is used, then annotation accuracy is improved, but annotation time and cost increase

Engineering Contradiction:

Improveannotation accuracyVSAvoidannotation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary automated annotation using a machine learning model to generate an initial spline approximation before the expert reviewer examines it. This preliminary action provides a head start, reducing the time experts need to spend on each annotation while maintaining accuracy through their final review and adjustment of the pre-generated spline.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an automated spline-generating system as an intermediary between the raw data and the final annotation. This intermediary produces an initial approximation that the expert reviewer then refines, effectively mediating between automated speed and expert accuracy to resolve the contradiction between annotation time and annotation accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If manual annotation by experts is used, then annotation quality is improved, but cost increases

Engineering Contradiction:

Improveannotation qualityVSAvoidannotation cost

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system performs preliminary automated annotation using a machine learning model to generate an initial spline approximation before the expert reviewer examines it. This preliminary action provides a head start, reducing the time experts need to spend on each annotation while maintaining accuracy through their final review and adjustment of the pre-generated spline.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an automated spline-generating system as an intermediary between the raw data and the final annotation. This intermediary produces an initial approximation that the expert reviewer then refines, effectively mediating between automated speed and expert accuracy to resolve the contradiction between annotation time and annotation accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If automated machine learning models are used, then annotation speed is improved, but annotation accuracy deteriorates

Engineering Contradiction:

Improveannotation speedVSAvoidannotation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent merges automated machine learning annotation with expert human review into a hybrid system. The automated model handles the initial spline generation to provide speed, while the expert reviewer performs final adjustments to ensure accuracy. This combination merges the advantages of both automated and manual approaches, resolving the contradiction between annotation speed and annotation accuracy.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system performs preliminary automated annotation using a machine learning model to generate an initial spline approximation before the expert reviewer examines it. This preliminary action provides a head start, reducing the time experts need to spend on each annotation while maintaining accuracy through their final review and adjustment of the pre-generated spline.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11205287B2Annotation of digital images for machine learning

Publication Date: 2021.12.21 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Computer-implemented methods and apparatus are provided for annotating digital images of line plots with ground truth labels. For each digital image, such a method includes supplying image data defining the image of a line plot to a machine-learning model trained to generate a set of control points defining a spline corresponding to the line plot. The method further comprises displaying the spline, and the set of control points, superimposed on the image in a graphical user interface and, in response to user manipulation via the graphical user interface of one or more control points, dynamically adjusting the displayed spline in accordance with manipulated control points whereby the displayed spline can be adjusted for conformity with the line plot. The set of control points for the adjusted spline is then stored as a ground truth label for the image.