AR Data Collection Feedback for Computer Vision

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current techniques lack a systematic process for collecting high-quality and diverse training data for machine learning in computer vision tasks, leading to inefficient model performance due to uneven dataset quality and coverage, requiring costly re-trips to capture missing views and conditions.

Innovation Solution

A system using augmented reality (AR) provides interactive feedback for data collection, measuring image diversity based on distance, angle, lighting, and occlusion, and generates visual instructions for capturing additional images, ensuring comprehensive data coverage and error analysis through cross-validation, thereby improving dataset quality and model training.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If technicians manually collect images without a systematic process, then data collection is flexible and adaptable, but the dataset quality and coverage are uneven, requiring costly re-trips

Engineering Contradiction:

Improveflexibility in data collectionVSAvoiddataset quality and coverage

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system implements feedback by analyzing collected images for diversity and quality metrics, then generating guidance instructions to capture additional images that fill coverage gaps. This closed-loop feedback mechanism ensures systematic improvement of dataset quality without sacrificing collection flexibility.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary analysis of collected images to identify coverage gaps before finalizing the dataset. By proactively determining which views and conditions are missing, the system enables targeted data collection that ensures comprehensive coverage while maintaining operational flexibility.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If the dataset size is increased without ensuring diversity, then more data is available, but model performance does not improve due to similar examples

Engineering Contradiction:

Improvedataset sizeVSAvoidmodel performance

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system changes the parameter of data selection by prioritizing diverse examples over quantity. It analyzes image characteristics (view angles, lighting conditions, occlusion levels) and selectively collects images that expand the diversity of the dataset, ensuring that each added image contributes to improving model performance rather than simply increasing dataset size.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If images are collected without systematic quality control, then data collection is rapid, but the dataset contains excessive blurry or dark images that are infeasible for training

Engineering Contradiction:

Improvedata collection speedVSAvoidimage quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system continuously monitors image quality during collection by analyzing focus quality metrics and lighting conditions. When images are collected, the system provides immediate feedback about quality issues and generates guidance to capture replacement images, ensuring high quality standards are maintained without significantly reducing collection speed.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs self-service quality control by automatically analyzing collected images for blurriness and lighting conditions. This automated self-assessment eliminates the need for manual quality checking while ensuring that only suitable images are included in the final dataset, maintaining both speed and precision.

Inventive Principle:

Principle #25Self-service

4Device complexity

If no systematic process is used for data collection, then the collection process is simple, but it is impossible to quantify the quality of the collected dataset

Engineering Contradiction:

Improvecollection process complexityVSAvoiddataset quality quantification

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system replaces manual, subjective quality assessment with automated computational analysis. It uses image processing algorithms to objectively measure diversity metrics, focus quality, and lighting conditions, enabling precise quantification of dataset quality without adding significant complexity to the collection process.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11917289B2System and method for interactive feedback in data collection for machine learning in computer vision tasks using augmented reality

Publication Date: 2024.02.27 GENESEE VALLEY INNOVATIONS LLC

AI summary

A system is provided which obtains images of a physical object captured by an AR recording device in a 3D scene. The system measures a level of diversity of the obtained images, for a respective image, based on at least: a distance and angle; a lighting condition; and a percentage of occlusion. The system generates, based on the level of diversity, a first visualization of additional images to be captured by projecting, on a display of the recording device, first instructions for capturing the additional images using the AR recording device. The system trains a model based on the collected data. The system performs an error analysis on the collected data to estimate an error rate for each image of the collected data. The system generates, based on the error analysis, a second visualization of further images to be captured. The model is further trained based on the collected data.