AR Information Extraction Framework with Prediction Engine

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users face difficulties in extracting relevant information from real-world objects, such as menus or posters, due to the lack of digital availability and insufficient context, leading to delayed decision-making and inefficient information retrieval.

Innovation Solution

An end-to-end software framework for information extraction, enrichment, and caching in augmented reality (AR) applications, utilizing a client-server architecture and cross-modal data fusion to retrieve supplementary information from cloud-based datastores, allowing users to interact with real-world data through a camera-enabled interface and perform gesture-based queries.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If users query multiple cloud-based datastores to retrieve supplementary information, then the completeness of information is improved, but the time required for information retrieval increases

Engineering Contradiction:

Improvecompleteness of informationVSAvoidtime required for information retrieval

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The system performs preliminary actions by predicting which data will be needed before the user actually queries for it. The prediction engine analyzes user behavior patterns and pre-retrieves supplementary information from cloud-based datastores, storing it in local cache. This eliminates the need to wait for multiple sequential queries when the user needs comprehensive information about real-world objects.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The information retrieval system is segmented into multiple independent components: a prediction engine that forecasts data needs, a cache manager that handles local storage, and cloud-based datastores for comprehensive data. This segmentation allows parallel operation of multiple data retrieval processes without mutual interference, reducing overall retrieval time while maintaining information completeness.

Inventive Principle:

Principle #1Segmentation

2Loss of information

If the system retrieves and processes large amounts of data from cloud-based datastores, then the quality of information enrichment is improved, but the system complexity increases

Engineering Contradiction:

Improvequality of information enrichmentVSAvoidsystem complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent introduces a prediction engine as an intermediary component that sits between the user interface and the cloud-based datastores. This intermediary analyzes user intent and behavior patterns to predict which data will be needed, filtering and prioritizing data retrieval requests. This reduces the complexity of directly managing multiple cloud datastore queries while maintaining high information enrichment quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements self-service through automated prediction and cache management. The prediction engine autonomously determines what data to retrieve and when, without requiring manual user requests for each piece of supplementary information. The cache manager automatically manages local storage, deciding what to cache and when to evict data, reducing the operational complexity of the system.

Inventive Principle:

Principle #25Self-service

3Loss of information

If the system provides comprehensive supplementary information, then the user decision-making capability is improved, but the data processing time increases

Engineering Contradiction:

Improvecomprehensive information availabilityVSAvoiddata processing speed

Core Design Contradiction:

Loss of informationVSProductivity

Solution Approach 1:

The system performs data retrieval and processing in advance by predicting user information needs. The prediction engine forecasts which supplementary information will be required based on user behavior patterns, and the system proactively retrieves and processes this data before the user actually needs it. This preliminary action ensures comprehensive information is ready when needed, eliminating processing delays during user interaction.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts its data processing strategy based on real-time user behavior analysis. The prediction engine continuously learns from user interactions and adapts its predictions, retrieving only the most relevant supplementary information dynamically. This dynamic approach ensures comprehensive information availability while optimizing processing speed by focusing computational resources on high-priority data.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11687581B2Information extraction, enrichment, and caching framework for augmented reality applications

Publication Date: 2023.06.27 OHIO STATE INNOVATION FOUND

AI summary

An end-to-end software framework is provided for information extraction, enrichment, and caching in applications that involve exploring real-world data through a camera-based viewport in an augmented reality (AR) setting. It allows users to gather supplementary information through cross-modal data fusion against cloud-based datastores. Additional context is provided by retrieving data units from remote datastores that are relevant to the queried item. Given that such datastores often do not fit in main memory, a client-server architecture is used. Users interact with real-world objects via a camera-enabled client interface, and queries are executed using gesture-based interactions as supplementary information is retrieved from a back-end server (if necessary) and overlaid in the client's live camera-view.