AR Image Recognition via Field Segmentation and Feedback

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In many augmented reality (AR) applications, the composition of the input image affects the effective operation of AR objects, making reliable information provision challenging due to dependencies on viewing angle, zoom position, and camera orientation.

Innovation Solution

An information processing system that acquires images, compares them to templates with defined fields, and outputs recognition results for each field, allowing users to adjust the image composition for improved recognition and information provision.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If image recognition is performed based on input image composition, then AR objects can be displayed at positions associated with real objects, but the effective operation of AR applications is affected by viewing angle, zoom position, and camera orientation

Engineering Contradiction:
Improvereliability of information provisionVSAvoiddependence on image composition
Core Design Contradiction:
ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent segments the image recognition process into multiple independent field recognition operations. Instead of recognizing the entire image at once, the system divides the image into multiple fields (e.g., first field, second field, third field) and performs recognition on each field separately. This segmentation allows the system to maintain reliable information provision even when the overall image composition varies due to different viewing angles, zoom positions, or camera orientations, as long as individual fields are properly captured.

Inventive Principle:
Principle #1Segmentation

2Loss of information

If AR objects are selected and laid out based on image recognition, then additional information can be presented to the user, but the composition of the input image affects the effective operations of the AR application

Engineering Contradiction:
Improveinformation provision to userVSAvoiduser adjustment of image composition
Core Design Contradiction:
Loss of informationVSEase of operation

Solution Approach 1:

The patent implements a feedback mechanism where the system provides recognition results for each field back to the user. The display unit shows whether recognition of each field was successful, allowing the user to understand what information has been captured and what may need adjustment. This feedback loop enables users to make informed adjustments to the image composition without needing to understand the complex underlying recognition processes, thereby maintaining ease of operation while ensuring complete information provision.

Inventive Principle:
Principle #23Feedback

3Measurement precision

If template matching is used for field recognition, then recognition results can be obtained for each field, but the system must compare the image to multiple templates

Engineering Contradiction:
Improverecognition accuracyVSAvoidnumber of templates and comparison operations
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the template matching process into field-level operations. Instead of comparing the entire image against a single complex template, the system segments both the image and templates into corresponding fields. The comparison unit then performs matching operations on individual fields against corresponding template fields, reducing the complexity of each comparison operation while maintaining overall recognition accuracy through the cumulative effect of multiple field-level matches.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS9355496B2Image processing apparatus, image processing method, and medium to display augmented reality objects
Publication Date: 2016.05.31 SONY GROUP CORP
  • US9355496B2 patent drawing
  • US9355496B2 patent drawing
  • US9355496B2 patent drawing

AI summary

An information processing system acquires an image captured by an image pickup unit and one or more templates where each template includes one or more fields. The acquired image is then compared to the templates and a result based on the comparison is generated such that the result indicates whether recognition of the fields of the templates was successful.