Hand-Recognized Image Processing for Immersive AR Selection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In image processing systems, especially those using wearable displays, superimposed advertising banners and control panels can be distracting and difficult to navigate, disrupting the user's immersive experience and making selection operations cumbersome.

Innovation Solution

An image processing apparatus that recognizes a user's hand state and superimposes item images on the hand, allowing users to select items by hand motion, enabling intuitive control of displayed options without interfering with the primary content.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If advertising banners and control panels are superimposed on content images, then information delivery and user interaction options are improved, but user comfort and ease of operation deteriorate due to visual obstruction and difficulty in making selections

Engineering Contradiction:
Improveinformation deliveryVSAvoiduser comfort
Core Design Contradiction:
Loss of informationVSEase of operation

Solution Approach 1:

The patent segments the superimposed content into distinct functional layers: advertising banners are separated from control panels, and each is assigned to different spatial zones or depth planes. This segmentation allows users to interact with control elements without the visual clutter of advertisements, improving ease of operation while maintaining information delivery.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces depth dimensioning to the display system, where superimposed images are positioned at different depth levels relative to the primary content. Control panels and advertisements are placed in elevated depth planes that do not obstruct the main visual field, allowing users to access interactive elements through depth-based selection rather than spatial occlusion.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If multiple superimposed images are displayed on content, then functionality and information provision are improved, but selection operation difficulty increases

Engineering Contradiction:
ImprovefunctionalityVSAvoidselection operation
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent introduces a depth-based intermediary system that mediates between multiple superimposed images and the user's selection action. By assigning different depth values to various superimposed elements, the system creates a natural selection mechanism where users can selectively focus on and interact with elements at specific depth levels, simplifying the selection process among multiple functional options.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Adaptability or versatility

If superimposed images are used to provide additional information and controls, then content functionality is improved, but immersion in the primary content is reduced

Engineering Contradiction:
Improvecontent functionalityVSAvoidimmersion disruption
Core Design Contradiction:
Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent utilizes the depth dimension to position superimposed functional elements outside the primary visual focus zone. By placing controls and advertisements in elevated depth planes, the system maintains the immersive quality of the primary content while providing accessible functional overlays that do not disrupt the user's immersion in the main experience.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS11380021B2Image processing apparatus, content processing system, and image processing method
Publication Date: 2022.07.05 SONY INTERACTIVE ENTERTAINMENT LLC
  • US11380021B2 patent drawing
  • US11380021B2 patent drawing
  • US11380021B2 patent drawing

AI summary

Disclosed herein is an image processing apparatus including a hand recognizing section configured to recognize a state of a hand of a user, an item image superimposing section configured to superimpose images of items as selection targets attached to the fingers of the hand that are assigned to the items on either an image of the hand being displayed or an image representing the hand being displayed, and a selecting operation detecting section configured to detect that one of the items is selected on the basis of a hand motion performed on the images of the items before performing processing corresponding to the detected selection.