Real-World Object Highlighting via Camera and Projector

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current systems for user interaction with virtual objects in online environments, especially in real-world settings, are limited in providing comprehensive information and require significant user input, often relying on separate devices and cellular connections, with limited input and output capabilities, especially when users are away from laptops or other computing devices.

Innovation Solution

A system that uses a camera and projector to identify objects in real-world settings, projecting information directly onto the objects or displaying it on a mobile device, allowing users to receive detailed data such as product comparisons, allergens, and recommendations through image analysis and user data analysis, with the option to send data to a remote server for further processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If a camera and projector system is used to provide real-world object information, then information accessibility is improved, but device complexity increases

Engineering Contradiction:
Improveinformation accessibilityVSAvoidsystem complexity
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

The patent introduces a camera as an intermediary to capture images of real-world objects and a projector as an intermediary to display information about those objects. These intermediaries bridge the gap between the physical object and the user, allowing information access without direct device interaction. The camera captures visual data, processes it to identify objects, and the projector presents relevant information, creating a mediator system that resolves the contradiction between information accessibility and direct user-device interaction.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Loss of information

If mobile devices are used for object information retrieval, then information capability is improved, but ease of operation deteriorates due to requiring device retrieval

Engineering Contradiction:
Improveinformation capabilityVSAvoidoperation convenience
Core Design Contradiction:
Loss of informationVSEase of operation

Solution Approach 1:

The system enables self-service by allowing the environment itself to provide information. Instead of requiring the user to actively retrieve and operate a mobile device, the projector automatically displays information about objects in the user's field of view. The system serves itself by capturing images, processing them to identify objects, and presenting relevant information without requiring the user to manually initiate the information retrieval process.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The system performs preliminary actions by continuously capturing images and pre-processing them to identify objects before the user even asks for information. The camera is already taking pictures, the system is already analyzing the images and identifying objects in the scene, so when the user looks at an object, the information is ready to be projected immediately, eliminating the need for device retrieval and manual information requests.

Inventive Principle:
Principle #10Preliminary action

3Loss of information

If comprehensive object information is provided, then information completeness is improved, but loss of time increases due to processing requirements

Engineering Contradiction:
Improveinformation completenessVSAvoidprocessing time
Core Design Contradiction:
Loss of informationVSLoss of time

Solution Approach 1:

The system performs preliminary image capture and object identification continuously in the background before the user needs information. Images are being captured, processed, and objects are being identified in advance, so when the user looks at an object, the system already knows what it is and can immediately project relevant information without requiring real-time processing delays.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system projects comprehensive information about objects including multiple attributes, specifications, and details that may not all be immediately necessary. This excessive action of providing more information than strictly needed ensures information completeness is achieved, and the system accepts the processing time cost as worthwhile for providing thorough, comprehensive object data.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11367130B2Method for in-store object highlighting by a real world user interface
Publication Date: 2022.06.21 SONY INTERACTIVE ENTERTAINMENT LLC
  • US11367130B2 patent drawing
  • US11367130B2 patent drawing
  • US11367130B2 patent drawing

AI summary

Systems and methods according to present principles involved highlighting objects such as on a store shelf. Highlighting is displayed by projecting a light directly on the object by means of a light source or may alternatively be indicated by displaying an indicator or image of the object on a display screen. Information about the targeted object may appear in a headmounted display, on a user's mobile device, or may be projected on a surface via the projector. Information provided to a user may include comparison product data, data about potential allergens, and the like. Highlighting can also be employed to suggest items of interest to the user. For example, products may be recommended to a user through analysis of user data and such products illuminated by a projector as a user walks through a store. Other highlighted objects may be those on a user grocery list or the like.