Gaming Controller Integrated Depth Camera for 3D Position Tracking

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video game consoles rely on traditional controllers for user interaction, which limits the potential for enhanced gaming experiences and immersive interactions.

Innovation Solution

Integration of a mobile depth camera into gaming controllers to capture image and depth data, allowing for advanced user input processing and control of interactive sessions by analyzing movement and position within a three-dimensional space.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional controllers are used for user interaction, then the controller structure remains simple and reliable, but the user interaction capability and immersion experience are limited

Engineering Contradiction:
Improveuser interaction capabilityVSAvoidcontroller structure
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent combines a camera module with a traditional game controller to create an integrated device. The camera captures images of the user and environment, while the controller provides input mechanisms, merging two functional devices into one unified system that enhances interaction capability without requiring separate devices

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The integrated controller-camera system serves multiple functions: it captures images for analysis, detects user gestures, tracks movement, and provides traditional controller inputs. This multi-functional design allows a single device to perform various interaction tasks that would otherwise require multiple separate devices

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Ease of operation

If a camera is integrated into the controller to capture image and depth data, then the user can control games with natural gestures and movements, but the device complexity and processing requirements increase

Engineering Contradiction:
Improvegesture control capabilityVSAvoidimage processing requirements
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The patent introduces an image processing system that acts as an intermediary between the camera and the game control system. The camera captures images, the processing system analyzes these images to detect gestures and movements, and then translates these detections into controller inputs. This intermediary layer simplifies the overall system architecture by handling the complex image analysis separately from the control logic

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system replaces traditional mechanical controller inputs with optical-based gesture recognition. Instead of requiring physical manipulation of controller components, the system uses camera-based image analysis to detect and interpret user gestures and movements, substituting mechanical interaction with optical sensing and digital processing

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentEP2255269B1Controller with an integrated camera and methods for interfacing with an interactive application
Publication Date: 2022.06.15 SONY INTERACTIVE ENTERTAINMENT LLC
  • EP2255269B1 patent drawingFigure 1
  • EP2255269B1 patent drawingFigure 2A-1~2B
  • EP2255269B1 patent drawingFigure 3A~3B

AI summary

Determining a position of a controller in a three-dimensional space includes an operation to calibrate a computer program to identify a group of recognized fixed points from a plurality of fixed points within the three-dimensional space using a depth camera integrated with the controller. Another operation activates use of the controller and depth camera during interface with the application. During use of the controller, image and depth data within the three-dimensional space is captured with the depth camera of the controller. In another operation, the image and depth data is analyzed to find one or more of the group of recognized fixed points. The previous two operations can be repeated and changes of position of the controller determined based on a change in position of the found ones of the group of recognized fixed points.