Synchronized Sound and Vibration Control for Immersive Haptics

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional information processing systems, such as game apparatus, can only impart monotonous vibrations to users, lacking the ability to create unconventional vibration experiences.

Innovation Solution

An information processing system that includes sound source position setting, sound control, and vibration control means to generate synchronized sound and vibration signals, allowing the user to perceive vibrations as originating from a localized sound source, which can shift and change intensity, expanding the perceived vibration source area.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional vibration methods are used, then the device structure is simple, but the vibration experience is monotonous and lacks variety

Engineering Contradiction:
Improvevibration experience varietyVSAvoidcontrol system complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent combines sound output and vibration output into a unified control system. The control section simultaneously manages both sound signals from speakers and vibration signals from vibrators, coordinating them to produce integrated sensory experiences. This merging allows the system to create complex vibration patterns without proportionally increasing hardware complexity, as the control logic is integrated rather than separate.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent implements dynamic vibration control where the vibration characteristics (frequency, intensity, pattern) change in real-time based on game events, sound localization position, and user interaction. The control section continuously adjusts vibration parameters rather than using fixed patterns, enabling diverse vibration experiences from a single vibrator through dynamic parameter modification.

Inventive Principle:
Principle #15Dynamics

2Area of stationary object

If vibration sources are limited to physical vibrator positions, then the device structure is simple, but the perceived vibration source area is restricted

Engineering Contradiction:
Improveperceived vibration source areaVSAvoidvibration generation system complexity
Core Design Contradiction:
Area of stationary objectVSDevice complexity

Solution Approach 1:

The patent uses sound as an intermediary to extend the perceived vibration source area. By coordinating sound localization (through speaker output and head-related transfer functions) with vibration output, the system creates the perception that vibrations originate from spatial locations beyond the physical vibrator positions. The sound field acts as a mediator that maps virtual vibration sources to actual vibrator activations.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the need for multiple physical vibrators distributed throughout the device with a single or few vibrators controlled through acoustic-field-based positioning. Instead of mechanically distributing vibration sources across the device structure, the system uses sound field processing and signal coordination to create virtual vibration sources at arbitrary positions, substituting mechanical complexity with acoustic processing.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If vibrations are generated without sound coordination, then the system is simpler, but the immersive experience is reduced

Engineering Contradiction:
Improvesensory integration capabilityVSAvoidsynchronized control complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements feedback loops where the control section continuously monitors sound localization position and adjusts vibration characteristics accordingly. The system processes the relationship between sound source position, head orientation, and expected vibration perception, then feeds this information back to modulate vibrator output in real-time. This feedback mechanism coordinates auditory and tactile senses to create unified immersive experiences.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The control section serves multiple functions simultaneously: it processes sound output signals, calculates sound localization positions using head-related transfer functions, determines appropriate vibration patterns, and coordinates both output channels. This multi-functionality is achieved through a unified control architecture that handles both audio and haptic processing, reducing overall system complexity despite the sophisticated coordination required.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentEP2949371B1Information processing system, information processing apparatus, information processing program, and information processing method
Publication Date: 2020.11.11 NINTENDO CO LTD
  • EP2949371B1 patent drawingFigure 1
  • EP2949371B1 patent drawingFigure 2
  • EP2949371B1 patent drawingFigure 3

AI summary

Sound source position setting means sets a sound source localization position indicating a position where a sound source is localized. Sound control means generates a sound signal for localizing the sound source at the set sound source localization position, and outputs the generated sound signal. Vibration control means generates a vibration signal for, in conjunction with the sound source localization position, vibrating a vibration section for imparting a vibration to a user, and outputs the generated vibration signal.