3D Point Cloud Sound Visualization for Real-Time Virtual Spaces

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques struggle to visualize sound information from a real space in a virtual space in real time, particularly for hard-of-hearing individuals or when sound reproduction devices malfunction, and existing speech-to-text conversion methods fail to visualize environmental sounds.

Innovation Solution

An information processing device that integrates a communication unit, point group data management, sound information analysis, and attribute change units to analyze ambient sound and modify 3D point group data attributes based on sound type, constructing a virtual space that reflects real-space sound.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If sound information is acoustically reproduced on a virtual space side, then sound information can be reflected in the virtual space, but it becomes difficult to recognize sound information when users are hard of hearing or reproduction devices malfunction

Engineering Contradiction:

Improvesound information recognition reliabilityVSAvoidaccessibility for hard of hearing users

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent introduces text information as an intermediary medium to convey sound information. Instead of directly reproducing sound, the system converts sound information into text that can be displayed to users, providing an alternative channel for information delivery that works regardless of hearing ability or sound reproduction device functionality.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the parameter of sound information representation from acoustic form to textual form. By converting the medium through which sound information is delivered (from sound waves to text characters), the system makes the information accessible to users with hearing impairments while maintaining the same informational content.

Inventive Principle:

Principle #35Parameter changes

2Ease of manufacture

If sound effects and visualization data are prepared in advance and loaded into memory, then visualization can be achieved, but real-time sound information from real space cannot be visualized

Engineering Contradiction:

Improvevisualization implementation easeVSAvoidreal-time sound visualization capability

Core Design Contradiction:

Ease of manufactureVSProductivity

Solution Approach 1:

The system performs preliminary preparation of text display templates and visualization frameworks in advance, but leaves the actual sound information content to be filled in real-time. This allows the visualization infrastructure to be ready beforehand while still accommodating real-time sound data from the real space environment.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system transitions from static pre-loaded sound effect visualizations to dynamic real-time sound information visualization. By making the visualization content updateable and responsive to real-time acoustic sensor data, the system achieves both the ease of having pre-prepared visualization frameworks and the productivity of real-time sound representation.

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If speech-to-text conversion is used to visualize sound, then communication can be improved for hard of hearing people, but environmental sounds cannot be visualized

Engineering Contradiction:

Improvecommunication accessibilityVSAvoidsound type coverage

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system extends the functionality of sound visualization beyond speech recognition to include all types of environmental sounds. By making the sound information acquisition and visualization system universal, it can handle both human speech and non-speech sounds (nature sounds, mechanical sounds, etc.) through the same text-based visualization mechanism.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system replaces the speech-specific acoustic processing mechanism with a more general sound information acquisition and text conversion mechanism. This substitution allows the system to handle diverse sound types not limited to human speech, thereby improving versatility while maintaining accessibility benefits.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250349078A1Information processing apparatus, information processing method, and program

Publication Date: 2025.11.13 NT T INC

AI summary

An information processing device includes a communication unit, a point group data management unit, a sound information analysis unit, an attribute change unit, and a virtual space construction unit. The communication unit communicates with a real world information acquisition device including an environmental sensor and an acoustic sensor. The point group data management unit stores 3D point group data of a real space acquired from the real world information acquisition device. The sound information analysis unit analyzes sound information data to obtain a type of the sound information data. The attribute change unit changes attribute information of the 3D point group data stored in the point group data management unit on the basis of the type of the sound information data. The virtual space construction unit constructs a virtual space video on the basis of the 3D point group data in which the attribute information is changed.