Metaverse Environment Reader for Accessible Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current metaverse platforms are inaccessible to visually-impaired and hearing-impaired users due to their visual and auditory nature, which creates barriers for navigation and interaction within virtual environments.
Innovation Solution
A metaverse environment reader that performs semantic segmentation and object detection to index objects in a scene, creating an audio, haptic, or braille description of the scene, and conveying this information to users through electrical signals for presentation on a user interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If metaverse platforms use visual and auditory interfaces, then immersion and presence are enhanced, but accessibility to visually-impaired and hearing-impaired users deteriorates
Solution Approach 1:
The system provides multiple output modalities (audio descriptions, haptic feedback, braille display) through a single metaverse platform, enabling it to serve both visually-impaired and non-impaired users simultaneously. The environment reader converts visual scene information into multiple accessible formats while maintaining the core immersive experience.
Solution Approach 2:
The environment reader acts as an intermediary component between the metaverse rendering system and the user interface. It receives visual scene data, processes it through semantic segmentation and object detection, and converts it into accessible audio, haptic, or braille representations that can be consumed by users with visual or hearing impairments.
2Loss of information
If the system performs semantic segmentation and object detection to create detailed scene descriptions, then accessibility information completeness is improved, but processing time and computational complexity increase
Solution Approach 1:
The system divides the metaverse scene into semantically meaningful segments through semantic segmentation, identifying and categorizing different objects and elements. This segmentation allows the environment reader to process and describe scene elements in an organized manner, balancing detail completeness with processing efficiency by focusing on relevant objects.
Solution Approach 2:
The system performs object detection and semantic segmentation at appropriate levels of detail based on user needs and context. Rather than processing every possible scene element with maximum detail, it identifies and describes the most relevant objects and features, providing sufficient accessibility information without unnecessary computational overhead.
Data Source
AI summary
A metaverse environment reader performs semantic segmentation and object detection steps to identify a plurality of objects in a metaverse scene. Next, the reader determines an order of importance of the plurality of objects in the scene based at least on a location and a size of each object. Then, the reader sorts the plurality of objects of the scene based on the determined order of importance. Next, the reader indexes the objects based on the segmenting of the scene and based on the determined order of importance. Then, the reader creates a description of the scene based on the indexing, where the description is an audio, haptic, or braille representation of the scene. Next, the reader generates and conveys one or more electrical signals which include an encoding of the description of the scene to a user device to be presented on a user interface.