Blindness Assist Glasses Using Camera-Based Audio Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users with nearsightedness, farsightedness, color blindness, or blindness face challenges in using portable eyewear devices like smart glasses due to their inability to effectively interpret visual information from see-through displays.
Innovation Solution
The eyewear device employs a camera-based compensation system using a region-based convolutional neural network (RCNN) to identify objects in images, convert them to text, and then generate audio descriptions, enabling users with partial or complete blindness to perceive their environment through audio cues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If see-through displays are used in eyewear devices, then visual information is provided to users, but users with visual impairments cannot effectively interpret the visual information
Solution Approach 1:
The patent introduces an intermediary system that converts visual information from the see-through display into audio format. The eyewear device captures images through integrated cameras, processes them to identify objects and text, and then converts this information into spoken audio descriptions that blind or visually impaired users can understand and interpret.
Solution Approach 2:
The patent replaces the visual display mechanism with an auditory output mechanism for impaired users. Instead of relying on optical see-through displays that require visual processing, the system substitutes with camera-based image capture, computational processing, and audio synthesis to deliver information through a different sensory modality.
2Ease of operation
If camera-based compensation system is implemented, then auditory feedback is provided to visually impaired users, but device complexity increases
Solution Approach 1:
The eyewear device integrates multiple functions into a single platform: it serves as both a conventional smart glass with see-through display for sighted users and as an accessibility device with camera-based compensation for visually impaired users. The same hardware components (cameras, processors, speakers) support both visual and auditory information delivery modes.
Solution Approach 2:
The patent combines the camera system, image processing unit, text recognition engine, and audio synthesis components into an integrated eyewear device. Rather than adding separate standalone devices, the system merges multiple functional elements into a unified wearable platform that provides both visual and auditory compensation capabilities.
3Loss of information
If RCNN and text-to-speech conversion are used, then object identification and audio description are achieved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary processing by capturing images continuously or at anticipated moments before user inquiry. The RCNN model pre-identifies objects and text in the scene, and the text-to-speech system pre-generates audio descriptions, so that when a visually impaired user requests information, the processing is already complete or near-complete, reducing perceived latency.
Data Source
AI summary
An eyewear device with camera-based compensation that improves the user experience for user's having partial blindness or complete blindness. The camera-based compensation determines objects, converts determined objects to text, and then converts the text to audio that is indicative of the objects and that is perceptible to the eyewear user. The camera-based compensation may use a region-based convolutional neural network (RCNN) to generate a feature map including text that is indicative of objects in images captured by a camera. Relevant text of the feature map is then processed through a text to speech algorithm featuring a natural language processor to generate audio indicative of the objects in the processed images.