Image-Guided AAC Vocabulary Retrieval for Contextual Word Suggestions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Augmentative and Alternative Communication (AAC) tools are inadequate for individuals with developmental disabilities such as autism spectrum disorder (ASD) and lexical and semantic processing impairments, as they struggle to provide relevant vocabulary words, especially non-core words, and often require tedious customization, limiting their effectiveness in real-time communication scenarios.
Innovation Solution
A system that generates candidate vocabulary words by comparing an input image to reference images, analyzing associated stories for frequent words, and using a vocabulary database to provide relevant words, including both tangible and intangible concepts, emotions, and sensations, through a process involving machine learning and semantic analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional NLP-based word prediction tools are used, then spelling and grammar support is improved, but they are ineffective for individuals with developmental disabilities who use symbol-based systems
Solution Approach 1:
Instead of using traditional NLP approaches that analyze text patterns from large corpora, the patent inverts the approach by using image-based contextual cues and story analysis to generate vocabulary recommendations. The system processes images and associated narratives to understand context, then retrieves relevant words from a vocabulary database, effectively reversing the conventional text-first NLP paradigm to accommodate symbol-based AAC users.
Solution Approach 2:
The patent changes the fundamental parameters of vocabulary generation by shifting from statistical text analysis to multi-modal processing combining image recognition, story analysis, and database retrieval. This parameter change enables the system to provide contextually relevant vocabulary for individuals with developmental disabilities who cannot rely on traditional keyboard-based prediction tools.
2Ease of operation
If physical word boards are used, then simple communication is enabled, but they are limited by physical size and do not include desired words
Solution Approach 1:
The patent creates a digital system that performs multiple functions: it captures images, analyzes contextual stories, retrieves vocabulary from databases, and presents personalized word recommendations. This multi-functional approach replaces the static, limited physical word board with a dynamic system that can adapt to any vocabulary need while maintaining ease of use through automated processing.
Solution Approach 2:
The system transitions from the static nature of physical word boards to a dynamic digital platform that continuously adapts vocabulary recommendations based on real-time image input and story analysis. The vocabulary list is not fixed but dynamically generated and updated based on contextual relevance, allowing unlimited vocabulary coverage without compromising operational simplicity.
3Adaptability or versatility
If mobile word board applications are used, then larger vocabulary variety is provided, but they require substantial customization for non-core words which is tedious and impractical
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate personalized vocabulary recommendations without requiring manual customization by users or caregivers. The automated pipeline processes images and stories to identify contextually relevant words, eliminating the tedious setup process while maintaining high vocabulary variety and adaptability to individual needs.
Solution Approach 2:
The system performs preliminary action by pre-processing images and stories to extract contextual information before vocabulary retrieval. This preliminary analysis enables the system to proactively identify relevant vocabulary without waiting for user input or manual configuration, thereby reducing setup complexity while providing extensive vocabulary variety tailored to each user's context.
4Productivity
If it is difficult to know in advance what non-core words will be desired, then customization can be avoided, but relevant vocabulary cannot be provided in real-time scenarios
Solution Approach 1:
The patent implements feedback by using image and story analysis to continuously inform vocabulary recommendations in real-time. The system processes contextual information from user-provided images and associated narratives, then feeds this information back into the vocabulary retrieval process to generate relevant word suggestions. This feedback loop ensures contextual relevance is maintained without requiring advance knowledge of desired words.
Solution Approach 2:
The system performs preliminary analysis of images and stories to pre-identify contextual themes and concepts before vocabulary retrieval. This preliminary action enables real-time provision of relevant vocabulary by having the contextual understanding ready in advance, eliminating the need for both manual customization and advance planning while maintaining high relevance.
Data Source
AI summary
System and method for generating candidate vocabulary words and presenting them to a user. The system compares an input image to reference images, based on text descriptions of the images, and determines relevant images. Stories associated with the relevant images are then analyzed to determine frequent words in the stories. In one embodiment, the frequent words are presented to the user as candidate words. In another embodiment, a vocabulary database is first accessed to determine other words that are associated with the frequent words in the stories. The frequent words in the stories and the other words from the vocabulary database are then provided to the user as candidate vocabulary words. The user can select desired candidate words through their computing device. In some embodiments, the stories are clustered together before word frequency is analyzed.