Speech Search Using Descriptive Object Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech-based natural language systems require users to remember exact object names for searching, which can be challenging in stressful situations like a vehicle environment where drivers may not recall indicator meanings quickly.
Innovation Solution
A natural language query arrangement for mobile environments that allows users to describe objects using properties like color, location, and behavior, enabling advanced searching in knowledge bases and databases using ASR and NLU to provide information about surrounding and in-car objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users search for objects using exact object names in conventional speech-based systems, then search accuracy is improved, but user burden increases due to the need to remember exact names
Solution Approach 1:
The system changes the search parameters from requiring exact object names to accepting descriptive features such as color, shape, size, and location. This allows users to search using natural language descriptions rather than memorized names, reducing user burden while maintaining search accuracy through multi-feature matching
Solution Approach 2:
The system introduces an intermediary natural language processing layer between the user and the object database. This intermediary translates descriptive user inputs into meaningful search queries, bridging the gap between casual user language and precise object identification without requiring users to know exact object names
2Loss of information
If speech-based search requires exact object names, then information precision is improved, but response time increases in stressful situations
Solution Approach 1:
The system accepts partial information from users in the form of descriptive features rather than requiring complete and precise object names. By processing multiple partial descriptions (color, shape, location) simultaneously, the system achieves accurate object identification faster than waiting for users to recall and articulate exact names
Solution Approach 2:
The system pre-processes and indexes objects by multiple descriptive features rather than just by names. This preliminary organization allows for rapid retrieval based on any combination of features, enabling fast response times even when users can only provide partial descriptions under stress
Data Source
Figure 1A~1C
Figure 2
Figure 3
AI summary
A natural language query arrangement is described for a mobile environment. An automatic speech recognition (ASR) engine can process an unknown speech input from a user to produce corresponding recognition text. A natural language understanding module can extract natural language concept information from the recognition text. A query classifier uses the recognition text and the natural language concept information to assign to the speech input a query intent related to one or more objects in the mobile environment. An environment database contains information descriptive of objects in the mobile environment. A query search engine searches the environment database based on the query intent, the natural language concept information, and the recognition text to determine corresponding search results, which can be to the user.