Token-Based Dynamic Data Indexing for Ambiguous Query Resolution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing business intelligence (BI) solutions are inefficient in processing large volumes of data, leading to a complex and time-consuming process for business users to obtain meaningful information, and internet search engines lack the ability to perform structured data searches and correlations.

Innovation Solution

Implementing semantic analysis to resolve ambiguous user input data by using a finite state machine and token index to identify tokens and join paths in a database, allowing for automatic generation of candidate results and selection of the appropriate join path based on user input.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional BI solutions are used to process large volumes of data, then data processing capability is maintained, but the process becomes complex and time-consuming

Engineering Contradiction:

Improvedata retrieval speedVSAvoidtime to obtain meaningful information

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the data processing system into multiple independent components: columnar storage structures for efficient data organization, semantic analysis modules for understanding user intent, and pre-computed result caches for quick retrieval. This segmentation allows each component to operate independently and optimize its function, dramatically reducing the time to transform raw data into meaningful information.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-computing and caching query results, organizing data into columnar storage formats in advance, and pre-processing data to enable faster retrieval. When a user queries the system, the results are already prepared or can be quickly assembled from pre-organized data structures, eliminating the need for time-consuming real-time processing of raw data.

Inventive Principle:

Principle #10Preliminary action

2Ease of operation

If IT specialists manually create BI reports from raw data, then data accuracy is maintained, but the process involves multiple steps and takes several weeks

Engineering Contradiction:

Improveease of data retrievalVSAvoidtime from data request to final report

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent implements self-service capabilities where the system automatically performs semantic analysis on user queries, identifies the required data and relationships, executes the appropriate queries against columnar storage structures, and delivers results without requiring IT specialist intervention. Business users can directly retrieve meaningful information by simply describing their data needs in natural language.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system replaces the manual mechanical process of IT specialists creating reports with automated computational processes. Semantic analysis algorithms automatically interpret user intent, query generation systems automatically construct appropriate data retrieval operations, and automated processing systems execute queries and format results, eliminating the manual steps that previously required weeks of work.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If internet search engines are used for data searching, then simple string matching capability is provided, but the ability to perform structured data searches and correlations is lacking

Engineering Contradiction:

Improvesearch capabilityVSAvoidloss of relationship information in structured data

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent creates a universal search system that combines the simple string matching capability of traditional search engines with the structured data processing capabilities of database systems. The semantic analysis module can interpret natural language queries and translate them into structured data operations, while the columnar storage structures enable efficient processing of both simple and complex queries, providing multi-functional search capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system introduces semantic analysis as an intermediary layer between the user's natural language query and the structured database operations. This intermediary translates user intent into appropriate data retrieval operations, preserves relationship information by understanding the semantic context of queries, and enables correlations across structured data while maintaining the simplicity of natural language interaction.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250173319A1Token Based Dynamic Data Indexing With Integrated Security

Publication Date: 2025.05.29 THOUGHTSPOT INC

AI summary

Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.