Database Indexing via Word Segmentation for Mixed Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in querying mixed data types, as B+ tree indices are insufficient for unstructured text, leading to time-intensive two-step searches that hinder performance, especially in large-scale OLTP services.
Innovation Solution
A database system that performs word segmentation on unstructured text to generate semantically meaningful word segments, combining these with structured data to create index entries, allowing for efficient lookup and query resolution through an index table.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If B+ tree indices are used for structured data and string matching operations are performed on unstructured text, then the database can handle mixed data types, but the lookup process becomes time-intensive and inefficient for large numbers of records
Solution Approach 1:
The patent segments unstructured text into individual words and creates separate index entries for each word. This segmentation allows the database to index and search unstructured text efficiently, avoiding the need for iterative string matching operations on large datasets while maintaining the ability to handle mixed data types.
Solution Approach 2:
The patent performs preliminary word segmentation and indexing of unstructured text during data insertion or batch processing, before actual queries are executed. This preliminary action transforms unstructured text into a searchable format, enabling fast lookup operations without time-intensive iterative searching when queries are performed.
2Measurement precision
If iterative string matching operations are performed on unstructured text to determine query response, then complete query accuracy is achieved, but the process becomes inefficient when the number of records is large
Solution Approach 1:
By segmenting unstructured text into individual words and creating index entries for each segment, the patent enables precise matching of query terms against indexed words. This segmentation approach maintains query accuracy by ensuring exact word matches while dramatically improving efficiency by eliminating iterative string matching operations.
Solution Approach 2:
The patent creates index entries that are copies or representations of the original unstructured text data in a structured, searchable format. These index entries contain the essential information needed for accurate query matching, allowing the system to perform fast lookups on index copies rather than repeatedly accessing and searching the original large-volume unstructured text.
3Speed
If the database maintains comprehensive index entries for both structured data and unstructured text, then query resolution is accelerated, but the index table size and storage requirements increase
Solution Approach 1:
The patent segments unstructured text into individual words and creates index entries only for these segmented words rather than storing complete text strings. This segmentation reduces the storage requirement for index entries while maintaining query resolution speed, as the indexed words are sufficient for accurate matching without requiring the full unstructured text to be stored in the index.
Solution Approach 2:
The patent extracts only the essential elements (individual words) from unstructured text for indexing, rather than storing or processing the complete text. This extraction approach reduces index table size by eliminating redundant information while preserving the key elements needed for fast and accurate query resolution.
Data Source
AI summary
Embodiments described herein provide a system for facilitating efficient indexing in a database. The system can maintain a data table in the database. The data table includes a first column for structured data and a second column for unstructured text. A row of the data table can include a first piece of data in the first column and a second piece of data in the second column. During operation, the system performs word segmentation on the second piece of data to generate a first set of semantically meaningful word segments. The system then generates a set of index entries that corresponds to the row. A respective index entry includes an index generated from the first piece of data and a corresponding word segment of the first set of semantically meaningful word segments.