Parallel Character-Matching Processor for Index-Free Full-Text Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional full-text search technologies face challenges such as reliance on inverted indexes, difficulty in real-time processing, high memory usage, and language-specific issues, making them inefficient and difficult to standardize globally.
Innovation Solution
A full-text search processor using semiconductor devices that performs keyword searches without creating indexes, utilizing character storage elements, character detection circuits, and character string detection circuits to achieve parallel processing and efficient search operations across various languages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If inverted indexes are created to speed up full-text search, then search speed is improved, but memory area and storage area are expanded making expansion difficult
Solution Approach 1:
The patent segments the full-text search process into multiple parallel processing stages: character code detection, N-gram sequence matching, and position identification. Each stage processes data independently and simultaneously, eliminating the need for large inverted indexes while maintaining high search speed through parallel computation architecture.
Solution Approach 2:
The patent replaces the traditional mechanical index-based search system with an algorithmic parallel processing system. Instead of physically storing and searching inverted indexes in memory, the system uses sequential circuit logic and parallel data processing to achieve fast search without expanding memory area.
2Speed
If inverted indexes are created in advance to enable full-text search, then retrieval is sped up, but real-time processing of stream information becomes difficult
Solution Approach 1:
The patent implements a dynamic search architecture that can adapt to different input types (batch data and stream data) and search conditions. The parallel processing circuitry can be reconfigured through control signals to handle various N-gram lengths, search patterns, and data formats in real-time without requiring pre-created indexes.
3Quantity of substance
If morphological analysis is used for Japanese text indexing, then the number of indexes is reduced, but full-text search for middle and backward matches becomes difficult
Solution Approach 1:
The patent creates a universal search mechanism that handles multiple search patterns (forward, middle, backward, and new term matching) simultaneously through parallel N-gram sequence comparison. The system processes all possible N-gram positions in parallel, making it versatile for different search patterns while keeping the index structure simple and language-agnostic.
4Adaptability or versatility
If N-gram indexes are used to handle full-text search for all matching types, then search versatility is improved, but the number of indexes becomes very large
Solution Approach 1:
The patent replaces the static N-gram index storage system with a dynamic parallel processing system. Instead of pre-storing all possible N-gram indexes, the system uses sequential logic circuits that generate and compare N-gram sequences on-the-fly during search operations, achieving comprehensive search coverage without expanding the index quantity.
Data Source
AI summary
[Problem to solve]To provide a hardware accelerator processor for full-text searches.[Solution]There is provided a full-text search processor, comprising: character storage elements for assigning and temporarily storing therein, search target text data to be searched through to a first address to an Nth address byte by byte; character detection circuits for receiving coded characters included in the search keyword byte by byte as comparison data, and sequentially detecting storage positions, on the character storage elements, of all of coded characters included in a search keyword; character string detection circuits for sequentially detecting positions, on the character storage elements, of coded characters which match a sequence of all of the coded characters included in the search keyword; and result output circuits for receiving search results of the character string detection circuits and outputting a position of the beginning or a position of the end of the character string that matches the search keyword.