Skip Table Data Retrieval Device for High-Speed Similarity Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data retrieval methods require extensive similarity calculations, leading to slow retrieval speeds, even when using self-similarity tables, due to the need for sequential subtraction and threshold determination for each data point.
Innovation Solution
A data retrieval device employing a first skip correspondence table that records skip destination data for each data point, allowing for the selection of necessary similarity calculations based on predetermined similarity ranges, thereby reducing unnecessary calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If similarity calculation is performed for all data in the retrieval target data series, then retrieval completeness is ensured, but retrieval time becomes excessively long
Solution Approach 1:
The patent pre-calculates and stores self-similarity values for all data in the retrieval target data series before actual retrieval operations. This preliminary action enables fast retrieval by avoiding redundant similarity calculations during the retrieval process, thus reducing retrieval time while maintaining completeness through the pre-computed self-similarity information
Solution Approach 2:
The patent implements a skip table mechanism that allows the retrieval system to skip data points that are guaranteed to not meet the similarity threshold. By using the pre-computed self-similarity values and threshold logic, the system can rapidly determine which data points can be skipped without performing full similarity calculations, thereby significantly reducing retrieval time while ensuring all potentially relevant data are still examined
2Productivity
If self similarity table is used to reduce calculation scope, then retrieval speed is improved, but the process becomes more complex with sequential subtraction and threshold determination
Solution Approach 1:
The patent introduces a skip table as an intermediary data structure that pre-determines which data points should be skipped based on self-similarity values and threshold comparisons. This intermediary structure simplifies the retrieval process by providing direct skip instructions rather than requiring complex sequential subtraction and threshold determination logic for each data point, thus reducing process complexity while maintaining high retrieval speed
Data Source
AI summary
The data retrieval device includes a first skip correspondence table which corresponds to each piece of data in a retrieval target data series, and, for each possible similarity range which is taken by a similarity between corresponding data and retrieval data, records skip destination data information for specifying the data which appears first after the corresponding data among pieces of data in which similarities with the retrieval data have the possibility to have a predetermined relationship in comparison with a predetermined threshold. The data retrieval device also includes a control unit which, when retrieving data in which a similarity with the retrieval data is smaller than or equal to the threshold from among the retrieval target data series, selects data in the retrieval target data series for which calculation of a similarity with the retrieval data is necessary, using the first skip correspondence table.