Intersection Algorithm Search System Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing real-time search technologies face challenges with high memory consumption, difficulty in extension, and slow search response times due to the inefficiencies in performing intersections between high-frequency and low-frequency words, particularly when using inverted index structures.
Innovation Solution
An intersection algorithm-based search method and system that analyzes search keywords to generate a syntax tree, selects a document set with the shortest inverted block as a minimum candidate set, evaluates the number of inverted and sequential intersections, and chooses the algorithm with fewer operations to generate a search result, thereby optimizing memory usage and search speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If inverted intersection is performed on high-frequency words using bit map storage format, then retrieval efficiency is improved, but memory occupation increases significantly
Solution Approach 1:
The patent segments the inverted index into multiple inverted blocks, where each block corresponds to a specific document range. This segmentation allows the system to load only relevant blocks into memory during retrieval operations, significantly reducing memory occupation while maintaining retrieval efficiency. The segmenting principle is applied by dividing the large inverted index structure into manageable chunks that can be selectively accessed.
Solution Approach 2:
The patent introduces a new dimension of organization by creating inverted blocks with specific start and end document identifiers. This dimensional change from a flat inverted index to a hierarchical block structure enables more efficient memory management, allowing the system to navigate through blocks without loading the entire index into memory.
2Reliability
If inverted intersection is performed to determine all inverted blocks of high-frequency words, then complete search results are obtained, but time consumption increases
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and storing the start and end document identifiers for each inverted block. This preliminary organization allows the retrieval process to directly navigate to relevant blocks without performing exhaustive searches, significantly reducing time consumption while ensuring complete search results are obtained.
Solution Approach 2:
The patent enables skipping of irrelevant inverted blocks during retrieval operations. By using the pre-stored block range information, the system can rapidly skip through blocks that do not contain the search terms, rushing through unnecessary processing steps while ensuring that all relevant blocks are examined for complete results.
3Productivity
If bit map storage format is used for inverted index, then retrieval efficiency is improved, but system extension and flexibility become difficult
Solution Approach 1:
The patent implements a dynamic inverted block structure that can be flexibly created, modified, and extended. Unlike static bit map storage, the block-based structure allows the system to dynamically adjust to different search scenarios, add new blocks as needed, and adapt to varying memory constraints, thereby maintaining both retrieval efficiency and system flexibility.
Solution Approach 2:
The inverted block structure serves multiple functions: it enables efficient retrieval operations, allows flexible system extension, supports various memory configurations, and can adapt to different query types. This multi-functionality replaces the specialized but rigid bit map approach with a versatile structure that maintains performance while enabling system evolution.
Data Source
AI summary
The present disclosure provides an, intersection algorithm-based search method and system, and a storage medium, where the search method includes: analyzing search keywords to generate a syntax tree; flattening the syntax tree and selecting a document set having a shortest inverted block as a minimum candidate set; traversing the minimum candidate set and evaluating a traversing result to separately generate a quantity of times of inverted intersections and a quantity of times of sequential intersections; and selecting an intersection algorithm having the less quantity of times as a target intersection algorithm and generating a search result according to the target intersection algorithm.