Index Segment Construction for Parallel Search Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search technologies are hindered by the need to merge new data with small index segments and then with large index segments before providing search services, which affects search speed.
Innovation Solution
Constructing primary index segments at preset time intervals, merging them into a primary index segment set, copying and merging these into a large index segment set, and sending both to search devices to provide simultaneous index construction and search services, allowing parallel execution of indexing and searching processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If new data is merged with small index segment and then with large index segment before providing search service, then data consistency is ensured, but search speed is reduced
Solution Approach 1:
The patent divides the index into multiple independent segments (first index segment, second index segment, third index segment) that can be constructed and maintained separately. Each segment corresponds to different time periods or data batches, allowing parallel processing and independent optimization of each segment without blocking search operations on other segments.
Solution Approach 2:
The patent pre-constructs multiple index segments in advance before they are needed for searching. Index segments are built during off-peak periods or in parallel threads, so that when search operations need to access the data, the index segments are already ready and can be used immediately without waiting for construction to complete.
2Measurement precision
If index segments are merged sequentially before providing search service, then accurate search results are obtained, but time delay increases
Solution Approach 1:
The patent implements a dynamic index management system where multiple index segments can be accessed simultaneously based on their construction status and time validity. The system dynamically selects which index segments to use for search operations, allowing flexible switching between fully constructed segments and partially constructed segments with appropriate time-based filtering to maintain accuracy while reducing delay.
Solution Approach 2:
The patent ensures continuous search capability by maintaining multiple index segments in different construction states simultaneously. While one index segment is being constructed, other segments remain available for search operations, ensuring that search functionality continues without interruption or significant delay.
3Speed
If parallel construction of indexes and searching is implemented, then search speed is improved, but system complexity increases
Solution Approach 1:
The patent divides the indexing and searching processes into separate, independent segments that can run in parallel. Index construction operates on divided data segments while search operations query different segments simultaneously, reducing resource contention and enabling parallel execution without requiring complex coordination mechanisms.
Solution Approach 2:
The patent introduces an intermediary layer (the segmented index structure itself) that decouples the indexing process from the searching process. This intermediary structure allows independent optimization of both processes and provides a clear interface for parallel operation without direct interference between indexing threads and search threads.
Data Source
AI summary
The present disclosure discloses an index constructing method, search method, device and system. The index constructing method includes: constructing data obtained in each preset time period into a primary index segment, respectively, and merging the constructed primary index segments into a primary index segment set after each preset time period is over; copying and merging each primary index segment in the primary index segment set into a corresponding large index segment, to obtain a large index segment set; and sending the primary index segments in the primary index segment set or the large index segments in the large index segment set to a search device to provide a search service. The present disclosure constructs the primary index segments on the basis of time, merges the constructed primary index segments into the large index segment, and provides the search service simultaneously with the primary index segments or the large index segments, thus, improving search speed.