Subset Index Creation Using Existing Data Indexes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for creating and updating indexes of computer data are not optimized, leading to significant computational effort and time loss due to the cumbersome process of parsing and storing billions of data, especially in archival storage where data is large and often difficult to access.
Innovation Solution
A computer-implemented method that receives a set of data and its index, and creates or updates a subset index by comparing the existing index with identifiers of the subset data, optimizing the process by using only relevant information and storing residual data to efficiently compute updated indexes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional indexing methods are used to create indexes of large data sets, then complete indexing coverage is achieved, but computational effort and time consumption increase significantly
Solution Approach 1:
The patent divides the large data set into multiple subsets and creates indexes for each subset separately. This segmentation allows parallel processing of different subsets, significantly reducing the total time required while maintaining complete indexing coverage across the entire data set.
Solution Approach 2:
The patent performs preliminary filtering to identify and exclude irrelevant data elements before creating indexes. By pre-processing the data to remove unnecessary components, the indexing process operates only on relevant subsets, reducing computational effort and time while preserving completeness for the actual data of interest.
2Reliability
If traditional indexing methods are used to update indexes, then all data is reprocessed, but computational resources are wasted on unchanged data
Solution Approach 1:
The patent extracts and processes only the changed portions of data during index updates. By identifying and isolating modified data elements, the system updates only the affected index subsets rather than reprocessing entire data sets, maintaining index accuracy while dramatically reducing computational resource consumption.
Solution Approach 2:
The patent implements a dynamic index update mechanism that adapts to changes in the data set. The system continuously monitors for changes and selectively updates only those index portions that correspond to modified data, ensuring reliability while optimizing resource usage through adaptive processing.
3Adaptability or versatility
If complete data sets are indexed, then comprehensive search capability is provided, but storage requirements and processing overhead increase
Solution Approach 1:
The patent segments the index into multiple smaller subset indexes, each covering specific portions of the data. This segmentation reduces the storage requirements for each individual index while maintaining comprehensive search capability through coordinated access to multiple segments.
Solution Approach 2:
The patent creates indexes for subsets of data rather than complete data sets, using partial indexing where appropriate. This approach provides sufficient search capability for common queries while reducing overall data volume and processing overhead by excluding unnecessary data from indexing.
Data Source
Figure 1
AI summary
The invention is directed to a method, computer program product and computer system for creating an index (201 - 204) of a subset of data, comprising: - a first step (S100) of receiving: - a set of data ; - a first index (101 - 104) of said set of data; and - identifiers of a subset of data of said set; and - a second step (S220, S221, S222, S223) of creating a second index (201 - 204) of said subset of data according to the first index (101 - 104) and said identifiers.