Selective Database Re-indexing via Schema Version Mismatch Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database re-indexing processes are time and resource intensive, particularly because they do not discriminate between active and inactive users, leading to wasteful re-indexing of data that may not be searched frequently, and can result in stale data due to synchronization failures or delays.
Innovation Solution
A selective re-indexing technique that uses a schema with version numbers to determine when data in a search database needs updating, only re-indexing documents that are actively sought by users and have stale data, by comparing schema version numbers with target version numbers stored in a lookup table, and triggering re-indexing only when mismatches are found or re-indexing markers are present.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data in the primary database is re-indexed in the search database, then data synchronization is maintained, but processing time and resources are excessively consumed
Solution Approach 1:
The patent segments the re-indexing process by dividing the database into individual documents and further segmenting the re-indexing trigger mechanism. Instead of re-indexing all documents simultaneously, the system identifies and re-indexes only specific documents that require updates based on change notifications from the primary database, thereby reducing overall processing time while maintaining synchronization reliability.
Solution Approach 2:
The patent applies partial action by re-indexing only the necessary portion of the database rather than the entire dataset. The system determines which specific documents need re-indexing based on change notifications and re-indexes only those documents, avoiding the excessive processing that would result from re-indexing all documents regardless of whether they changed.
2Productivity
If the indexing service is used to copy data from primary database to search database, then search query processing burden is reduced, but the system becomes vulnerable to indexing service failures and outages
Solution Approach 1:
The patent implements a feedback mechanism where the primary database sends change notifications to the search database when data changes occur. This feedback loop allows the search database to stay synchronized with the primary database without requiring continuous operation of the indexing service, thereby improving reliability while maintaining search efficiency.
Solution Approach 2:
The patent applies preliminary action by having the primary database prepare and send change notifications before the search database needs to update its index. This proactive approach ensures that the search database is ready to receive updates without waiting for the indexing service to detect and process changes, reducing vulnerability to indexing service failures.
3Adaptability or versatility
If document properties are added to the schema, then search functionality is enhanced, but re-indexing costs increase significantly
Solution Approach 1:
When new document properties are added to the schema, the patent applies partial action by re-indexing only the documents that contain or will contain the new properties, rather than re-indexing the entire database. This selective approach allows the system to enhance search functionality with new properties while minimizing the time and resources required for re-indexing.
Solution Approach 2:
The patent applies local quality by treating different documents differently based on their relationship to the new schema properties. Documents that are affected by the new properties receive updated indexing, while documents that are not affected maintain their existing index structure, thereby reducing overall re-indexing time while still providing enhanced search functionality where needed.
4Reliability
If the search database is kept synchronized with the primary database, then data accuracy is maintained, but processing resources are continuously consumed
Solution Approach 1:
The patent implements periodic action through event-driven synchronization, where the search database updates its index only when triggered by change notifications from the primary database. This periodic update mechanism maintains data accuracy between updates while consuming minimal processing resources during idle periods, avoiding continuous resource consumption associated with constant synchronization.
Solution Approach 2:
The patent applies self-service by enabling the search database to automatically update itself based on notifications from the primary database. The system uses change notifications to trigger self-updates in the search database, reducing the need for continuous external processing resources while maintaining data accuracy through automated, on-demand synchronization.
Data Source
AI summary
Techniques are provided for selective re-indexing of a database. A pre-defined set of document properties are associated with one or more documents stored in a primary database. A schema defines which of the document properties are stored in a search database. When a search query is executed against a search database, a document satisfying the search query is returned from the search database along with the corresponding schema version number. If the schema version number associated with the document stored in the storage database is different from a target version number stored in a schema version lookup table, the data in the search database is presumed to be stale and a re-indexing of the search database is requested to bring the database current.