Searchable Data Service Indexing for Flexible Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage solutions lack flexibility in retrieving data objects based on attributes other than file paths, requiring significant administrative effort and being expensive in the long term, with scalability issues as data stores grow.
Innovation Solution
A general-purpose searchable data service that allows developers to store attributes as {name, value} pairs associated with data objects, enabling flexible retrieval through searchable indexes, scalable, and transparent to clients, with mechanisms for redundancy, reliability, and high availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional data storage solutions use file path locators for data retrieval, then the system structure is simple, but the flexibility to retrieve data objects according to attributes other than file paths is lost
Solution Approach 1:
The patent segments the data storage system into multiple independent components: a data store for storing data objects, a separate indexing service for creating and maintaining indexes based on attributes, and a query interface for flexible retrieval. This segmentation allows the system to maintain simplicity in each component while achieving overall flexibility through their coordination.
Solution Approach 2:
The patent introduces an intermediary indexing service that acts as a mediator between the data store and the query interface. This indexing service maintains attribute-based indexes that enable flexible data retrieval without requiring changes to the underlying data store structure, thus providing versatility while maintaining system simplicity.
2Productivity
If conventional data storage solutions are used, then initial setup is straightforward, but significant administrative effort is required for monitoring, partitioning, query optimizations, and maintenance as data stores grow
Solution Approach 1:
The patent implements self-service mechanisms where the indexing service automatically performs monitoring, partitioning, and query optimization as the data store grows. The system autonomously manages index creation, maintenance, and distribution across multiple servers, eliminating the need for manual administrative intervention and enabling seamless scalability.
Solution Approach 2:
The patent employs dynamic mechanisms that allow the system to adapt automatically as data grows. The indexing service dynamically partitions indexes, distributes them across servers, and optimizes query performance based on current system state and workload characteristics, enabling the system to scale without fixed administrative overhead.
3Ease of manufacture
If off-the-shelf database products are used for data storage, then initial implementation is easier, but long-term costs are expensive
Solution Approach 1:
The patent employs cost-effective, simple indexing structures and algorithms that can be implemented with minimal resources. Rather than relying on expensive proprietary database systems, the invention uses lightweight, open-source-compatible approaches for indexing and query processing, reducing both initial implementation costs and long-term operational expenses.
Solution Approach 2:
The patent creates a universal indexing service that can work with various data storage backends and supports multiple query types and data formats. This multi-functional approach eliminates the need for specialized expensive database products, as the same indexing infrastructure serves diverse data storage and retrieval needs efficiently.
4Adaptability or versatility
If table schemas are changed to support new attributes for querying, then new query capabilities are added, but system maintenance complexity increases
Solution Approach 1:
The patent implements a dynamic attribute-based indexing system where new query capabilities are added by simply registering new attributes with the indexing service, without requiring changes to table schemas or data structures. The system dynamically adapts to new attributes by automatically creating corresponding indexes, maintaining versatility while avoiding maintenance complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A searchable data service implementation may include, but is not limited to, a Web services platform (200), one or more coordinator nodes (350), one or more query nodes, referred to as query TSAR (Top Search AggregatoR) nodes (360), and one or more storage nodes (370). Each coordinator node (350) may include, but is not limited to, at least one instance of request router (202). A client system (330) may submit service requests (query node requests and/or storage node requests) to the searchable data service in accordance with the Web service interface of the Web services platform (200) via Internet (334). The Web services platform (200) may route the service request (s) to a coordinator node (350). A coordinator node (350) routes the service requests to the appropriate node(s) , collects result, and send the results back to the web services platform (200). A request router on the coordinator node (350) may receive the service request (s) from the Web services platform (200) and determine whether each service request is a storage node request or a query node request.