Intelligent Parallel Search Engine for Heterogeneous Data Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search systems fail to conduct intelligent parallel searches of multiple data sources with different formats within a second, requiring data conversion to a common format for sub-second search capabilities.
Innovation Solution
A method utilizing a search engine that extracts inquiry parameters and maps them to corresponding indices across various data sources, allowing for parallelized searches in a medium-agnostic and data type-agnostic manner, merging results to provide target data quickly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored in different formats in multiple data sources, then data diversity and adaptability are improved, but search speed and response time deteriorate because data conversion is required
Solution Approach 1:
The patent introduces an intermediary layer (search interface and indexing system) that translates diverse data formats into a unified searchable structure without requiring actual data conversion. The system maintains data in its native formats while creating format-agnostic indexes that enable direct parallel searching across heterogeneous sources, eliminating the conversion bottleneck.
Solution Approach 2:
The system segments the search process into independent parallel operations across multiple data sources. Each data source is searched independently using format-specific adapters, and results are merged afterward. This segmentation allows simultaneous processing of diverse formats without sequential conversion overhead.
2Speed
If data is converted to a common format for searching, then search speed is improved, but system complexity and data processing overhead increase
Solution Approach 1:
Instead of converting actual data to a common format, the system creates lightweight format descriptions and indexes that copy only the essential searchability metadata. These indexes serve as simplified representations that enable direct searching without manipulating the original diverse data formats, reducing conversion complexity.
Solution Approach 2:
The system performs preliminary indexing of data in its native formats before searching occurs. This pre-processing creates format-agnostic search structures that eliminate the need for real-time conversion during search operations, achieving fast search speeds without complex conversion infrastructure.
3Productivity
If parallel searches are conducted across multiple data sources, then productivity and search coverage are improved, but coordination complexity and result merging difficulty increase
Solution Approach 1:
The system enforces homogeneous result structures from diverse data sources by using standardized schemas and format descriptions. All search results are normalized to a common structure with consistent fields and data types, making merging straightforward regardless of the source data's original format diversity.
Solution Approach 2:
The patent implements a universal result merging framework that handles multiple data source formats through a single standardized interface. The merging system uses format descriptions and schemas to universally process and combine results from any data source, eliminating the need for source-specific merging logic.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are provided for conducting intelligent parallel searches of data sources. A search engine receives a request to access target data that is stored in at least one of multiple data sources. Each data source has a candidate index. The search engine extracts inquiry parameters from the request. The inquiry parameters corresponds to sub-indices of respective general indices. Each general index includes an index of relationships between data from at least two of the data sources. Each sub-index includes a subset of the respective general index. The search engine performs parallel searches of the general indices common to the data sources. Each parallel search includes searching the sub-indices based on corresponding inquiry parameters for the sub-indices. The search engine performs additional parallel searches of the candidate indices based on results of the parallel searches. The search engine extracts an output from results of the additional parallel searches.