Data Pattern Metadata for Rules-Based Search Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rules-based search systems face performance bottlenecks when dealing with large datasets due to the need to compare each entity's attributes with input attributes, leading to inefficient processing.
Innovation Solution
The use of metadata, specifically data patterns and attribute groups, reduces the amount of information to be searched by representing attribute keys and values in a way that multiple entities sharing common attributes map to the same data pattern, allowing for faster search processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional relational database traversal is used to search entities by comparing each entity's attributes with input attributes, then complete search coverage is achieved, but processing time increases linearly with the number of attributes and entities
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing data patterns (combinations of attribute keys) and attribute groups (combinations of attribute values) for all entities before search operations. This preprocessing creates metadata structures that enable rapid search without comparing every entity's attributes during query execution, thus reducing processing time while maintaining search completeness
Solution Approach 2:
The patent introduces data patterns and attribute groups as intermediary structures between the input attributes and the entity data. Instead of directly comparing input attributes with all entity attributes, the search process uses these intermediaries to filter and identify candidate entities first, then performs verification only on those candidates, significantly reducing the number of comparisons needed
2Measurement precision
If all entity attributes are stored and searched in a relational database, then accurate attribute matching is achieved, but the amount of data to be processed increases with the number of entities
Solution Approach 1:
The patent extracts only the essential structural information from entity attributes by creating data patterns (attribute key combinations) and attribute groups (attribute value combinations). This extraction separates the search-critical structural metadata from the full entity data, allowing the system to work with a much smaller representation that preserves matching accuracy while reducing data volume
Solution Approach 2:
The patent creates simplified copies of entity attributes in the form of data patterns and attribute groups stored in metadata tables. These copies contain only the combination structures needed for search operations rather than complete entity data, enabling efficient processing while maintaining the ability to accurately match attributes through the copy-comparison mechanism
Data Source
AI summary
In various embodiments, systems and methods are provided that can facilitate searching for entities, such as rules, that apply to search criteria. The disclosed systems and methods can reduce some of the performance bottlenecks associated with, for example, rules-based search systems by using metadata. The metadata may be generated to reduce the size of information about a set of entities that is required to be searched. In some embodiments, the metadata may represent one or more tuple elements, such as keys and values of keys in key-value pairs.