Configurable Criteria Processing via Data Partitioning and Parallel Workers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for processing large and complex data sets with arbitrary criteria are inefficient, leading to slow and computationally intensive processing, which can result in manual intervention and increased risk of errors, especially when criteria change over time.
Innovation Solution
The implementation of enhanced techniques that allow for the efficient processing of configurable criteria using a processor to apply user-defined attributes to data sets, enabling faster and more efficient determination of elements that meet specific criteria through the use of algorithms and data structures that reduce time complexity to linear time, facilitating parallel execution and dynamic criteria application.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional techniques are used to process large and complex data sets with arbitrary criteria, then processing can be performed with simple systems, but processing time and computational complexity become excessive
Solution Approach 1:
The patent segments the data processing task by dividing the data set into partitions and distributing processing across multiple workers. Each worker handles specific partitions with specific criteria, enabling parallel processing that reduces overall processing time while maintaining simplicity of individual processing units.
Solution Approach 2:
The patent introduces a new dimension of parallel processing by executing multiple criteria evaluations simultaneously across different data partitions. This transforms the single-threaded sequential processing into multi-threaded parallel processing, dramatically improving productivity without increasing the complexity of individual processing logic.
2Adaptability or versatility
If manual scrutiny is used to evaluate data lines against external criteria, then flexibility in applying arbitrary criteria is maintained, but processing efficiency decreases and error risk increases
Solution Approach 1:
The system enables self-service processing where the computational infrastructure automatically evaluates data lines against configured criteria without requiring manual intervention. The workers autonomously retrieve data, apply criteria, and generate results, maintaining the flexibility of arbitrary criteria while eliminating manual scrutiny bottlenecks.
Solution Approach 2:
The patent replaces the mechanical process of manual scrutiny with an automated computational system. Instead of human evaluators manually comparing data lines against criteria, the system uses programmed workers that automatically execute criteria evaluations, thereby maintaining criteria flexibility while dramatically improving processing efficiency and reducing error risk.
3Reliability
If sophisticated database management systems are used for data storage and retrieval, then data management capability is improved, but processing overhead increases and computational efficiency decreases
Solution Approach 1:
The patent extracts the data retrieval operation from the complex database management system and performs it directly within the worker processes. By taking out the retrieval step and executing it in-memory or from simplified storage within each worker, the system reduces the overhead imposed by sophisticated DBMS while maintaining reliable data access through proper data loading strategies.
Data Source
AI summary
Disclosed herein are computer-implemented method, system, and computer-program product (computer-readable storage medium) embodiments for efficiently processing configurable criteria. An embodiment includes at least one computer processor configured to receive a first configuration corresponding to a plurality of attributes, access a data set of one or more items, and compute a first key based at least in part on the first configuration. In a further embodiment, the at least one computer processor may be configured to populate a first set of selected items based at least in part on the first key, and may further be further configured to output the first set of selected items. The first key may uniquely correspond to the first configuration, and the first set of selected items may include any item, of the data set of one or more items, selected based at least in part on the first key, according to some embodiments.