Dynamic Discovery Scaling for Partitioned Search Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional decision-making systems struggle to execute analytical models across vast datasets in real-time due to computational intensity and underutilization of resources, especially when dealing with negotiable parameters, leading to inefficient processing and suboptimal transaction outcomes.
Innovation Solution
A distributed approach utilizing the Apache™ licensed Akka framework for thread management and on-demand scalability, enabling parallel computing and data segmentation to process large datasets across multiple CPUs, ensuring timely and tailored transaction outcomes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computing systems are used to execute analytical models across vast datasets, then real-time processing is required, but computational intensity and resource underutilization make this infeasible
Solution Approach 1:
The patent segments the vast dataset into multiple partitions and distributes them across multiple computing nodes in a cluster. Each node processes a specific partition independently, enabling parallel execution of analytical models. This segmentation resolves the contradiction by dividing the computationally intensive task into manageable chunks that can be processed simultaneously, improving productivity while optimizing resource utilization across the distributed system.
Solution Approach 2:
The patent transitions from single-node sequential processing to multi-node parallel processing by adding a spatial dimension to the computational architecture. By distributing data and processing across multiple nodes in a cluster, the system achieves real-time processing capability without overloading individual resources, thus resolving the contradiction between processing speed and resource utilization.
2Productivity
If a single node serves all traffic with available CPUs, then processing capacity is maximized on that node, but other nodes in the cluster remain underutilized
Solution Approach 1:
The patent segments the data processing workload by partitioning the dataset and assigning different partitions to different nodes in the cluster. This allows multiple nodes to contribute their CPU resources simultaneously, maximizing overall cluster processing capacity while preventing resource underutilization. The segmentation principle transforms a single-node bottleneck into a distributed parallel processing system.
Solution Approach 2:
The patent makes the cluster function as a unified parallel processing system where each node can handle different data partitions. This multi-functionality allows the entire cluster to contribute to processing requests, ensuring that all nodes are utilized effectively rather than leaving resources idle, thus resolving the contradiction between processing capacity and resource underutilization.
3Adaptability or versatility
If more negotiable parameters are added to transaction structures, then transaction flexibility improves, but decision-making system performance deteriorates
Solution Approach 1:
The patent segments the analytical model execution into parallel tasks distributed across multiple nodes. Each node independently evaluates transaction scenarios with various negotiable parameters simultaneously. This parallel evaluation maintains transaction flexibility while improving decision-making speed by processing multiple parameter combinations concurrently rather than sequentially.
Solution Approach 2:
The patent performs preliminary data filtering and preprocessing to identify only those data records that are relevant to the current transaction parameters. By pre-filtering the dataset before analytical model execution, the system can efficiently evaluate multiple negotiable parameters without processing the entire vast dataset, thus maintaining both transaction flexibility and decision-making productivity.
Data Source
AI summary
Aspects described herein may relate to methods, systems, and apparatuses that partitions searchable content and distributes the segments across a plurality of processing nodes, which in turn further sub-partitions the partitions for processing by local search actor in order to increase the speed with which a search request from a user is processed. Processing nodes available to receive partitioned searchable content are registered with an external storage device. The external storage device also maintains a global results collector that compiles results from the partitions of searchable content. Respective local collector actors receive compiled results from local search actors for a processing node and the compiled results are sent to the global results collector for compiling for the plurality of processing nodes. Results of the user search request are then provided to the user.