Compute Request Routing with Historical Query Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale multicomputer datacenters face inefficiencies in routing compute requests due to lack of domain knowledge in load balancers, leading to inefficient resource utilization and increased risk from user-defined code, without considering historical information for query routing.
Innovation Solution
A server system with a software architecture for intelligently dispatching compute requests to appropriate back-end resources, utilizing a routing pipeline that scores resources based on query analysis, historical data, and lifecycle management to optimize resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If generic load balancers are used to route queries, then device complexity is reduced, but resource utilization efficiency deteriorates due to lack of domain knowledge about queries
Solution Approach 1:
The patent introduces a query analysis intermediary component that sits between the load balancer and compute resources. This intermediary analyzes query characteristics, identifies similarity patterns, and provides enhanced routing decisions, thereby improving resource utilization without significantly increasing overall system complexity.
Solution Approach 2:
The routing system is segmented into multiple specialized components: query analysis module, similarity identification module, caching functionality module, and load balancing module. Each component performs a specific function, allowing the system to achieve high resource utilization efficiency while maintaining manageable complexity through modular design.
2Device complexity
If caching functionality is not utilized due to inability to identify query similarity, then device complexity is reduced, but productivity deteriorates from not taking full advantage of caching
Solution Approach 1:
The system performs preliminary query analysis and similarity identification before routing decisions are made. By pre-processing queries to identify similarity patterns and potential cache matches, the system enables effective caching functionality that improves throughput without adding significant operational complexity.
3Productivity
If more compute resources are allocated to handle complex queries, then productivity is improved, but resource waste increases due to inefficient routing
Solution Approach 1:
The system dynamically changes routing parameters based on query characteristics, complexity metrics, and historical performance data. By adjusting routing decisions according to these parameters, the system optimizes resource allocation to match actual workload requirements, improving productivity while minimizing resource waste from inefficient routing.
4Adaptability or versatility
If user defined code is allowed in compute requests, then adaptability is improved, but system reliability deteriorates due to introduced security risks
Solution Approach 1:
The patent introduces a query analysis intermediary that acts as a mediator between user-defined code and the compute resources. This intermediary performs security validation, risk assessment, and filtering on user-defined code before execution, thereby maintaining system reliability while preserving the adaptability benefits of allowing custom code in queries.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and methods for determining computer resource allocation, the system having a network communication interface configured to receive a query from a device, the query indicating a request to perform a processing operation on a portion of one or more data set. The system may also include data storage for data including historical information related to processing of data sets by back-end computer resources, and hardware processors configured to determine one or more scores associated with a query and based at least in part on the historical information. The system may determine a particular back-end computer resource with a highest score, provide a compute request to the particular back-end computer resource to perform a processing operation on a portion of one or more data sets, and store processing information related to the processing of the compute request by the particular back-end computer resource as historical information.