Database Resource Policy for External UDF Process Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in monitoring and controlling resource consumption by user-defined functions (UDFs) executed in external processes, as existing methods either impose hard memory limits leading to out-of-memory errors or require cumbersome code instrumentation.
Innovation Solution
A resource policy is defined with threshold values for resource consumption, allowing for actions such as queuing, rejection, or forced cancellation of UDFs to manage resource usage dynamically without altering the UDF code, using operating system functions to monitor and control external processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a hard memory limit is imposed on external processes, then memory consumption is controlled, but out-of-memory errors occur and processes terminate
Solution Approach 1:
The patent introduces an intermediary mechanism (resource policy with threshold values) between the database management system and external processes. This intermediary monitors resource consumption and triggers actions (queuing, rejection, cancellation) before hard limits are reached, preventing out-of-memory errors while maintaining process continuity.
Solution Approach 2:
The system performs preliminary actions by defining resource policies with threshold values in advance. When resource consumption approaches these thresholds, the system proactively takes actions (queuing new requests, rejecting them, or canceling running UDFs) to prevent exceeding memory limits, thereby avoiding process termination.
2Ease of operation
If memory limit is applied to each UDF process individually, then per-process control is achieved, but aggregated memory consumption cannot be controlled
Solution Approach 1:
The patent merges individual process control with aggregate control by implementing resource policies at the database management system level. The system monitors and controls the cumulative resource consumption of all external processes while maintaining the ability to apply policies to individual processes, thus achieving both per-process and aggregated memory control.
3Measurement precision
If code instrumentation is used to intercept memory allocations, then resource monitoring is achieved, but programming complexity increases
Solution Approach 1:
Instead of instrumenting the target program code, the patent introduces an intermediary resource control mechanism at the database management system level. This intermediary monitors resource consumption of external processes through system-level interfaces without requiring any modifications to the user-supplied code, thereby achieving precise resource monitoring while avoiding programming complexity.
4Adaptability or versatility
If multiple separate user-defined processes are spawned, then functional versatility is improved, but control over aggregated resource consumption is lost
Solution Approach 1:
The resource policy mechanism serves multiple functions simultaneously: it monitors individual processes, controls aggregated resource consumption, and provides hierarchical control (from individual process to system-wide level). This universal mechanism handles both per-process and aggregate resource management without requiring separate control systems.
Data Source
AI summary
Embodiments relate to managing resource consumption in a computing system. An aspect includes providing a resource policy by defining a plurality of threshold values relating to the resource consumption, wherein the resources are consumed by a plurality of user-defined functions performing tasks for a database management system, wherein the user-defined functions are executed by a plurality of processes external to the database management system. Another aspect includes performing an action, as defined by the resource policy, on at least one of the user-defined functions.