Learning-based workload resource optimization for database management systems

Abstract

The DBMS training subsystem utilizes training data to train a DBMS workload manager model, the training data identifying resources used to execute previous DBMS data access requests. The subsystem integrates high-level features of each request and compile-time operations into a vector, and clusters similar vectors into templates. The requests are divided into workloads, each workload represented by a training histogram that describes the distribution of templates associated with the workload and identifies the total amount and type of resources consumed when executing the entire workload. The resulting knowledge is used to train the model to predict production resource requirements by the following steps: i) organizing production queries into candidate workloads; ii) deriving for each candidate a histogram that is similar in form and function to the training histograms; iii) using the newly derived histograms to predict resource requirements for each candidate; iv) selecting the candidate with the largest resource requirements that can be satisfied with available resources; and v) executing the selected workload.

Description

BACKGROUND

[0001] The present invention relates generally to database management systems (DBMSs), and more particularly to prediction or estimation of resource requirements caused by execution of database queries.

[0002] Current workloads of a DBMS are a set of data access requests (such as structured query language (SQL) queries) that seek to analyze or revise the contents of a database that is currently being processed by the DBMS. DBMS workloads can require very different amounts and types of resources, with each workload requiring a specific combination and specific amount of computer memory, CPU cycles, I / O bandwidth, secondary storage, and other resources.

[0003] A DBMS runs more efficiently if it can accurately predict the type and amount of resources needed to process each workload statement. Because a workload query or other type of data access request does not explicitly identify the resources it consumes, known DBMSs can at best guess the resource requirements by applying generic static empirical rules to each query. For example, one rule can be tuned to predict the resources needed to process a SQL GROUP BY predicate. When a query containing a GROUP BY predicate is received, the DBMS will parse the query to identify dozens of database operations needed to process the GROUP BY functionality of the query. The DBMS will then apply a unique estimation model to analyze each combination of the extracted low-level features and resource types to guess the resource requirements caused by processing the feature. The DBMS will then predict the overall requirements of the GROUP BY predicate by aggregating its resource consumption estimates for each low-level feature. This process is repeated for each resource-consuming operation or predicate of the query in order to estimate the resource requirements of the entire query.

[0004] Such rules typically only consider the expression language of each query statement, often forcing the DBMS to extract low-level features extracted from each workload statement and then apply a specific resource estimation model to each parsed feature. For example, one rule can be tuned to predict the resources needed to process a SQL GROUP BY predicate. When a query containing a GROUP BY predicate is received, the DBMS will parse the query to identify dozens of database operations needed to process the GROUP BY functionality of the query. The DBMS will then apply a unique estimation model to analyze each combination of the extracted low-level features and resource types to guess the resource requirements caused by processing the feature. The DBMS will then predict the overall requirements of the GROUP BY predicate by aggregating its resource consumption estimates for each low-level feature. This process is repeated for each resource-consuming operation or predicate of the query in order to estimate the resource requirements of the entire query. SUMMARY

[0005] Embodiments of the present invention include systems, methods, and computer program products for self-learning a database management system (DBMS). These embodiments include an expert workload manager component and a training subsystem that uses machine learning techniques to train a cognitive model to estimate resource requirements of a batch of queries or other types of data access requests.

[0006] In a first type of "training mode" embodiment, a training subsystem of the DBMS submits training data to the workload management model that identifies the number and type of resources previously used by the DBMS in servicing previous data access requests. The system identifies various high-level features of each request, such as tokens parsed from the text of the request, as well as metrics and compile-time operations referenced by the execution plans generated by the query optimizer component of the DBMS. The system packages the high-level features of each request into a query vector, and the vectors are clustered into mutually exclusive subsets of similar vectors (or "templates"). The subsystem organizes the data access requests into batches of workloads, and each workload is associated with a histogram that represents the distribution of templates associated with the batch of data access requests that make up that workload. Because the training data identifies the number and type of resources incurred by the previous execution of each request, in some embodiments the histogram is also associated with a resource requirement set that specifies the number and type of resources required to service the corresponding workload of the histogram.

[0007] In other embodiments, each workload is associated with multiple histograms. In such embodiments, the resource requirement set associated with each histogram specifies the number of only one of the types of resources required to service the corresponding workload of the histogram.

[0008] In either case, this knowledge is then formatted into the form of a machine learning corpus, such as a triple store data structure that represents concepts, rules, and dependencies comprising the artificial intelligence "knowledge" as a set of 3-tuple vectors, and is used to train the model. During production, in response to receiving a histogram representing the distribution of templates associated with all of the requests in one production workload, the fully trained model will then be able to predict the aggregate cost of executing the entire production workload as a unique function of the received histogram.

[0009] In a second type of "production mode" embodiment, a production DBMS applies the knowledge derived by the training mode embodiments to estimate the resources required to execute batches of incoming data access requests. The DBMS associates each incoming request with a query vector that is subsequently assigned to one of the previously generated templates in a similar manner as was performed during the training mode. The DBMS builds a series of candidate workloads by adding incoming requests one at a time to create increasingly resource-intensive workloads. As in the training phase, each candidate workload is associated with a histogram that identifies the distribution of templates among the statements that make up the workload. When each new candidate workload is generated, the DBMS sends the corresponding histogram or histograms for the workload to the trained model. In response, the model predicts the resource requirements for the workload based on the previous training of the model. This process continues until the requirements for a candidate workload become too large to be satisfied by the resources currently available. The next-to-maximum workload is then considered to be the maximum workload that can be executed with the available resources, and that workload is forwarded to the query execution engine of the DBMS for further processing.

[0010] According to an aspect of the present application, there is provided a training subsystem of a database management system (DBMS), comprising a processor, a memory coupled to the processor, and a computer readable hardware storage device coupled to the processor, the storage device containing program code configured to be executed by the processor via the memory to implement a method for self-learning binned resource optimization of a database management system, the method comprising: the training subsystem receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and a record of types and amounts of resources required by the DBMS to execute each statement in the set of training statements; the subsystem generating a set of query vectors, each query vector identifying high-level characteristics of a corresponding statement in the training statements; the subsystem assigning the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one template in the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity between them that exceeds a threshold similarity; the subsystem dividing the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one template in the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem associating each corresponding histogram with a set of resources that were consumed in the past by execution of each training statement of the corresponding training workload of that histogram; the subsystem adding knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem training a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

[0011] According to another aspect, the present application provides a method for self-learning box-wise resource optimization of a database management system, the method comprising: a training subsystem of a database management system DBMS receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and a record of types and amounts of resources required by the DBMS to execute each statement in the set of training statements; the subsystem generating a set of query vectors, each query vector identifying high-level characteristics of a corresponding statement in the training statements; the subsystem assigning the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one template in the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity between them that exceeds a threshold similarity; the subsystem dividing the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one template in the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem associating each corresponding histogram with a set of resources consumed in the past by execution of each training statement of the corresponding training workload of that histogram; the subsystem adding knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem training a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

[0012] According to another aspect of the present application, there is provided a computer program product for self-learning boxy resource optimization of a database management system, DBMS, the computer program product comprising a computer readable storage medium having program instructions executable by a processor to cause the processor to perform a method for self-learning boxy resource optimization of a database management system, the method comprising: a training subsystem of a database management system, DBMS, receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and a record of types and amounts of resources required by the DBMS to execute each statement in the set of training statements; the subsystem generating a set of query vectors, each query vector identifying high-level characteristics of a corresponding statement in the training statements; the subsystem assigning the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one template in the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity between them that exceeds a threshold similarity; the subsystem dividing the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one template in the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem associating each corresponding histogram with a set of resources consumed in the past by execution of each training statement of the corresponding training workload of that histogram; the subsystem adding knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem training a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

[0013] According to another aspect of the present application, there is provided a database management system (DBMS) comprising a processor, a memory coupled to the processor, and a computer readable hardware storage device coupled to the processor, the storage device containing program code configured to be executed by the processor via the memory to implement a method for self-learning bin-wise resource optimization for a database management system, the method comprising: the DBMS receiving a set of production statements, wherein each production statement requests access to a database managed by the DBMS, and a cognitive model trained to predict resource requirements for batch data access requests; the DBMS generating a set of query vectors, wherein each vector in the set of query vectors identifies high-level characteristics of a corresponding statement in the set of production statements, and wherein each element of a particular vector in the set of query vectors identifies a unique high-level characteristic of a corresponding statement in the set of production statements; the DBMS assigning each vector in the set of query vectors to a corresponding template in a set of model templates maintained by a clustering model of the cognitive model; the DBMS iteratively generating a final production workload, wherein each iteration comprises the steps of: the DBMS adding a next statement selected from the set of production statements to a candidate production workload, wherein the next statement was not previously added to the candidate production workload, the DBMS attempting to associate the candidate production workload with a candidate histogram, wherein each bin of the candidate histogram identifies a total number of statements of the candidate production workload that are associated with templates in the set of model templates that are uniquely assigned to that bin; the DBMS, if the candidate production workload is able to be associated with the candidate histogram, predicting, according to training of the cognitive model, a quantity and type of resources that will be required to execute the candidate production workload equaling those identified by the cognitive model as resulting from executing any workload characterized by a matching histogram according to training of the cognitive model, the DBMS, if it is determined that executing the candidate production workload will require more resources than are currently available to the DBMS, removing a most recently added statement from the candidate production workload and considering the resulting workload to be the final production workload, and the DBMS, if it is determined that executing the candidate production workload will not require more resources than are currently available, continuing with a next iteration of generating the final production workload; and the DBMS forwarding the final production workload to a query execution engine of the DBMS for execution by the DBMS.

[0014] According to another aspect of the present application, there is provided a method comprising: a production database management system (DBMS) receiving a set of production statements and a cognitive model, wherein each production statement requests access to a database managed by the DBMS, and wherein the cognitive model is trained to predict resource requirements of batch data access requests; the DBMS generating a set of query vectors, wherein each vector in the set of query vectors identifies high-level characteristics of a corresponding statement in the set of production statements, and wherein each element of a particular vector in the set of query vectors identifies a unique high-level characteristic of a corresponding statement in the set of production statements; the DBMS assigning each vector in the set of query vectors to a corresponding template in a set of model templates maintained by a clustering model of the cognitive model; the DBMS iteratively generating a final production workload, wherein each iteration comprises the steps of: the DBMS adding a next statement selected from the set of production statements to a candidate production workload, wherein the next statement was not previously added to the candidate production workload, the DBMS attempting to associate the candidate production workload with a candidate histogram, wherein each bin of the candidate histogram identifies a total number of statements of the candidate production workload that are associated with templates in the set of model templates that are uniquely assigned to that bin; the DBMS, if the candidate production workload is able to be associated with the candidate histogram, predicting, in accordance with training of the cognitive model, a quantity and type of resources that will be required to execute the candidate production workload equal to those identified by the cognitive model as resulting from execution of any workload characterized by a matching histogram in accordance with the training of the cognitive model, the DBMS, if a matching histogram is able to be derived, predicting a quantity and type of resources that will be required to execute the candidate production workload equal to those identified by the cognitive model as resulting from execution of any workload characterized by the matching histogram in accordance with the training of the cognitive model, the DBMS, if it is determined that execution of the candidate production workload will require more resources than are currently available to the DBMS, removing a most recently added statement from the candidate production workload and considering the resulting workload to be the final production workload, and the DBMS, if it is determined that execution of the candidate production workload will not require more resources than are currently available, continuing with a next iteration of generating the final production workload; and the DBMS forwarding the final production workload to a query execution engine of the DBMS for execution by the DBMS. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A cloud computing environment according to an embodiment of the present application is shown;

[0016] Figure 2An abstraction model layer according to embodiments of the application is shown;

[0017] Figure 3 A structure of a computer system and computer program code that can be used to implement a method for self-learning box resource optimization for a database management system according to embodiments of the application is shown;

[0018] Figure 4 An architecture of a self-learning workload management component of a database management system and related components according to embodiments of the application is shown;

[0019] Figure 5 is a high level flowchart showing a method for training a self-learning DBMS workload manager to perform box resource optimization and then using the results of that training to optimize the estimation of resource requirements in a production environment according to embodiments of the application;

[0020] Figure 6 is a flowchart showing steps for training a DBMS system with self-learning box resource optimization according to embodiments of the application;

[0021] Figure 7 is a flowchart showing steps of a DBMS system with self-learning box resource optimization according to embodiments of the application. DETAILED DESCRIPTION

[0022] Known methods of estimating DBMS resource requirements employ relatively uncomplicated workload management mechanisms that must extract low-level features from individual data access statements and then associate each extracted feature with a corresponding set of requirements. These mechanisms do not take into account more nuanced factors related to the parallel processing of multiple statements in a workload and do not provide the flexibility and robustness needed to operate without human database administrator maintenance.

[0023] Known DBMS workload manager technologies also suffer from the effects of complexity and overhead caused by their need to use different resource consumption models to analyze each possible combination of low-level features and resource types. Known implementations also require continuous monitoring and maintenance in order to manually modify these models whenever there are changes to the DBMS application, database schema, computing platform, or general composition or timing of submitted workloads.

[0024] Embodiments of the invention provide methods, systems, and computer program products that improve the workload management subsystems and techniques employed by known DBMSs. These improvements include a self-learning workload management function or component that incorporates a cognitive model trained to predict resource requirements at the workload level. Unlike existing DBMS workload managers that attempt to guess the requirements for one statement at a time, the invention uses a top-down approach that estimates the resource requirements for an entire workload.

[0025] Generally, top-down estimation models are easier to implement than bottom-up models that must analyze the resource requirements of each low-level query feature. However, top-down models are generally less accurate because such models have access to less information when generating resource estimates for individual query statements.

[0026] The invention addresses this problem by analyzing the resource requirements of the query statements of an entire workload batch, rather than just by parsing isolated queries. Because a workload can contain a large number of heterogeneous data access statements that compete for various resources, embodiments estimate resource requirements according to specific high-level feature characteristics of the entire workload.

[0027] In this document, the terms "data access request," "database access request," "database query," "data access statement," and similar terms will sometimes be used interchangeably to refer to instructions received by a DBMS to add, delete, update, and retrieve data elements, indexes, and other content of a database. Although specific definitions of these terms can distinguish, for example, "queries" from "statements," such distinctions are not closely related to the form or function of the invention, for example when a SQL query is defined as consisting of multiple statements. Depending on the implementer's preference and technical considerations, various embodiments can be configured to use the same steps to process individual SQL statements, entire queries, or other types of data access requests.

[0028] These characteristics include syntactic elements (such as structured query language (SQL) operators), semantically meaningful identifiers (such as table, column, and subpattern aliases), and features of query execution plans generated by the optimizer module of the DBMS. Particular embodiments implement at least a portion of these analysis steps as a distribution-regression learning problem that generates a vector data structure that can be submitted to a deep learning neural network or incorporated into a machine learning corpus of training data. When put into production, the improved workload manager that has been trained by the top-down, batch-oriented approach of this embodiment can more accurately determine whether adding a single query to an existing DBMS workload will over-allocate limited resources.

[0029] Embodiments of the present invention also improve upon known DBMS workload management components by freeing the DBMS from the need to extract and analyze every low-level feature of every query statement. Therefore, the embodiments do not require unique resource estimation models to analyze each type of low-level feature and query operator or predicate. Furthermore, because the present invention leverages experience to intelligently learn which resource consumption predictions are most accurate or produce the greatest performance improvements, the embodiments do not need to be manually reconfigured when the database schema, application or computing platform configuration, or the overall timing or composition of the workload submitted to the DBMS changes.

[0030] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0031] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0032] The features are as follows:

[0033] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider.

[0034] Extensive network access: Capabilities are available on the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0035] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. Location independence has significance because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0036] Rapid Flexibility: In some cases, the ability to scale outwards and inwards quickly and flexibly can be provided. For consumers, the available capacity often appears unlimited and can be purchased in any quantity at any time.

[0037] Measurement services: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

[0038] Service models are as follows:

[0039] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0040] Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0041] Infrastructure as a Service (IaaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include an operating system and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0042] Deployment models are as follows:

[0043] Private cloud: the cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0044] Community cloud: the cloud infrastructure is shared by several organizations and supports mission-oriented business

[0045] Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

[0046] Hybrid cloud: cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that enable data and application portability, for example, cloud bursts for load balancing between clouds.

[0047] A cloud computing environment is service-oriented, focusing on stateless, low-coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure comprising a network of interconnected nodes.

[0048] Referring now to the drawing Figure 1 , an illustrative cloud computing environment 50 is depicted. As shown, cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone 54A, desktop computer 54B, laptop computer 54C, and / or automobile computer system 54N can communicate. Nodes 10 can communicate with one another. They can be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows cloud computing environment 50 to offer infrastructure, platforms and / or software as services with Figure 1 The types of computing devices 54A-N shown in FIG. 10 are intended to be illustrative only and computing nodes 10 and cloud computing environment 50 can communicate with any type of computerized devices over any type of network and network addressable connection (e.g., using a web browser).

[0049] Referring now to FIG. 11, Figure 2 , a set of functional abstraction layers provided by cloud computing environment 50 Figure 1 is shown. It should be understood that Figure 2 The components, layers, and functions shown in FIG. 11 are intended to be illustrative only and embodiments of the application are not limited in their scope to what is depicted in FIG. 11. As is depicted, the following layers and corresponding functions are provided:

[0050] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; RISC (Reduced Instruction Set Computer) architecture based servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0051] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0052] In one example, management layer 80 can provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 82 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 85 provide pre-arrangement for, and procurement of, cloud computing resources for which future requirements are anticipated in accordance with an SLA.

[0053] Workloads layer 90 provides examples of functionality that can be utilized by the cloud computing environment. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and coordination of optimized resource requirements estimation for batch workloads against database management systems.

[0054] The present application can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0055] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0056] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adaptation card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to storage media within the respective computing / processing device for execution.

[0057] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine- related instructions, microcode, firmware instructions, state-setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and a procedural programming language such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0058] Aspects of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0059] These computer readable program instructions can be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions for

[0060] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational acts to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0061] The flow diagrams and block diagrams in the drawings are illustrative of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions (‘instructions’). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or with partial or total temporal overlap, or even in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0062] Figure 3 The structure of a computer system and computer program code according to embodiments of the application that can be used to implement the method for self-learning chest resource optimization for a database management system is shown. Figure 3 Objects 301-315 are involved.

[0063] In Figure 3 The computer system 301 comprises in this embodiment a processor 303 coupled to one or more hardware data storage devices 311 and one or more I / O devices 313 and 315 via one or more I / O interfaces 309.

[0064] The hardware data storage device 311 can include, but is not limited to, a tape drive, a fixed or removable hard disk drive, an optical drive, a mobile device with memory, and a solid-state random access or read-only memory device. The I / O devices can include, but are not limited to, input devices 313, such as a keyboard, a scanner, a handheld

[0065] The processor 303 can also be connected to one or more storage devices 305, which can include, but are not limited to, a dynamic RAM (DRAM), a static RAM (SRAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a solid-state drive (SSD), a flash drive, a floppy disk drive, a magnetic or optical disk drive, a magnetic or optical disk, a removable memory card or other storage device.

[0066] The at least one storage device 305 contains stored computer program code 307, which is a computer program comprising computer-executable instructions. The stored computer program code comprises a program implementing the method for self-learning box resource optimization for a database management system according to embodiments of the present application, and can implement other embodiments described in this specification, including the method shown in Figures 1-7 The data storage device 311 can store the computer program code 307. The computer program code 307 stored in the storage device 311 is configured to be executed by the processor 303 via the storage device 305. The processor 303 executes the stored computer program code 307.

[0067] In some embodiments, rather than being stored in and accessed from a hard disk drive, optical disk, or other writable, rewritable, or removable hardware data storage device 311, the stored computer program code 307 can be stored on a static, non-removable read-only storage medium, such as a read-only memory (ROM) storage device 305, or can be accessed directly by the processor 303 from such a static, non-removable read-only medium storage device 305. Similarly, in some embodiments, the stored computer program code 307 can be stored as computer-readable firmware, or can be accessed directly by the processor 303 from such firmware, rather than from a more dynamic or removable hardware data storage device 311 such as a hard disk drive or optical disk.

[0068] Accordingly, the present application discloses a process for supporting computer infrastructure, integrating computer readable code with the processes of the application into computer systems 301, and maintaining and deploying computer code in connection with the computer system 301, wherein the code in combination with the computer system 301 is capable of performing methods for self-learning box resource optimization for database management systems.

[0069] Any of the components of the application can be created, integrated, hosted, maintained, deployed, managed, serviced, supported, and so on by a service provider who offers services that facilitate methods for self-learning box resource optimization for database management systems. Accordingly, the present application discloses a process for deploying or integrating computing infrastructure, comprising integrating computer readable code with the processes of the application into computer systems 301, wherein the code in combination with the computer system 301 is capable of performing methods for self-learning box resource optimization for database management systems.

[0070] One or more data storage devices 311 (or Figure 3 One or more additional storage devices (not shown in the figure) can be used as computer readable hardware storage devices having computer readable programs and / or having other data stored therein, wherein the computer readable programs include stored computer program code 307. Generally, a computer program product (or alternatively, an article of manufacture) of the computer system 301 can include the computer readable hardware storage devices.

[0071] In embodiments that include components of a networking computing infrastructure, cloud computing environment, client-server architecture, or other type of distributed platform, the functionality of the application can be implemented solely on the client or user device, solely on a remote server, or as a service of a cloud computing platform, or split between local components and remote components.

[0072] While it is understood that the program code 307 for methods for self-learning box resource optimization for DBMS can be deployed via a direct manual loading of the program code 307 into a client, server, and proxy computers (not shown) by loading the program code 307 into a computer readable storage medium (e.g., the computer data storage devices 311), the program code 307 can also be deployed automatically or semi-automatically by sending the program code 307 to a central server (e.g., the computer system 301) or a group of central servers. The program code 307 can then be downloaded to client computers (not shown) that will execute the program code 307.

[0073] Alternatively, the program code 307 can be sent directly to the client computer via email. The program code 307 can then be detached to a directory on the client computer, or loaded into a directory on the client computer by selecting the email option of the program that detaches the program code 307 into a directory.

[0074] Another alternative is to send the program code 307 directly to a directory on the hard drive of the client computer. If a proxy server is configured, the process selects the proxy server code, determines which computers to place the code of the proxy server on, sends the proxy server code, and then installs the proxy server code on the proxy computers. The program code 307 is then sent to the proxy server and stored on the proxy server.

[0075] In one embodiment, the program code 307 for the method of self-learning box resource optimization for DBMS coexists with the software applications (not shown), the operating system (not shown), and the network operating system software (not shown) by providing the program code 307, and then the program code 307 is installed on the clients and servers in the environment in which the program code 307 will work, the program code 307 is integrated into the client, server, and network environment.

[0076] The first step in the integration of the code included in the program code 307 described above is to identify any software on the clients and servers, including the network operating system (not shown), in which the program code 307 required to deploy the program code 307 or that works in conjunction with the program code 307. The software identified includes the network operating system, where the network operating system includes software that enhances a basic operating system by adding networking features. Next, the software applications and version numbers are identified and compared to a list of software applications and correct version numbers that have been tested to work with the program code 307. Software applications that are missing or do not match the correct version numbers are upgraded to the correct version.

[0077] The program instructions that pass parameters from the program code 307 to the software applications are examined to ensure that the list of parameters for the instructions matches the list of parameters required by the program code 307. Conversely, the parameters passed by the software applications to the program code 307 are examined to ensure that the parameters match the parameters required by the program code 307. The client and server operating systems, including the network operating system, are identified and compared to a list of operating systems, version numbers, and network software programs that have been tested to work with the program code 307. Operating systems, version numbers, or network software programs that do not match the entries in the list of tested operating systems and version numbers are upgraded to the listed level on the client computer and to the listed level on the server computer.

[0078] Integration is accomplished by installing the program code 307 on the client and server after ensuring that the software in which the program code 307 is to be deployed is at the correct version level that has been tested to work with the program code 307.

[0079] Embodiments of the invention can be implemented as a method, a computer program product, a computer system, or a process or service for supporting processor execution of computer infrastructure, executed by a processor of a computer system.

[0080] Figure 4 The architecture of the self-learning workload management component and related components of a DBMS according to embodiments of the invention is shown. Figure 4 Projects 410-470 and 4000-4001 are shown.

[0081] In Figure 4 In embodiments of the invention, the self-learning workload manager 410 component of the DBMS 4000 is trained by the machine learning training system 4001 to more accurately predict the resource requirements of incoming data access requests by organizing and providing these requests as batch workloads.

[0082] This figure shows a subset of the components of the DBMS 4000:

[0083] - an improved workload manager 410 that relies on machine learning models 460 trained by embodiments of the invention to predict the resource requirements of production workloads submitted to the DBMS 4000. In some embodiments, a unique machine learning model 460 will be used to predict the requirements for one (or another appropriate subset) of all possible types or categories of resources. In this case, each machine learning model 460 is independently trained to predict the requirements of production workloads for one or more associated resources of the particular model by repeating Figures 4-5 the method of the invention, and the total resource requirements of production workloads are derived by aggregating the requirements identified by each machine learning model 460;

[0084] - a query optimizer 420 that generates execution plans for incoming SQL queries and other types of data access requests, as is well known in the art;

[0085] - a resource prediction pipeline 430 that transfers the resource prediction rules, concepts, inferences, and other training and artificial intelligence knowledge included in the machine learning models 460 trained for resource requirement estimation by the training system 4001 to the workload manager 410; and

[0086] - a query execution engine 440 that manages the execution of each statement of a production DBMS workload assembled by the workload manager 410 by the DBMS, as is well known in the art.

[0087] Machine learning training system 4001 employs methods of artificial intelligence, cognitive computing, or machine learning to train a self-learning machine learning model 460 that predicts resource requirements of submitted DBMS workloads. Training system 4001 includes the following components:

[0088] - a machine learning training pipeline 450 that receives raw training data in the form of query logs from DBMS 4000, the query logs identifying types and amounts of resources required to service data access requests previously submitted to the DBMS;

[0089] - a machine learning model 460 trained by training system 4001 to predict resource requirements of incoming DBMS statements that have been batched into workloads; and

[0090] - an inference pipeline 470 that connects components that perform various pre-processing steps, dictionary generation, and general modeling operations (as described in subsequent figures) and transfers resulting artificial intelligence inferences, concepts, and rules from machine learning model 460 to production DBMS 4000.

[0091] Some embodiments can include a slightly different architecture than that of Figure 4 that divides the functionality of a particular component into multiple components or combines the functionality of two or more components into a single component. For example, training system 4001 can be an internal component of DBMS 4000 in some implementations, or both DBMS 4000 and training system 4001 can be components of a single larger system.

[0092] As will be discussed in greater detail in subsequent figures, embodiments of the present invention can be deployed on a platform of Figure 4 to perform steps that follow a general process:

[0093] i) in a training phase, machine learning model 460 for resource requirement estimation is trained to estimate resource requirements of various workloads processed by DBMS 4000. During this phase, training system 4001 first receives a collection of raw training data, such as logs of data access requests or other types of production workloads previously submitted to DBMS 4000 or a similar DBMS. In some embodiments, this data is received from DBMS 4000 by training pipeline 450. The raw training data also describes types and amounts of resources required by DBMS 4000 to service each recorded request.

[0094] ii) The training system 4001 processes the training data to generate a training corpus that is formatted to conform to known standards or conventions used by the machine learning application. In some embodiments, such a corpus is formatted into the triply stored data structure class categories known to be used in the machine learning training domain. Such a corpus represents the knowledge received and generated by the training system as data structure types known in the machine learning domain as a way of representing the training knowledge. The training system 4001 then uses the machine learning corpus to train the machine learning model 460 for resource requirement estimation to predict resource requirements for batch workloads.

[0095] iii) The rules, inferences, concepts, relationships, and other components of the artificial intelligence knowledge that the training machine learning model 460 comprises, pass through the inference pipeline 470 and the resource prediction pipeline 430 of the training system 4001 to the workload manager 410 component of the DBMS 4000. In some embodiments, the entire model is passed to the DBMS 4000.

[0096] iv) The improved workload manager 410 uses the latest knowledge or model received from the training system 4001 and, in response to the execution plans generated by the query optimizer 420, estimates resource requirements for production workloads assembled from data access requests submitted to the DBMS 4000. The workload manager 410 then uses this knowledge to ensure that production workloads do not require more resources to be allocated than are currently available to the DBMS 4000 and also helps to ensure that the DBMS utilizes available resources in an optimal manner. Both types of improvements contribute to maximizing the throughput and performance of the DBMS.

[0097] Figure 5 is a high-level flowchart showing a method for training a self-learning DBMS workload manager to perform box-style resource optimization and then using the results of that training to optimize the estimation of resource requirements in a production environment, according to embodiments of the present invention. Figure 5 contains steps 500-590, which are described in more detail in subsequent figures, and which can be performed by embodiments similar to those shown in Figures 1-4 the embodiments shown in FIG. 5.

[0098] In step 500, the training system 4001 receives a training data set that includes a log of SQL DBMS queries or other types of data access requests. Depending on the implementation, each request can be a single statement or can include multiple statements. In some embodiments, the data set is generated by a component of the production DBMS 4000 as, for example, a query log. In addition to including the actual text or other representation of past data access requests, the data set also identifies the types and amounts of resources that the DBMS 4000 consumed in order to service each request.

[0099] In step 510, training system 4001 pre-processes each sentence of the received training dataset. In Figure 6 This pre-processing, described in more detail in the detailed description, includes normalization and standardization operations that reduce the complexity of subsequent steps of the present invention.

[0100] In step 520, training system 4001 performs high-level feature engineering tasks on the pre-processed training sentences. In Figure 6 These tasks, described in more detail in the detailed description, produce a dictionary of keywords parsed from the training sentences, identify high-level compile-time features specified by the query execution plans generated by DBMS optimizer module 420 for each sentence, and generate a set of query vectors, each representing this information for a corresponding training sentence.

[0101] In step 530, training system 4001 organizes the query vectors into "template" clusters. Each template includes a subset of query vectors that have been deemed to have a similarity that exceeds a pre-defined similarity threshold. The implementer can choose any known similarity quantification mechanism (e.g. known clustering algorithms) to determine which query vectors have a similarity sufficient to be assigned to the same template. In Figure 6 This step is described in more detail in the detailed description, and in particular embodiments, each query vector is assigned to one and only one template.

[0102] Many clustering and similarity determination processes are known in the art, and depending on the implementer's preferences and technical constraints, embodiments can employ a particular process selected from known mechanisms, such as the k-means clustering or DBSCAN algorithms. The present invention is flexible enough to accommodate any clustering method, or any other method known in the art for quantifying similarity between n-dimensional vectors.

[0103] In one example, a particular known clustering algorithm is used to determine whether a subset of vectors is sufficiently similar to be assigned to the same template. Here, the determination includes plotting each n-element vector as a point in an n-dimensional space, and measuring the distance between that n-dimensional location and the centroid (or arithmetic mean) value of the template. The similarity of that vector to other vectors can then be derived from the magnitude of that distance. In this example, all vectors associated with a distance less than a pre-determined threshold will be deemed sufficiently similar to be assigned to the same template.

[0104] At the end of step 530, training system 4001 will have generated templates of query vectors such that the query vectors contained in each template represent a subset of training queries or sentences that share similar high-level features. In some embodiments, training system 4001 can generate an additional template that is reserved for production workloads that do not correspond to any of the previously generated templates.

[0105] In step 540, the training system 4001 generates a set of histograms, each histogram describing the distribution of vectors in a particular workload of training queries. In Figure 6 The process is described in more detail in

[0106] The "binning" process begins by randomly dividing the training queries (or data access statements) into equally sized subsets. Each subset represents a random "training workload" of DBMS queries or other types of data access requests that can be submitted to the DBMS 4000. Because each training query is represented by only one template, but each template can represent multiple training queries, each training workload is associated with a respective number of instances of each template. A training workload can be associated with zero instances of a particular template.

[0107] Each histogram includes a set of unique bins, and each bin is uniquely associated with one of the templates generated in step 530. The value stored in a particular bin of a particular histogram identifies the number of instances of the template associated with that particular bin that are associated with any queries that make up the corresponding workload of that particular histogram.

[0108] In one example, the system 4001 generates five templates in step 530: T1, T2, T3, T4, and T5. Accordingly, each histogram generated in step 540 will have five bins, one for each template. If the training workload selected in step 540 includes three queries that are respectively associated with templates T1, T5, and T1, then the five bins of the histogram for that workload will each identify the number of queries of the workload that are associated with one of the templates. In this case, the five bins of the histogram will be populated with the values: T1 = 2, T2 = 0, T3 = 0, T4 = 0, and T5 = 1.

[0109] Because each training query is known to have consumed a particular amount of resources, each histogram is also associated with its own set of resource requirements. In particular embodiments, the resource requirements of a histogram are derived as the sum of the requirements of each query of the training workload associated with that histogram. In some embodiments, each histogram or template is associated with the requirements of only one type of resource, resulting in a set of resource requirements for a particular workload being associated with a single-resource training histogram.

[0110] In step 550, the training system 4001 formats the information represented by at least some of the training workloads, histograms, resource requirements, and query vectors into a formally structured machine learning corpus. This corpus is not simply a body of purely unstructured or arbitrarily structured data. Rather, the corpus contains artificial intelligence knowledge formatted in a manner consistent with known knowledge base data structures used in the field of artificial intelligence, such as expert systems and machine learning. In representative embodiments, this knowledge is formatted as a set of vector inputs suitable for submission to a deep learning neural network, or as a triple store that mathematically represents expert resource requirement estimation knowledge as a set of rules, concepts, and relationships. The training system 4001 then uses the machine learning corpus to train the machine learning model 460 to associate specific sets of resource requirements with each workload represented by one of the histograms, by employing known machine learning methods.

[0111] In other embodiments, the training system 4000 applies additional intelligence to the data structures and knowledge developed in steps 500-550 to infer associated resource requirements for additional histograms that are not directly related to a particular training workload. For example, if the model identifies patterns and relationships between specific high-level characteristics represented by query vectors, specific types of resource requirements, or specific vector distributions associated with training workloads, then these additional histograms can be intelligently derived. The resulting additional histograms can be associated with resource requirements that are deduced or inferred from the patterns or relationships identified from the training workloads, even if the additional histograms do not precisely represent a particular training workload.

[0112] Steps 560-590 describe a phase or embodiment of the invention in which the knowledge and experience incorporated into the machine learning model 460 by the training process of steps 500-550 is applied by the production DBMS 4000. This application includes the following steps: batch processing incoming production data access requests into production workloads, and then estimating resource requirements for each production workload by associating the distribution of high-level characteristics of each production workload with the knowledge of workload resource requirements learned during the training process. Further details of steps 560-590 are described in subsequent figures.

[0113] In step 560, the DBMS 4000 receives a set of data access requests from users or from other computerized systems while operating in a production environment. The DBMS 4000 can already be servicing other requests. Depending on implementation details, the data access requests can include any kind of request known in the art, such as database queries in SQL format.

[0114] In step 570, DBMS 4000 processes all or a portion of the received set of requests to generate a query vector for each processed request. This processing is performed in a manner analogous to that of steps 510 and 520.

[0115] DBMS 4000 can use any known method preferred by the implementer to determine which requests to process. For example, if step 570 is performed whenever a particular condition is met, then DBMS 4000 can process all queries that have been received but not yet processed since the condition was last met. The condition can be set by a human administrator, manager, or programmer, or can be set and later adjusted by any known automated means. Examples of such conditions include: detecting that a particular duration of time has elapsed since step 570 was last performed; the occurrence of a certain time of day or day of week; detecting that the number of queued queries that are unprocessed has fallen below a threshold, or that the number of unqueued queries received exceeds a threshold; and detecting that the available number of a particular resource combination has exceeded a threshold.

[0116] In other embodiments, DBMS 4000 can perform step 570 on a predetermined number of queries, selecting which queries to process according to the temporal order in which the queries were submitted or received. In other embodiments, more complex selection mechanisms use a combination of other factors, such as the characteristics of each query, the characteristics of each user or system that submitted each query, or the characteristics of particular data stores accessed by each query, to determine how many and which queries to select for processing in step 570.

[0117] At the end of step 570, DBMS 4000 will have generated a query vector for each data access request selected from the subset of previously received incoming production data access requests.

[0118] In step 580, DBMS 4000 generates the optimal production workload by adding each processed query or other type of data access request to the existing workload of DBMS 4000 one at a time. As with the method used in step 570 to select which incoming data access requests to process, the order in which the processed requests are added to the workload can be selected in any manner preferred by the implementer. In some embodiments, the order is the temporal order in which the incoming requests were submitted or received.

[0119] When each query is added, DBMS 4000 uses the knowledge and experience gained by constructing machine learning model 460 during the training phase to construct a histogram representing the distribution of queries in the current workload by template. DBMS 4000 then estimates the resource requirements of the workload according to the resources associated with the newly identified histogram.

[0120] DBMS 4000 repeats this iterative process until the workload requires significantly more resources than are available. Then, the most recently added statements are removed from the workload. The resulting workload is then considered the maximum workload that DBMS 4000 can serve without requiring unavailable resources.

[0121] In step 590, the DBMS 4000 adds the selected workload to the query execution queue, where each data access request is served by a DBMS component that accesses the database's data and indexes. In some implementations, these operations are managed by the DBMS 4000's query execution engine 440 component.

[0122] In some embodiments, the results of processing the selected workload are returned to the training system 4001 via the training pipeline 450. These results may include information such as: the internal composition of the selected workload, a histogram associated with the workload, a query vector associated with each access request of the workload, or a comparison of the resource requirements specified by the histogram with the actual amount of resources required by the service for the selected workload.

[0123] Figure 6 This is a flowchart illustrating the steps for training a DBMS system using self-learning boxed resource optimization according to an embodiment of the present invention. Figure 6 Includes steps 600-680, which provide Figure 5 Details of steps 500-540.

[0124] In step 600, the DBMS training system 4001 begins preprocessing. Figure 5 The training query (or other type of training data access statement) received in step 500. One goal of these preprocessing steps is to force each training statement to conform to a set of common conventions or standards. This conformity reduces the number of unique characteristics of the elements of each statement that will be represented as a query vector. Because the complexity of the implementation increases with the size of the query vector, reducing the number of vector elements improves the efficiency of the implementation and reduces the maintenance requirements of the implementation.

[0125] This is particularly true in embodiments where the machine learning model 460 used for resource prediction is a deep neural network, where histograms and their associated aggregated resource requirements, or other data structures derived from the embodiments, are submitted as input to the deep neural network. Because the size of a deep neural network increases rapidly with the number of elements in the network's vector input, smaller input vectors can significantly reduce model complexity.

[0126] In some embodiments, the first normalization step is to ensure that all statement texts have a common case, such as all upper case or all lower case text. This step allows the training system 4001 to avoid errors in which, for example, the equivalent aliases "COL008" and "col008" are incorrectly assumed to be different tokens when the query language does not require statements to be case sensitive.

[0127] In step 610, the training system 4001 replaces predicate constants in the training statements with markers, so that the constants are removed from further analysis in subsequent steps. This step further reduces the complexity of the analysis, because the exact value of a constant has little effect on the amount or type of resources required to process the statement. For example, replacing the constant "21" in the clause "CustomerAge < 21" with a marker prevents the exact value of the constant from being incorrectly identified as a token element that can change the amount or type of resources required to execute the clause.

[0128] In step 620, the training system 4001 continues to pre-process the training statements by removing other tags from each statement that do not affect the amount or type of resources required to execute the statement. These other tags include pattern references and table aliases.

[0129] For example, because the amount or type of resources required to execute the query "SELECT * FROM CustomerTable CI WHERE CI.MyColumn > 100" is almost identical to the amount or type of resources required to execute the query "SELECT * FROM CustomerTable C2 WHERE C2.MyColumn > 100", the training system 4001 does not attempt to distinguish between the table aliases "CI" and "C2" when determining the resource requirements of these statements.

[0130] In step 630, the training statements pre-processed in steps 600-620 are tokenized in a manner similar to the tokenization steps used by known DBMS front-ends. This tokenization includes the steps of extracting syntactic keywords (such as the SQL predicate "JOIN") and semantic keywords (such as literal table names) and removing stop words such as the SQL keywords "FROM" and "AND". As previously described, keywords are identified because keywords identify operations that consume significant resources, but stop words are discarded because stop words do not cause resource utilization.

[0131] At the end of step 630, the training system 4001 has accumulated a list of tokens that have been extracted from the training statements after the statements have been standardized. This standardization can reduce the number of extracted tokens by removing elements from the statements that do not significantly affect the resource requirements, and resolve ambiguities and redundancies that can cause unnecessary additional tokens to be incorrectly identified.

[0132] In step 640, the training system 4001 constructs a data dictionary that lists the number of occurrences of each unique token extracted from the training statements in step 630.

[0133] In step 650, the training system 4001 removes from the data dictionary any tokens that have not occurred at least a minimum number of times. This determination is performed by comparing the number of occurrences of each token to a predetermined minimum threshold. This threshold can be identified by any method preferred by the implementer, including known methods based on empirical observations of how various threshold levels affect the performance of the method, or by any method known to DBMSs for generating data dictionaries during normal operation. Figure 5

[0134] In step 660, the training system 4001 reviews the execution plans generated by the query optimizer 420 (or a module providing similar functionality) as the training statements are processed. In some embodiments, these execution plans are included in the training data set received in step 500. In other embodiments, the modules of the DBMS 4000 or the training system 4001 generate the execution plans by known methods similar to those performed by the DBMS query optimizer 420. The resulting execution plans, and the detailed steps by which the plans are generated so as to indicate how the DBMS most efficiently processes data access requests, are known in the art of database management.

[0135] The training system 4001 continues in this step by next extracting from the execution plans of each training statement the compile-time features. These high-level features (such as the tokens extracted from the training statements in step 630) are known in the art to be capable of causing a significant amount of resource consumption during the execution of the statements that include these features.

[0136] As is known in the art, examples of such high-level compile-time features include operations to be performed when executing the query and estimated metrics such as table scans or sort operations and cardinality estimates (e.g., estimating the number of rows of a database table that must be traversed in order to perform an operation such as searching a table or joining a column of a table with another column).

[0137] Other examples of high-level features that can be identified by various types of embodiments as significantly affecting the resource requirements of a statement include:

[0138] - the maximum size of the sort heap available to the DBMS,

[0139] - the number of aggregation operators (such as MAX(), AVG(), and COUNT()) in the SQL query,

[0140] ​- the number of aggregation operators in SQL queries that include the SQL DISTINCT keyword,

[0141] - the number of rows in tables that must be scanned before or after operations can be performed, and

[0142] - the number of threads available to the DBMS that are configured to multithread execution of concurrent statements.

[0143] In step 670, the trained clustering model of training system 4001 assembles a query vector for each statement. Each element of each query vector corresponds uniquely to one of the high-level features identified in steps 610-650, such as a unique token parsed from a training statement or a compile-time operation or metric referenced by a query execution plan generated by a DBMS query optimizer component. Each element of a query vector associated with a particular data access statement associates the particular data access statement with a value of the high-level feature corresponding to the element.

[0144] At the end of step 670, each training statement will have been associated with a corresponding query vector that identifies the high-level features (e.g., tokens and compile-time features) characterizing that statement. The vector does not identify low-level features associated with the corresponding training statement. More than one training statement can be associated with the same template, but no statement can be associated with more than one template.

[0145] In step 680, the clustering model of training system 4001 organizes the query vectors into clusters called templates, divides the training dataset into workload subsets, and associates each workload with a histogram that identifies the distribution of query vectors among the training statements of that workload.

[0146] In this step, as previously described in step 540 of Figure 5 step 540 of

[0147] The number of statements or queries in each training workload can also be selected by any means known in the art or preferred by the implementer. For example, in some embodiments, the size of the training workloads is selected to optimize the performance of the method of Figure 6

[0148] ​The trained machine learning model 460 of the training system 4001 then characterizes each workload with a corresponding histogram. Each bin of the histogram identifies the number of queries or statements in the corresponding workload that are associated with a particular template. Because more than one query or statement can be associated with the same template, each bin can contain a zero or positive value. Each histogram is also associated with a set of resource requirements derived by adding the amount of resources actually consumed by each training statement or query of the respective workload of the histogram when these statements or queries were originally executed by the DBMS 4000.

[0149] As Figure 5 described in step 550, an artificial intelligence machine learning model 460, such as a deep neural network, a deep learning neural network, or another type of machine learning model or expert system, is trained using the resulting histograms, query vectors, and resource consumption records to predict resource requirements for a particular type of workload, Figure 6 The method ends.

[0150] Figure 7 is a flowchart showing steps of a DBMS system with self-learning bin-based resource optimization according to an embodiment of the present invention. Figure 6 includes steps 600-680, which provide details of Figure 5 steps 500-540 of the method of Figure 7 includes steps 700-780, which provide details of Figure 5 steps 570-590 of the method of

[0151] Figure 7 shows how the improved workload manager 410 component of the DBMS 4000 uses the machine learning model 460 trained as described in the method of Figure 6 and Figure 5 steps 500-550 of the method ofThe method includes the following steps:

[0152] - dividing the incoming production queries into a "production workload" subset,

[0153] - associating each production workload with a matching histogram, where the matching histogram is constructed in a similar way to the training histograms derived by the method of Figure 5 and 6

[0154] - using the knowledge obtained by the machine learning model 460 during the training phase to infer the resource requirements of each production workload from the characteristics of the matching histogram.

[0155] In Figure 5Step 700, which is executed immediately after step 560, the DBMS 4000 begins processing the set of incoming production queries (or data access statements) received in step 560. This processing is similar to Figure 5 Steps 510-520 of the training system 4001 and Figure 6 Steps 610-670, as performed by the training system 4001 on the training data sets. At the end of step 700, the DBMS 400 will have generated production query vectors for at least a subset of the production queries or statements received in step 560.

[0156] In step 710, the DBMS 400 associates each production query vector generated in step 700 with a template that is similar in form and function to the templates derived by the training system 4001 during the training phase of Figures 5-6 This association can be performed by a clustering model developed and trained during the training phase, such as a model based on the k-means clustering algorithm.

[0157] In some embodiments, the process includes a direct step of selecting the templates that contain vectors that match each of the production query vectors derived in step 700. If any of the generated vectors cannot be matched to a template, some embodiments can use a similarity quantification algorithm to derive a template that identifies the closest matching vector or that identifies vectors that are less similar to the generated vector than a predetermined tolerance level. Other embodiments can simply assign the generated vectors that do not have a matching template to a reserved "catch-all" template, such as the template generated at the completion of step 530 of Figure 5

[0158] In step 720, the DBMS 4000 assembles an initial production workload that contains a subset of the production queries processed in steps 700-720. If the DBMS 4000 has not yet completed executing all previously queued queries, the initial workload can include one or more pending queries or statements currently in the execution queue, as a supplement or alternative to the subset of production queries, if the implementer prefers. In some embodiments, if the DBMS 4000 is not currently executing any production queries, the initial workload is set to a single query (or statement, depending on implementation details) selected from the production queries processed in steps 700-720.

[0159] In step 730, the DBMS 4000 derives a histogram that identifies the distribution of templates associated with the initial workload. This step is performed by a process similar to the process used by the training system 4001 in step 540 to generate the training histograms for each training workload. Figure 5

[0160] ​​Step 740 begins the iterative process of steps 740-770. This process is repeated until it is determined that the current candidate for the production workload requires more resources than are currently available to DBMS 4000.

[0161] In step 750, workload manager 410 (or another functional component of DBMS 4000) determines whether the current workload requires more resources than are currently available to DBMS 4000. This determination is made by comparing the resource requirements associated with the most recently selected or derived histogram to the types and amounts of resources currently available to DBMS 4000.

[0162] If the system determines that the current workload does not require more resources than are available, then Figure 7 The method of FIG. 7 continues with step 770, which adds the next production query or statement to the current workload. As previously described, this next query or statement can be selected by whatever means the implementer desires, such as by selecting according to the chronological order in which production queries or statements are submitted or received by DBMS 4000.

[0163] DBMS 4000 repeats the preceding steps to associate this updated workload with histograms that represent the distribution of templates associated with the production statements contained in the updated workload. In embodiments in which each template and histogram is associated with only a single type of resource or only a proper subset of resource types, the updated workload in this step will be associated with a set of histograms that collectively represent the distribution of templates associated with all types of aggregated resources required to execute the updated workload. The newly derived histogram(s) are derived by machine learning model 460 based on the prior training so as to associate the updated workload with a particular set of resource requirements that will be compared to the amount and types of resources currently available to DBMS 4000 during the next iteration of step 750.

[0164] The iterative process of steps 740-770 is then repeated, and the process continues until the production workload requires more resources than are currently available to DBMS 4000. At this point, conditional step 750 directs the system to step 760, in which the most recently added production statement or query is removed from the current workload. The final production workload is then selected as the largest workload that DBMS 4000 with the available resources is capable of executing. The iterative process of steps 740-770 then ends, Figure 7 The method of FIG. 7 continues to step 780.

[0165] In step 780, DBMS 4000 executes the final production workload that was exported in step 760. This execution can be performed by means known in the art of database management. However, in particular embodiments, this execution process can be improved by eliminating the step of checking the availability of resources needed to execute the final production workload. These steps can be eliminated by the implementer at their discretion, as it is known that the present invention has already ensured that only available resources will be needed to execute all queries, data access requests, or other types of statements contained in the workload.

[0166] If any production statements processed in steps 700-710 have not yet been executed, then Figure 7 Steps 720-780 of the method of FIG. 7 are repeated with the next production workload selected from the remaining statements. As with the previous execution of these steps, DBMS 4000 generates a next candidate initial production workload, associates the candidate with one or more newly exported histograms (the histograms exported by trained machine learning model 460 to be similar in form and function to the histograms generated by the method of Figures 5-6 FIG. 6), and continues to add statements to the workload until the resources needed by the workload exceed the resources available to DBMS 4000.

[0167] For the purposes of illustration, examples and embodiments of the present invention described in this document have been presented. They should not be interpreted as exhaustive nor should the embodiments of the present invention be limited to the examples and embodiments described herein. Many other modifications and variations of the present invention will be apparent to those skilled in the art without departing from the scope of these examples and embodiments. The terminology used in this document has been chosen to best explain the basic principles of these examples and embodiments in order to illustrate the practical application and technical improvements of the present invention over known technology and products, and to enable those skilled in the art to better understand the examples and embodiments disclosed herein.

Claims

1. A training subsystem of a database management system (DBMS), comprising a processor, a memory coupled to the processor, and a computer-readable hardware storage device coupled to the processor, the storage device containing program code configured to be executed by the processor via the memory to implement a method for self-learning bin-wise resource optimization of a database management system, the method comprising: the training subsystem receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and a record of types and amounts of resources required by the DBMS to execute each statement in the set of training statements; the subsystem generating a set of query vectors, each query vector identifying high-level characteristics of a corresponding statement in the training statements; the subsystem assigning the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one of the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity to each other that exceeds a threshold similarity; the subsystem dividing the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one of the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem associating each corresponding histogram with a set of resources past consumed by execution of each training statement of the corresponding training workload of that histogram; the subsystem adding knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem training a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

2. The subsystem of claim 1, wherein, each high-level characteristic of a first statement in the training statements is selected from the group consisting of: unique tokens in a set of candidate tokens that can be parsed from the first statement; and unique compile-time features identified by an execution plan of the first statement generated by a DBMS optimizer component.

3. The subsystem of claim 2, further comprising: identifying each occurrence of a first token in the set of candidate tokens in the first statement by pre-processing the first statement, wherein the pre-processing comprises: converting all text of the first statement to upper or lower case, replacing all constants in a predicate of the first statement with a parameter token, removing all pattern references and table aliases from text of the first statement, removing all stop words from text of the first statement, and parsing a set of tokens from the first statement, the set of tokens comprising all syntactic and semantic keywords contained in the remaining text of the first statement.

4. The subsystem of claim 3, further comprising: building a data dictionary that identifies a number of occurrences of each token parsed from the remaining text of any of the training statements; removing from the data dictionary any tokens associated with a number of occurrences that is below a predefined threshold; and defining the set of candidate tokens as equivalent to all tokens remaining in the data dictionary after the removal.

5. The subsystem of claim 1, wherein, training statements are randomly assigned to each training workload, wherein no training statement is assigned to more than one training workload, and wherein each training workload contains an equal number of training statements.

6. The subsystem of claim 1, further comprising: generating an empty template that does not correspond to any of the set of training statements.

7. The subsystem of claim 1, wherein, the first vector is assigned to the first template only if a clustering algorithm determines that a distance between a centroid point of the first template in the set of templates and a point located at coordinates specified by all elements of the first vector in the set of query vectors is smaller than a predefined threshold.

8. The subsystem of claim 7, wherein, the clustering algorithm produces a clustering model that is capable of intelligently clustering query vectors into templates, and wherein the clustering algorithm is selected from a group consisting of: a k-means clustering algorithm, an elbow method, and a DBSCAN algorithm.

9. The subsystem of claim 1, wherein, each statement in the set of training statements is a data access request that has been previously processed by the DBMS.

10. The subsystem of claim 1, wherein the cognitive model is a neural network, wherein the training further comprises training the neural network to predict a type and a number of production resources required by the DBMS to execute a production workload, wherein the production workload comprises a batch of production data access requests, and wherein the neural network predicts the type and the number of production resources by drawing inferences from a histogram derived by the neural network, the histogram relating the production workload to a distribution of production workload templates in the set of templates, and relating the production workload to aggregated resource requirements associated with production workload templates of the histogram.

11. The subsystem of claim 1, wherein the cognitive model is a deep learning neural network, wherein the training corpus is a formally structured machine learning corpus, the machine learning corpus comprising a triple store knowledge base that represents knowledge as a collection of concepts, rules, and relationships formatted into a triple store data structure, and wherein the knowledge comprises information selected from a group consisting of: each statement in the set of training statements, each vector in the set of query vectors, each template in the set of templates, each corresponding histogram, and each set of consumed resources associated with a corresponding histogram.

12. A method for self-learning chest resource optimization of a database management system, the method comprising: a training subsystem of a database management system DBMS receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and the DBMS to record a type and a quantity of resources required for execution of each statement in the set of training statements; the subsystem to generate a set of query vectors, each query vector identifying a high-level characteristic of a corresponding statement in the training statements; the subsystem to assign the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one template in the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity between them that exceeds a threshold similarity; the subsystem to divide the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one template in the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem to associate each corresponding histogram with a set of resources consumed in the past by execution of each training statement of the corresponding training workload of that histogram; the subsystem to add knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem to train a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

13. The method of claim 12, wherein, each high-level characteristic of a first statement in the training statements is selected from a group consisting of: unique tokens in a set of candidate tokens that can be parsed from the first statement; and unique compile-time features in a set of compile-time features of the first statement that are identified by an execution plan generated by a DBMS optimizer component.

14. The method of claim 12, wherein training statements are randomly assigned to each training workload, wherein no training statement is assigned to more than one training workload, and wherein each training workload contains an equal number of training statements.

15. The method of claim 12, further comprising: at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer readable program code in a computer system, wherein the computer readable program code in combination with the computer system is configured to implement the receiving, the generating, the assigning, the dividing, the associating, the adding, and the training.

16. A computer program product for self-learning bin-wise resource optimization of a database management system (DBMS), the computer program product comprising program instructions executable by a processor to cause the processor to perform a method for self-learning bin-wise resource optimization of a database management system, the method comprising: a training subsystem of a database management system (DBMS) receiving training data, the training data comprising: a set of training statements, each training statement requesting access to a database managed by the DBMS, and the DBMS to record a type and a quantity of resources required for execution of each statement in the set of training statements; a set of training statements, each training statement requesting access to a database managed by the DBMS, and the DBMS maintaining a record of types and amounts of resources required by the DBMS to execute each statement in the set of training statements; the subsystem generating a set of query vectors, each query vector identifying high-level characteristics of a corresponding statement in the set of training statements; the subsystem assigning the set of query vectors to a set of templates, wherein each vector in the set of query vectors is assigned to one template in the set of templates, and wherein each template in the set of templates is populated with query vectors that have a mathematical similarity to each other that exceeds a threshold similarity; the subsystem dividing the set of training statements into a set of training workloads, wherein each workload in the set of training workloads is associated with a corresponding histogram, wherein each bin of each corresponding histogram uniquely corresponds to one template in the set of templates, and wherein each bin of a first corresponding histogram identifies how many statements of the corresponding workload of the first corresponding histogram are associated with each template in the set of templates; the subsystem associating each corresponding histogram with a set of resources consumed in the past by execution of each training statement of the corresponding training workload of that histogram; the subsystem adding knowledge inferred from the training data to a training corpus that can be used to train a machine learning model; and the subsystem training a cognitive model of a workload management component of the DBMS to predict resource requirements of batch data access requests, wherein the training comprises submitting the training corpus to the cognitive model.

17. The computer program product of claim 16, wherein, each high-level characteristic of a first statement in the set of training statements is selected from the group consisting of: unique tokens in a set of candidate tokens that can be parsed from the first statement; and unique compile-time features identified by an execution plan generated for the first statement by a DBMS optimizer component.

18. The computer program product of claim 16, wherein training statements are randomly assigned to each training workload, wherein no training statement is assigned to more than one training workload, and wherein each training workload contains an equal number of training statements.

19. A database management system (DBMS) comprising a processor, a memory coupled to the processor, and a computer-readable hardware storage device coupled to the processor, the storage device containing program code configured to be executed by the processor via the memory to implement a method for self-learning bin-wise resource optimization for a database management system, the method comprising: the DBMS receiving a set of production statements, each production statement requesting access to a database managed by the DBMS, and a cognitive model trained to predict resource requirements of batch data access requests; the DBMS generating a set of query vectors, wherein each vector in the set of query vectors identifies high-level characteristics of a corresponding statement in the set of production statements, and wherein each element of a particular vector in the set of query vectors identifies a unique high-level characteristic of a corresponding statement in the set of production statements; the DBMS assigns each vector in the set of query vectors to a corresponding template in a set of model templates maintained by a clustering model of the cognitive model; the DBMS iteratively generates a final production workload, wherein each iteration comprises the steps of: the DBMS adds a next statement selected from the set of production statements to a candidate production workload, wherein the next statement was not previously added to the candidate production workload, the DBMS attempts to associate the candidate production workload with a candidate histogram, wherein each bin of the candidate histogram identifies a total number of statements of the candidate production workload that are associated with templates in the set of model templates that are uniquely assigned to that bin; the DBMS, if the candidate production workload can be associated with the candidate histogram, predicts, in accordance with the training of the cognitive model, a number and type of resources that will be required to execute the candidate production workload to be equal to those resources identified by the cognitive model, in accordance with the training of the cognitive model, as resulting from executing any workload characterized by a matching histogram, the DBMS, if it determines that executing the candidate production workload will require more resources than are currently available to the DBMS, removes the most recently added statement from the candidate production workload and considers the resulting workload to be the final production workload, and the DBMS, if it determines that executing the candidate production workload will not require more resources than are currently available, continues with a next iteration of generating a final production workload; and the DBMS forwards the final production workload to a query execution engine of the DBMS for execution by the DBMS.

20. The DBMS of claim 19, further comprising: the DBMS, if it is unable to derive a matching histogram, identifying the candidate production workload, the most recently added statement, and the candidate histogram to the cognitive model, removing the most recently added statement from the candidate production workload, and continuing with a next iteration of generating a final production workload.

21. The DBMS of claim 19, wherein each template in the set of model templates identifies a corresponding cluster of query vectors, and wherein a first vector is assigned to a particular template in the set of model templates if a size of a distance between: i) a centroid point of all vectors identified by the particular template, and ii) a point located at coordinates specified by the first vector, 22. The DBMS of claim 19, wherein, is not greater than a predefined threshold. each high-level characteristic of a first statement in the production statements is selected from the group consisting of: a unique token that is resolvable from the first statement; and a unique compile-time feature identified by an execution plan of the first statement generated by an optimizer component of the DBMS.

23. The DBMS of claim 19, wherein, The candidate production workload is initialized to an initial workload declaration set comprising at least one pending data access request queued for execution by the DBMS prior to generating a first iteration of a final production workload.

24. A method for a database management system (DBMS), comprising: The database management system (DBMS) receives a set of production statements and a cognitive model, wherein each production statement requests access to a database managed by the DBMS, and the cognitive model is trained to predict resource requirements for batch data access requests; The DBMS generates a set of query vectors, wherein each vector in the set of query vectors identifies high-level characteristics of a corresponding statement in the set of production statements, and wherein each element of a particular vector in the set of query vectors identifies a unique high-level characteristic of a corresponding statement in the set of production statements; The DBMS assigns each vector in the set of query vectors to a corresponding template in a set of model templates maintained by a clustering model of the cognitive model; The DBMS iteratively generates a final production workload, wherein each iteration comprises the steps of: The DBMS adds a next statement selected from the set of production statements to a candidate production workload, wherein the next statement was not previously added to the candidate production workload, The DBMS attempts to associate the candidate production workload with a candidate histogram, wherein each bin of the candidate histogram identifies a total number of statements of the candidate production workload that are associated with templates in the set of model templates that are uniquely assigned to that bin; The DBMS, if able to associate the candidate production workload with the candidate histogram, predicts a quantity and type of resources that will be required to execute the candidate production workload to be equal to those identified by the cognitive model as resulting from executing any workload characterized by the matching histogram according to the training of the cognitive model, The DBMS, if able to derive a matching histogram, predicts a quantity and type of resources that will be required to execute the candidate production workload to be equal to those identified by the cognitive model as resulting from executing any workload characterized by the matching histogram according to the training of the cognitive model, The DBMS, if it determines that executing the candidate production workload will require more resources than are currently available to the DBMS, removes a most recently added statement from the candidate production workload and considers the resulting workload to be a final production workload, and The DBMS, if it determines that executing the candidate production workload will not require more resources than are currently available, continues generating a next iteration of a final production workload; and The DBMS forwards the final production workload to a query execution engine of the DBMS for execution by the DBMS.

25. The method of claim 24, wherein each template in the set of model templates identifies a corresponding template comprising a cluster of query vectors, and wherein the first vector is assigned to a particular template in the set of model templates if the magnitude of the distance between: i) the centroid point of all vectors identified by the particular template, and ii) the point located at the coordinates specified by the first vector, is not greater than a predefined threshold.

Images