Predicate evaluation based on CPU utilization of shared storage device

Abstract

The invention provides predicate evaluation based on CPU utilization of a shared storage device. A hybrid transactional / analytic processing (HTAP) database includes an online transactional processing (OLTP) engine, an online analytic processing (OLAP) engine provided separately from the OLTP engine, and a shared storage that holds data for a plurality of tenants. The shared storage device is configured to receive a request from the OLAP engine for predicate evaluation for data stored on the shared storage device. The shared storage device is further configured to determine that the CPU utilization of the shared storage device exceeds a predicate rejection threshold. Upon determining that the CPU utilization of the shared storage device exceeds a predicate rejection threshold, the shared storage device is further configured to inhibit evaluation of at least a portion of the predicate.

Description

BACKGROUND

[0001] Databases are designed with various types of architectures. Online transaction processing (OLTP) architectures are oriented towards processing transactions. Like a bank account, transactions tend to be simple and numerous, and can include queries such as reads, inserts, updates, deletes, and the like. However, OLTP architectures lack massively parallel processing (MPP) capabilities. In contrast, online analytical processing (OLAP) architectures are oriented towards processing data analytics. Queries processed by OLAP architectures tend to be more complex, involve more data, and take more time to complete. For example, an OLAP architecture can be responsible for determining the average sales price of a particular product over the past year tracked by the database. Many database users use both types of processing on the same data, and thus a hybrid transaction / analytical processing (HTAP) database has been developed that includes both types of architectures in a single database.

[0002] Predicate pushdown can be an optimization technique to improve query performance in database systems, including HTAP databases. Predicate pushdown is a strategic approach that enables databases to work more intelligently by filtering data at the data source. By doing so, it reduces the amount of data transferred and processed during query execution. Specifically, predicate pushdown focuses on optimizing the filtering aspect of a query. It filters the data source before the data is fetched, thus improving query execution efficiency.

[0003] However, in the area of cloud-native deployments, storage devices are often shared by multiple tenants. This presents a challenge for performance-oriented optimizations such as predicate pushdown. When multiple tenants use the same storage node, high CPU utilization caused by predicate pushdown can impact all simultaneously incoming read and write requests. For example, if the CPU utilization of the storage node remains at 100%, it can result in frequent request timeouts. Therefore, in this scenario, it becomes critical to strike a balance between the optimization benefits of predicate pushdown and potential resource contention. SUMMARY

[0004] To address these issues, provided herein is a hybrid transactional / analytical processing (HTAP) database that includes an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage that holds data for a plurality of tenants. The shared storage can be configured to receive a request from the OLAP engine for predicate evaluation on data stored on the shared storage. The shared storage can be further configured to determine that a CPU utilization of the shared storage exceeds a predicate rejection threshold. The shared storage can be further configured to, upon determining that the CPU utilization of the shared storage exceeds the predicate rejection threshold, disallow evaluation of at least a portion of the predicate. The shared storage can be further configured to return a response to the request, the response including a partial or complete unprocessed result for the predicate evaluation from the data, and to append partial or complete predicate rejection metadata to the response to indicate that the partial or complete unprocessed result has not been evaluated.

[0005] This Summary is provided to introduce some concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all of the disadvantages noted in any part of this disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 A hybrid transactional / analytical processing (HTAP) database according to one example of the present disclosure is shown.

[0007] Figure 2 A predicate operation of a shared storage of the HTAP database is shown. Figure 1

[0008] Figure 3A Experimental results of a query workload without CPU utilization based predicate evaluation are shown.

[0009] Figure 3B Experimental results of a query workload with CPU utilization based predicate evaluation according to the present disclosure are shown.

[0010] Figure 4 An example flowchart of a method of evaluating a predicate at a shared storage in a HTAP database according to one example of the present disclosure is shown.

[0011] Figure 5 An illustrative view of an example computing environment in which the HTAP database of Figure 1 may be implemented is shown. ​DETAILED DESCRIPTION

[0012] Figure 1 A hybrid transaction / analytical processing (HTAP) database 10 is shown in accordance with one example of the present disclosure. The HTAP database 10 can be hybrid, in that it includes both an online transaction processing (OLTP) engine 12 and an online analytical processing (OLAP) engine 14 that can be provided separately from the OLTP engine 12. That is, the HTAP database 10 can include separate computing engines. Keeping the two engines 12, 14 separate can allow each individual engine 12, 14 to perform at its highest performance for respective workloads that it is specifically designed to handle, while avoiding interference between them. Examples of HTAP databases with unified engines include SAP HANA and MEMSQL, while examples of HTAP databases with separate engines include WILDFIRE and TIDB.

[0013] The OLTP engine 12 can include read-write (RW) nodes 16 and read-only (RO) nodes 18. Each node 16, 18 can include a respective auxiliary engine plugin 20, which is an extension that allows a proxy 22 to send all queries from a client 24 to the OLTP engine 12, and allows the OLTP engine 12 to determine which engine 12, 14 to use to process a given query based on various parameters, such as query complexity. For example, data manipulation language (DML) queries and data definition language (DDL) queries can be sent only to the RW nodes 16, while other types of queries can be handled by either the nodes 16 or the nodes 18. By directing queries in this way rather than at the proxy 22, the database 10 can support a read-committed isolation level across both engines 12, 14, guaranteeing the same results from either engine. To reduce latency, a user can choose to send a query directly to the OLAP engine 14 when the final destination is known from the start.

[0014] In contrast to separate engines 12, 14, the HTAP database 10 can include a shared storage 26 that holds data for multiple tenants. The shared storage 26 can be shared, in that it includes both an OLTP storage 28 that holds data in row format to respond to queries of the OLTP engine 12, and an OLAP storage 30 that holds data in column format to respond to queries of the OLAP engine 14. Typically, systems with separate computing engines also include separate storage, and the data freshness of such storage is typically lower for OLAP queries.

[0015] The HTAP database 10 can be configured to support a unified application programming interface (API) and use the broker 22 to automatically route queries from the clients 24 to the OLTP engine 12 and the OLAP engine 14. At a basic level, DML queries, DDL queries, simple queries, and other OLTP- appropriate queries (e.g., queries with predicates on indexes on OLTP tables) are included in the queries 32 that are routed to the OLTP engine 12, while complex queries (injecting queries with multiple joins and aggregations) are included in the queries 34 that are routed to the OLAP engine 14. In this way, interference between the workloads of OLTP and OLAP can be avoided while queries are executed by the appropriate engine 12, 14.

[0016] The OLTP storage 28 can include a log storage 36 configured to persist the logs 38 and a page storage 40 configured to store versions of data pages and apply the logs 38 to construct current versions of the data pages. The logs 38 can include various types of logs, such as redo logs, logical logs, and write-ahead logs (WALs). The page storage 40 can use redo logs to construct data pages, while logical logs for committed DML transactions, such as MYSQL binary logs, can be passed to the OLAP storage 30. The OLAP storage 30 can include a delta storage 42 configured to record changes made to the OLTP storage 28 and a base storage 44 configured to implement updates from the delta storage 42 in persistent storage. Although the OLAP storage 30 can be columnar in nature, it should be understood that the delta storage 42 can store changes in row format until they are implemented in the base storage 44 in column format. The delta storage 42 can reside in memory, while the base storage 44 can reside on disk.

[0017] To ensure data freshness of the base storage 44 for OLAP queries, the OLAP storage 30 can further include a replication framework 46 configured to perform log shipping of the logs 38 from the log storage 36 to the delta storage 42. In particular, the replication framework 46 can ship logical logs to the OLAP storage 30. The replication framework 46 can distribute the logs 38 to multiple storage nodes for each partition to construct a columnar data storage that resides on different storage nodes than the row storage to which it corresponds.

[0018] The OLAP engine 14 can include multiple coordinators 48 for distributed plan generation and optimization and multiple data servers 50 for distributed plan execution. A centralized cluster manager 52 can allow users to create resource groups from the data servers 50 and can assign data servers 50 from a default resource group 54 to new resource groups 56 as needed by the workload. To process the query 34 at the OLAP engine 14, the auxiliary engine plugin 20 can generate a MYSQL compatible query plan and perform simple optimizations before sending the partially optimized query plan along with the required metadata to the coordinators 48 for distributed plan generation and optimization. After optimization, a query scheduler located in the same location can send pieces of the plan to a group of data servers 50 for execution, which can read data from the OLAP storage 30 and periodically send heartbeats to the cluster manager 52. The final result of the query can be buffered at one of the data servers 50 and fetched by the auxiliary engine plugin 20.

[0019] At a basic level, the OLTP engine 12, the OLAP engine 14, and the shared storage 26 can be composed of multiple servers working together to form a compute layer (e.g., the OLTP engine 12 and the OLAP engine 14) and a storage layer (e.g., the shared storage 26). In one example, the HTAP database 10 includes multiple clusters managed by the cluster manager 52, and each cluster includes six database servers, with four data servers 50 forming the OLAP engine 14 of the cluster and one RW node 16 and one RO node forming the OLTP engine 12 of the cluster. The shared storage 26 can include multiple storage servers, e.g., three per cluster. It should be understood that this is just one example, and other suitable numbers of machines can be used.

[0020] Figure 2 It is shown Figure 1The predicate operation of the shared storage device 26 of the HTAP database 10. The shared storage device 26 consists of a storage node (computing device) 100 including a CPU 110A, memory 112, and mass storage device 114. The mass storage device 114 stores data 78 of multiple tenants. The shared storage device 26 further includes a predicate evaluator 84A that performs predicate evaluation. Unwanted data is filtered by performing predicates. The shared storage device 26 is configured to receive a request 80 from the OLAP engine 14 to perform predicate evaluation on the data 78 stored on the shared storage device 26. This request (80) corresponds to "predicate pushdown", that is, moving the filtering process closer to the shared storage device 26, the data source. Upon receiving the request 80, the shared storage device 26 is configured to determine, via a CPU utilization monitor 82, that the CPU utilization 104 of the shared storage device 26 exceeds the predicate rejection threshold 86. The CPU utilization monitor 82 monitors the CPU utilization 104 of the target storage node 100 in the shared storage device 26. Figure 2 As shown, CPU utilization 104 ranges from 0% (minimum utilization) to 100% (maximum utilization). The predicate rejection threshold 86 is set between 0% and 100%, such as 70%. High CPU utilization (e.g., 90-100%) often occurs due to many queries running concurrently, especially when these queries involve computationally intensive tasks, such as applying runtime filters to columns lacking indexes.

[0021] Continue to refer to Figure 2 When it is determined that the CPU utilization 104 of shared storage device 26 exceeds the predicate rejection threshold 86, shared storage device 26 is configured to prevent predicate evaluator 84A from evaluating at least a portion of the predicate. Shared storage device 26 is further configured to return a response 90 to request 80, including partially or completely unprocessed results 92, 94 from data 78 for predicate evaluation, and appending partially or completely unprocessed predicate rejection metadata 96, 98 to response 90 to indicate that the partially or completely unprocessed results 92, 94 have not yet been evaluated. Thus, for example, when the predicate rejection threshold 86 is set to 70%, once CPU utilization 104 reaches 70%, shared storage device 26 partially or completely disables predicate evaluation (runtime filter). Subsequently, when CPU utilization 104 drops below 70%, the system re-enables predicate evaluation for subsequently arriving data. This configuration allows the system to be controlled at the desired CPU utilization level, preventing CPU utilization from spikeing to or approaching 100% and overwhelming the storage engine.

[0022] Response 90 is returned to OLAP engine 14, which made request 80. The OLAP engine is configured to receive response 90, which includes partial or complete predicate rejection metadata 96, 98 and partial or complete unprocessed results 92, 94 from data 78 for predicate evaluation, and to determine, based on the partial or complete predicate rejection metadata 96, 98, that at least a portion of the predicate has not yet been evaluated. The OLAP engine is further configured to evaluate the predicate using the partial or complete unprocessed results 92, 94 received in response 90 via predicate evaluator 84B. The partial or complete predicate rejection metadata 96, 98 can be linked to each batch of returned data to indicate that predicate evaluation should be performed at OLAP engine 14 (i.e., the compute node (server) executing the OLAP engine). This ensures that OLAP engine 14 (or the compute node or server executing OLAP engine 14) evaluates the predicate as early as possible to provide the correct final result.

[0023] The following discussion details the operation and scenarios / scopes (A), (B), and (C) for generating a response of 90 based on a CPU utilization of 104 at shared storage device 26.

[0024] In scenario / scope (A), the predicate rejection threshold 86 is a partial predicate rejection threshold, and shared storage device 26 further determines a full predicate rejection threshold 88 that is higher than the partial predicate rejection threshold 86. The full predicate rejection threshold 88 can be closer to 100%. Shared storage device 26 is further configured to determine that the CPU utilization 104 of shared storage device 26 exceeds the partial predicate rejection threshold 86 but does not reach the full predicate rejection threshold 88, wherein the CPU utilization 104 falls within the range of (A). When it is determined that the CPU utilization 104 of shared storage device 26 exceeds the partial predicate rejection threshold 86 but does not reach the full predicate rejection threshold 88, shared storage device 26 is configured to prevent predicate evaluator 84A from evaluating a portion of the predicate. Shared storage device 26 is further configured to return a response 90 to request 80, the response 90 including a partially unprocessed result 92 from data 78 for predicate evaluation, and shared storage device 26 is further configured to append partial predicate rejection metadata 96 to the response 90 to indicate that a portion of the predicate has not yet been evaluated. Response 90 is returned to OLAP engine 14, and OLAP engine 14 is configured to receive response 90, which includes partial predicate rejection metadata 96 and partial unprocessed results 92 from data 78 for predicate evaluation. OLAP engine 14 is further configured to determine, based on the partial predicate rejection metadata 96, that a portion of the predicate has not yet been evaluated, and to evaluate the predicate using the partial unprocessed results 92 received in response 90 via predicate evaluator 84B.

[0025] In scenario / scope (B), shared storage device 26 is further configured to determine that the CPU utilization 104 of shared storage device 26 reaches a full predicate rejection threshold 88, wherein the CPU utilization 104 falls within the range of (B). Upon determining that the CPU utilization 104 of shared storage device 26 reaches the full predicate rejection threshold 88, shared storage device 26 is configured to prevent predicate evaluator 84A from evaluating the predicate. Shared storage device 26 is further configured to return a response 90 to request 80, the response 90 including a completely unprocessed result 94 for predicate evaluation from data 78, and shared storage device 26 is further configured to append full predicate rejection metadata 98 ​​to the response 90 to indicate that the predicate has not yet been evaluated. Response 90 is returned to OLAP engine 14, and OLAP engine 14 is configured to receive response 90, the response 90 including full predicate rejection metadata 98 ​​and a completely unprocessed result 94 for predicate evaluation from data 78. The OLAP engine 14 is further configured to determine that the predicate has not been evaluated based on the complete predicate rejection metadata 98, and to evaluate the predicate using the completely unprocessed result 94 received in the response 90 via the predicate evaluator 84B.

[0026] Finally, in scenario / scope (C), shared storage device 26 is further configured to determine that the CPU utilization 104 of shared storage device 26 does not exceed the predicate rejection threshold 86, wherein the CPU utilization 104 falls within the range of (C). Upon determining that the CPU utilization 104 of shared storage device 26 does not exceed the predicate rejection threshold 86, shared storage device 26 is further configured to evaluate the predicate at shared storage device 26 and return a response 90 to request 80, the response 90 including the complete processing result 102 of the predicate evaluation from data 78. Response 90 is returned to OLAP engine 14 for further processing.

[0027] Figure 3A The experimental results for querying workloads are shown without predicate evaluation based on CPU utilization. Figure 3B Experimental results are shown for query workloads obtained through predicate evaluation based on CPU utilization, according to this disclosure. For example... Figure 3A As shown, queries per second (QPS) fluctuate between 6K and 8K. In this scenario, performing workloads without predicate evaluation based on CPU utilization leads to storage overload, resulting in significant QPS variations. In contrast, as... Figure 3B As shown, in the case of predicate evaluation based on CPU utilization, the QPS ranges between 8K and 8.8K, and is consistent with... Figure 3ACompared to the previous set of queries, the average QPS was approximately 20% higher, with minimal QPS variation. This improvement is due to the system ensuring sufficient CPU resources are reallocated for the second set of queries, thereby increasing the overall QPS. These experiments highlight the effectiveness of this disclosure in maintaining query performance isolation, particularly under CPU overload conditions.

[0028] Figure 4 A flowchart of a data processing method 200 for a Hybrid Transaction / Analytical Processing (HTAP) database according to this disclosure is shown. Method 200 can be performed by... Figure 1 The HTAP database 10 shown is implemented and executed at a shared storage device 26. Therefore, the HTAP database may include an Online Transaction Processing (OLTP) engine, an Online Analytical Processing (OLAP) engine provided separately from the OLTP engine, and a shared storage device for storing data for multiple tenants.

[0029] Steps 202, 204, 206, 208, and 210 are performed at the shared storage device. At 202, method 200 may include receiving a request from the OLAP engine to perform predicate evaluation on data stored on the shared storage device. At 204, method 200 may include determining that the CPU utilization of the shared storage device exceeds a predicate rejection threshold. At 206, method 200 may include prohibiting the evaluation of at least a portion of the predicate when it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold. At 208, method 200 may include returning a response to the request, the response including partially or completely unprocessed results from the predicate evaluation of the data. At 210, method 200 may include appending predicate rejection metadata to the response to indicate that the partially or completely unprocessed results have not yet been evaluated. Furthermore, steps 212, 214, and 216 are performed at the OLAP engine. At 212, method 200 may include receiving a response including predicate rejection metadata and partially or completely unprocessed results from the predicate evaluation of the data. At 214, method 200 may include determining, based on predicate rejection metadata, that at least a portion of the predicate has not yet been evaluated. At 216, method 200 may include evaluating the predicate using partially or completely unprocessed results received in the response.

[0030] The system and method disclosed in this paper provide a hybrid database capable of maintaining system stability and Quality of Service (QoS) on a shared storage device. This is achieved through an adaptive predicate pushdown mechanism, where storage nodes on the shared storage device evaluate predicates based on CPU utilization and return responses to the OLAP engine, including partially or completely unprocessed results and partially or completely predicate rejection metadata. Therefore, the shared storage device engine can benefit from more available CPU cycles. This additional capacity can be allocated to other concurrent tasks, allowing them to run faster and thus improving overall processing speed.

[0031] In some embodiments, the methods and processes described herein may be associated with a computing system of one or more computing devices. Specifically, these methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0032] Figure 5 A non-limiting embodiment of a computing system 300 capable of implementing one or more of the methods and processes described above is schematically illustrated. The computing system 300 is shown in a simplified form. The computing system 300 can embody the above description and... Figure 2 The HTAP database 10 is illustrated in the diagram. The computing system 300 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones) and / or other computing devices, as well as wearable computing devices (such as smartwatches and head-mounted augmented reality devices).

[0033] The computing system 300 includes a logic processor 302, volatile memory 304, and non-volatile storage device 306. The computing system 300 may optionally include a display subsystem 308, an input subsystem 310, a communication subsystem 312, and / or... Figure 5 Other components not shown.

[0034] The logic processor 302 includes one or more physical devices configured to execute instructions. For example, the logic processor may be configured to execute instructions that are part of one or more application programs, programs, routines, libraries, objects, components, data structures, or other logical structures. Such instructions may be implemented to perform a task, implement a data type, change the state of one or more components, achieve a technical effect, or otherwise achieve a desired result.

[0035] A logic processor may include one or more physical processors (hardware) configured to execute software instructions. Alternatively, a logic processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of logic processor 302 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logic processor may optionally be distributed across two or more separate devices that may be remotely located and / or configured for coordinated processing. The various aspects of the logic processor may be virtualized and executed via remotely accessible networked computing devices configured in a cloud computing manner. It will be understood that, in this case, these virtualized aspects run on different physical logic processors on various different machines.

[0036] The non-volatile storage device 306 includes one or more physical devices configured to store instructions executable by a logic processor to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile storage device 306 may change, for example, to store different data.

[0037] Non-volatile storage device 306 may include removable and / or built-in physical devices. Non-volatile storage device 306 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.), and / or magnetic memory (e.g., hard disk drive, floppy disk drive, magnetic tape drive, MRAM, etc.) or other high-capacity storage technologies. Non-volatile storage device 306 may include non-volatile, dynamic, static, read / write, read-only, sequential access, location-addressable, file-addressable, and / or content-addressable devices. It should be understood that non-volatile storage device 306 is configured to retain instructions even when power to non-volatile storage device 306 is cut off.

[0038] Volatile memory 304 may include a physical device containing random access memory. Volatile memory 304 is typically used by logic processor 302 to temporarily store information during the processing of software instructions. It should be understood that volatile memory 304 will typically not continue storing instructions when power to it is cut off.

[0039] The logic processor 302, volatile memory 304, and non-volatile storage device 306 can be integrated together into one or more hardware logic components. Such hardware logic components may include field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), systems-on-a-chip (SoCs), and complex programmable logic devices (CPLDs), etc.

[0040] The terms "module," "program," and "engine" can be used to describe aspects of computing system 300 that are typically implemented in software by a processor to perform specific functions using portions of volatile memory. These functions involve transformation processing specifically configured for the processor to perform said functions. Therefore, a module, program, or engine can be instantiated via logic processor 302 using portions of volatile memory 304 to execute instructions stored in non-volatile storage device 306. It should be understood that different modules, programs, and / or engines can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same modules, programs, and / or engines can be instantiated from different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" can encompass single or multiple sets of executable files, data files, libraries, drivers, scripts, database records, etc.

[0041] Display subsystem 308 (when included) can be used to present a visual representation of data stored by non-volatile storage device 306. The visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data stored by the non-volatile storage device, and thus change the state of the non-volatile storage device, the state of display subsystem 308 can also be changed in the same way to visually represent changes in the underlying data. Display subsystem 308 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic processor 302, volatile memory 304, and / or non-volatile storage device 306 in a shared housing, or such display devices may be peripheral display devices.

[0042] Input subsystem 310 (when included) may include one or more user input devices, such as a keyboard, mouse, touchscreen, or game controller, or interface with such user input devices. In some embodiments, the input subsystem may include selected Natural User Input (NUI) components, or interface with such components. These components may be integrated or peripheral components, and the translation and / or processing of input actions may be performed on-board or off-board. Example NUI components may include a microphone for speech and / or voice recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; an electric field sensing component for assessing brain activity; and / or any other suitable sensor.

[0043] The communication subsystem 312 (when included) can be configured to communicatively couple the various computing devices described herein to each other and to other devices. The communication subsystem 312 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network, or a wired or wireless local area network or wide area network, such as HDMI via Wi-Fi. In some embodiments, the communication subsystem may allow the computing system 300 to send messages to and / or receive messages from other devices via a network such as the Internet.

[0044] The following paragraphs provide additional description of the subject matter of this disclosure. One aspect provides a hybrid transactional / analytic processing (HTAP) database, including an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage device storing data for multiple tenants. The shared storage device is configured to receive requests from the OLAP engine for predicate evaluation against data stored on the shared storage device. The shared storage device is further configured to determine that the CPU utilization of the shared storage device exceeds a predicate rejection threshold. The shared storage device is further configured to prohibit the evaluation of at least a portion of the predicate when it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold.

[0045] In this regard, the shared storage device is further configured to return a response to the request, the response including a partially or completely unprocessed result from the data for predicate evaluation, and the shared storage device is further configured to append partial or complete predicate rejection metadata to the response to indicate that the partially or completely unprocessed result has not yet been evaluated.

[0046] In this regard, the OLAP engine is configured to receive a response including partial or complete predicate rejection metadata and a partial or complete unprocessed result from that data for predicate evaluation. The OLAP engine is further configured to determine, based on the partial or complete predicate rejection metadata, that at least that portion of the predicate has not yet been evaluated. The OLAP engine is further configured to use the partial or complete unprocessed result received in the response to evaluate the predicate.

[0047] In this regard, the predicate rejection threshold can be a partial predicate rejection threshold. The shared storage device can further determine a full predicate rejection threshold higher than the partial predicate rejection threshold. The shared storage device is further configured to determine that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold. The shared storage device is further configured to prohibit the evaluation of a portion of the predicate when it is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold. The shared storage device is further configured to return a response to the request, the response including a partially unprocessed result of the predicate evaluation from the data; and to append partial predicate rejection metadata to the response to indicate that this portion of the predicate has not yet been evaluated.

[0048] In this regard, the OLAP engine is configured to receive a response including partial predicate rejection metadata and partial unprocessed results from that data for predicate evaluation. The OLAP engine is further configured to determine, based on the partial predicate rejection metadata, that part of the predicate has not yet been evaluated, and to evaluate the predicate using the partial unprocessed results received in the response.

[0049] In this regard, the predicate rejection threshold can be a partial predicate rejection threshold. The shared storage device can further determine a full predicate rejection threshold higher than the partial predicate rejection threshold. The shared storage device is further configured to determine that the CPU utilization of the shared storage device reaches the full predicate rejection threshold. The shared storage device is further configured to prohibit predicate evaluation when it is determined that the CPU utilization of the shared storage device reaches the full predicate rejection threshold. The shared storage device is further configured to return a response to the request, the response including a completely unprocessed result of predicate evaluation from the data. The shared storage device is further configured to append full predicate rejection metadata to the response to indicate that the predicate has not yet been evaluated.

[0050] In this regard, the OLAP engine is configured to receive a response including full predicate rejection metadata and a completely unprocessed result from that data for predicate evaluation. The OLAP engine is further configured to determine, based on the full predicate rejection metadata, that a predicate has not yet been evaluated, and to evaluate the predicate using the completely unprocessed result received in the response.

[0051] In this regard, the shared storage device is further configured to determine that the CPU utilization of the shared storage device does not exceed a predicate rejection threshold. The shared storage device is further configured to, upon determining that the CPU utilization of the shared storage device does not exceed the predicate rejection threshold, evaluate the predicate at the shared storage device and return a response to the request, the response including the complete processing result of the predicate evaluation from the data.

[0052] On the other hand, a data processing method for a Hybrid Transactional / Analytical Processing (HTAP) database is provided, the HTAP database including an Online Transaction Processing (OLTP) engine, an Online Analytical Processing (OLAP) engine provided separately from the OLTP engine, and a shared storage device storing data for multiple tenants. The method includes: at the shared storage device, receiving a request from the OLAP engine for predicate evaluation of data stored on the shared storage device; determining that the CPU utilization of the shared storage device exceeds a predicate rejection threshold; and when it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold, prohibiting the evaluation of at least a portion of the predicates.

[0053] In this respect, the data processing method may further include: at a shared storage device, returning a response to the request, the response including a partially or completely unprocessed result of the predicate evaluation from the data, and appending partial or complete predicate rejection metadata to the response to indicate that the partially or completely unprocessed result has not yet been evaluated.

[0054] In this regard, the data processing method may further include: at the OLAP engine, receiving a response including partial or complete predicate rejection metadata and a partial or complete unprocessed result of predicate evaluation from the data; determining, based on the partial or complete predicate rejection metadata, that at least that part of the predicate has not been evaluated; and using the partial or complete unprocessed result received in the response to evaluate the predicate.

[0055] In this regard, the predicate rejection threshold can be a partial predicate rejection threshold. The shared storage device can further determine a full predicate rejection threshold higher than the partial predicate rejection threshold. The method can further include: at the shared storage device, determining that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold; when it is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold, prohibiting the evaluation of a portion of the predicate; returning a response to the request, the response including a portion of the unprocessed result of the predicate evaluation from the data; and appending partial predicate rejection metadata to the response to indicate that the portion of the predicate has not yet been evaluated.

[0056] In this regard, the data processing method may further include: at the OLAP engine, receiving a response including partial predicate rejection metadata and partial unprocessed results from the data for predicate evaluation; determining, based on the partial predicate rejection metadata, that part of the predicate has not yet been evaluated; and using the partial unprocessed results received in the response to evaluate the predicate.

[0057] In this regard, the predicate rejection threshold can be a partial predicate rejection threshold. The shared storage device can further determine a full predicate rejection threshold higher than the partial predicate rejection threshold. The method can further include: at the shared storage device, determining that the CPU utilization of the shared storage device has reached the full predicate rejection threshold; when it is determined that the CPU utilization of the shared storage device has reached the full predicate rejection threshold, prohibiting predicate evaluation; returning a response to the request, the response including a completely unprocessed result for predicate evaluation from the data; and appending full predicate rejection metadata to the response to indicate that the predicate has not yet been evaluated.

[0058] In this regard, the data processing method may further include: at the OLAP engine, receiving a response including full predicate rejection metadata and a completely unprocessed result of predicate evaluation from the data; determining that the predicate has not yet been evaluated based on the full predicate rejection metadata; and evaluating the predicate using the completely unprocessed result received in the response.

[0059] In this regard, the data processing method may further include: determining, at the shared storage device, that the CPU utilization of the shared storage device does not exceed a predicate rejection threshold; evaluating the predicate at the shared storage device when it is determined that the CPU utilization of the shared storage device does not exceed the predicate rejection threshold; and returning a response to the request, the response including the complete processing result of the predicate evaluation from the data.

[0060] On the other hand, a hybrid transactional / analytic processing (HTAP) database is provided, comprising an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage device storing data for multiple tenants. The shared storage device is configured to receive a first request from the OLAP engine for evaluating a first predicate against data stored on the shared storage device; determine, in response to the first request, that the CPU utilization of the shared storage device exceeds a predicate rejection threshold but does not reach a full predicate rejection threshold; and, when it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold but does not reach the full predicate rejection threshold, prohibit the evaluation of a portion of the first predicate. The shared storage device is further configured to: receive a second request from the OLAP engine for evaluating a second predicate against data stored on the shared storage device; determine that the CPU utilization of the shared storage device reaches a full predicate rejection threshold; and, when it is determined that the CPU utilization of the shared storage device reaches the full predicate rejection threshold, prohibit the evaluation of the second predicate.

[0061] In this regard, the shared storage device is further configured to: upon determining, in response to a first request, that the CPU utilization of the shared storage device exceeds a predicate rejection threshold but does not reach a full predicate rejection threshold, return a first response to the first request, the first response including a partially unprocessed result of predicate evaluation from the data, and append first metadata to the first response to indicate that the portion of the predicate has not yet been evaluated. The shared storage device is further configured to: upon determining that the CPU utilization of the shared storage device has reached a full predicate rejection threshold, return a second response to a second request, the second response including a completely unprocessed result of predicate evaluation from the data, and append second metadata to the second response to indicate that the predicate has not yet been evaluated.

[0062] In this respect, the OLAP engine is configured to receive a first response or a second response including a partially unprocessed or completely unprocessed result of the predicate evaluation from the data and a first metadata or a second metadata; and to evaluate the first predicate or the second predicate using the partially unprocessed or completely unprocessed result received in the first response or the second response.

[0063] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as various variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Therefore, the various actions illustrated and / or described may be performed in the illustrated and / or described order, in a different order, in parallel, or omitted. Similarly, the order of the above processes may also be changed.

[0064] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as any and all equivalents thereof.

Claims

1. A hybrid transactional / analytic processing (HTAP) database, comprising: Online Transaction Processing (OLTP) engine; An online analytical processing (OLAP) engine is provided separately from the OLTP engine; as well as A shared storage device for storing data from multiple tenants, wherein the shared storage device is configured to: Receive a request from the OLAP engine for predicate evaluation of the data stored on the shared storage device; It was determined that the CPU utilization of the shared storage device exceeded the predicate rejection threshold; as well as When it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold, at least a portion of the predicate is prohibited from being evaluated.

2. The HTAP database of claim 1, wherein the shared storage device is further configured as follows: Return a response to the request, the response including partially or completely unprocessed results from the data for predicate evaluation; and Append partial or complete predicate rejection metadata to the response to indicate that the partial or complete unprocessed result has not yet been evaluated.

3. The HTAP database as described in claim 2, wherein the OLAP engine is configured as follows: Receive the response, which includes the partial or complete predicate rejection metadata and the partial or complete unprocessed result of the predicate evaluation from the data; Based on the partial or complete predicate rejection metadata, it is determined that at least a portion of the predicate has not been evaluated; and The predicate is evaluated using the partial or completely unprocessed result received in the response.

4. The HTAP database as described in claim 1, wherein, The predicate rejection threshold is a partial predicate rejection threshold. The shared storage device further determines a full predicate rejection threshold that is higher than the partial predicate rejection threshold, and The shared storage device is further configured as follows: It is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the complete predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold, evaluation of a portion of the predicate is prohibited. Return a response to the request, the response including a portion of the unprocessed results from the data for predicate evaluation; as well as Append partial predicate rejection metadata to the response to indicate that the portion of the predicate has not yet been evaluated.

5. The HTAP database as described in claim 4, wherein the OLAP engine is configured as follows: Receive the response, which includes the partial predicate rejection metadata and the partial unprocessed result of the predicate evaluation from the data; Based on the partial predicate rejection metadata, it is determined that the portion of the predicate has not yet been evaluated; as well as The predicate is evaluated using the partially unprocessed result received in the response.

6. The HTAP database as described in claim 1, wherein, The predicate rejection threshold is a partial predicate rejection threshold. The shared storage device further determines a full predicate rejection threshold that is higher than the partial predicate rejection threshold, and The shared storage device is further configured as follows: It is determined that the CPU utilization of the shared storage device has reached the complete predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device reaches the full predicate rejection threshold, the evaluation of the predicate is prohibited. Return a response to the request, the response including the completely unprocessed result of the predicate evaluation from the data; as well as Append the full predicate rejection metadata to the response to indicate that the predicate has not yet been evaluated.

7. The HTAP database of claim 6, wherein the OLAP engine is configured as follows: Receive a response including the complete predicate rejection metadata and the completely unprocessed result of the predicate evaluation from the data; Based on the complete predicate rejection metadata, it is determined that the predicate has not yet been evaluated; and The predicate is evaluated using the completely unprocessed result received in the response.

8. The HTAP database of claim 1, wherein the shared storage device is further configured to: It was determined that the CPU utilization of the shared storage device did not exceed the predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device does not exceed the predicate rejection threshold, the predicate is evaluated at the shared storage device; and Return a response to the request, the response including the full processing results of the predicate evaluation from the data.

9. A data processing method for a hybrid transactional / analytic processing (HTAP) database, the HTAP database comprising an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage device for storing data of multiple tenants, the method comprising, at the shared storage device: Receive a request from the OLAP engine for predicate evaluation of the data stored on the shared storage device; It was determined that the CPU utilization of the shared storage device exceeded the predicate rejection threshold; as well as When it is determined that the CPU utilization of the shared storage device exceeds the predicate rejection threshold, at least a portion of the predicate is prohibited from being evaluated.

10. The data processing method of claim 9, further comprising, at the shared storage device: Return a response to the request, the response including partially or completely unprocessed results from the data for predicate evaluation; and Append partial or complete predicate rejection metadata to the response to indicate that the partial or complete unprocessed result has not yet been evaluated.

11. The data processing method of claim 10, further comprising, at the OLAP engine: Receive a response including the partial or complete predicate rejection metadata and the partial or complete unprocessed result of the predicate evaluation from the data; Based on the partial or complete predicate rejection metadata, it is determined that at least a portion of the predicate has not been evaluated; and The predicate is evaluated using the partial or completely unprocessed result received in the response.

12. The data processing method as described in claim 9, wherein, The predicate rejection threshold is a partial predicate rejection threshold. The shared storage device further determines a full predicate rejection threshold that is higher than the partial predicate rejection threshold, and The method further includes, at the shared storage device: It is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the complete predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device exceeds the partial predicate rejection threshold but does not reach the full predicate rejection threshold, evaluation of a portion of the predicate is prohibited. Return a response to the request, the response including a portion of the unprocessed results from the data for predicate evaluation; as well as Append partial predicate rejection metadata to the response to indicate that the portion of the predicate has not yet been evaluated.

13. The data processing method of claim 12, further comprising, at the OLAP engine: Receive the response, which includes the partial predicate rejection metadata and the partial unprocessed result of the predicate evaluation from the data; Based on the partial predicate rejection metadata, it is determined that the portion of the predicate has not yet been evaluated; as well as The predicate is evaluated using the partially unprocessed result received in the response.

14. The data processing method as described in claim 9, wherein, The predicate rejection threshold is a partial predicate rejection threshold. The shared storage device further determines a full predicate rejection threshold that is higher than the partial predicate rejection threshold, and The method further includes, at the shared storage device: It is determined that the CPU utilization of the shared storage device has reached the complete predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device reaches the full predicate rejection threshold, the evaluation of the predicate is prohibited. Return a response to the request, the response including the completely unprocessed result of the predicate evaluation from the data; as well as Append the full predicate rejection metadata to the response to indicate that the predicate has not yet been evaluated.

15. The data processing method of claim 14, further comprising, at the OLAP engine: Receive a response including the complete predicate rejection metadata and the completely unprocessed result of the predicate evaluation from the data; Based on the complete predicate rejection metadata, it is determined that the predicate has not yet been evaluated; and The predicate is evaluated using the completely unprocessed result received in the response.

16. The data processing method of claim 9, further comprising, at the shared storage device: It was determined that the CPU utilization of the shared storage device did not exceed the predicate rejection threshold; When it is determined that the CPU utilization of the shared storage device does not exceed the predicate rejection threshold, the predicate is evaluated at the shared storage device; and Return a response to the request, the response including the full processing results of the predicate evaluation from the data.

Images