Compression service using FPGA compression

Abstract

A method, system, and computer program product for providing compression services, such as compression as a service (Caas). In some embodiments, a method is provided that includes receiving, at a compression service, metric information about one or more columns of a database server; in response to receiving the metric information, processing, by the compression service, the metric information to identify at least one column as a candidate for compression by the compression service; in response to identifying the at least one column, compressing, by the compression service, the at least one column, wherein the compression service uses a hardware accelerator to execute a compression algorithm that compresses the at least one column; and notifying, by the compression service, the database server of the at least one column compressed by the compression service. Related systems, methods, and articles of manufacture are also disclosed.

Description

Technical Field

[0001] This disclosure generally relates to data processing, and more specifically, to a database management system including compression provided by a separate hardware-based compression service. Background Technology

[0002] Database management systems have become an integral part of many computer systems. For example, some systems process hundreds or even thousands of transactions per second. On the other hand, some systems perform highly complex multidimensional analyses of data. In both cases, the underlying database may need to process responses to queries very quickly to meet system requirements regarding transaction time. Given the complexity and / or the number of these queries, the underlying database faces the challenge of optimizing performance. Summary of the Invention

[0003] In some implementations, a method is provided, comprising: receiving at a compression service metric information about one or more columns of a database server; in response to receiving the metric information, processing the metric information by the compression service to identify at least one column as a candidate for compression by the compression service; in response to identifying the at least one column, compressing the at least one column by the compression service, wherein the compression service uses a hardware accelerator to perform a compression algorithm for compressing the at least one column; and notifying the database server of the at least one column compressed by the compression service.

[0004] In some variations, one or more of the features disclosed herein, including the following features, may optionally be included in any feasible combination. The compression service includes a hardware data processor that provides hardware acceleration for compression provided by the compression service. The hardware data processor includes one or more of the following: at least one field-programmable gate array (FPGA), at least one graphics processing unit, and at least one tensor processing unit. The compression service includes compression-as-a-service provided by a platform acting as a service provider. The service provider offering the compression service is the same platform as the service provider offering the database server. The same data center as the service provider platform provides the compression service and the database server. A different data center as the service provider platform provides the compression service and the database server. The platform acting as the service provider offering the compression service is a different platform from the service provider offering the database server. The database server includes an in-memory columnstore database. The metric information includes one or more of the following: the size of at least one column in bytes, the access frequency of the size of at least one column, an access frequency threshold, and a size threshold. The processing also includes identifying at least one column in response to the size of at least one column exceeding the size threshold. The processing also includes processing at least one other column, and in response to the corresponding size of at least one other column being less than the size threshold, at least one other column is not identified as being compressed by the compression service. The hardware accelerator includes a floating-point gate array that provides a recursive paired compression algorithm. The notification also includes a list of at least one table for which the compression service provides compression and / or at least one column for which compression is provided.

[0005] The document also describes a non-transitory computer program product (i.e., a physically embodied computer program product) that stores instructions that, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform the operations described herein. Similarly, a computer system is described that may include one or more data processors and memory coupled to the one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. Furthermore, the methods may be implemented within a single computing system or distributed across two or more computing systems by one or more data processors. Such computing systems may be connected via one or more connections (including, but not limited to, connections via networks (e.g., the Internet, wireless wide area networks, local area networks, wide area networks, wired networks, etc.), via direct connections between one or more of the multiple computing systems, and may exchange data and / or commands or other instructions.

[0006] Details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the following description. Further features and advantages of the subject matter described herein will be apparent from the specification, drawings, and claims. Attached Figure Description

[0007] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate certain aspects of the subject matter disclosed herein and, together with the description, help to explain some principles associated with the disclosed embodiments. In the drawings,

[0008] Figure 1A Examples of systems according to some embodiments are described;

[0009] Figure 1B Examples of columnar storage layouts used in column-oriented databases according to some embodiments are depicted;

[0010] Figure 2 Another example of a system according to some embodiments is described;

[0011] Figure 3A Block diagrams and process flows according to some embodiments are depicted;

[0012] Figure 3B Examples of incremental fragments and main fragments of a column-stored database according to some embodiments are depicted;

[0013] Figure 3C An example of re-paired compression according to some embodiments is depicted;

[0014] Figure 4 Examples of process flows according to some embodiments are described;

[0015] Figure 5 Examples of hardware-based compression-as-a-service (CaaS) implementations using hardware accelerators such as field-programmable gate arrays (FPGAs) are described according to some embodiments; and

[0016] Figure 6 Another example of a system according to some embodiments is described. Detailed Implementation

[0017] In systems such as in-memory databases, memory is a precious resource, so optimizing the use of memory resources is an aspect of their operation. For example, compression is one way to optimize memory resources. Specifically, compression can be used to compress data, thereby saving memory footprint on the data. However, compressing data requires time and processing resources. In in-memory databases, time is also a precious resource because queries should be processed quickly, so the benefits of data compression should take into account the compression costs in terms of time (or processing resources). In some embodiments, a standalone data processing service is disclosed, such as a compression service or Compression as a Service (CaaS) hosted at a Platform as a Service (PaaS) provider. In some embodiments, the CaaS service employs hardware accelerators, such as Field Programmable Gate Arrays (FPGAs), Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and / or other types of accelerators for performing data processing (such as compression in the case of CaaS). In the case of CaaS, CaaS allows the database management system to offload compression to CaaS. The use of CaaS represents an example data processor, which can consist of software and hardware components, where the hardware components help offload work from or accelerate the CPU-based software processor.

[0018] Figure 1A An example of a system 100 according to some exemplary implementations is depicted. Figure 1A The examples depict one or more user devices (labeled as client user devices) 102A-102N. User devices may include or be included in computers, smartphones, tablets, Internet of Things (IoT) devices, and / or other types of computer devices or processor-based devices. User devices may also include user interfaces, such as browsers or other applications, to enable access to one or more applications, database layers, and / or databases, thereby enabling the sending of queries and the receiving of responses to those queries from one or more database management systems.

[0019] exist Figure 1AIn the example, user devices 102A-102N can access at least one database management system, such as database management system 150 hosted on a cloud-based hardware platform (such as Platform as a Service (PaaS)). An example of a PaaS provider is Amazon Web Services, although this is merely an example illustrating the concept of a PaaS provider. A PaaS can provide a host server configured to provide database server 160A, while other host servers at the PaaS can provide database server 160B, and so on, up to database server 160N. To further illustrate, a PaaS provider can provide resources that can be shared among multiple client devices. For example, a cloud PaaS provider can be configured to provide various additional services, including, for example, Software as a Service (SaaS), Infrastructure as a Service (IaaS), Database as a Service (DaaS), and the aforementioned PaaS. And these services can be accessed by one or more client devices. As described above, a PaaS provider can include resources such as at least one computer (e.g., a server), data storage devices, and a network (including network equipment) coupling (multiple) computers and storage devices. The hosting platform at a PaaS can provide (or, for example, include) other resources, such as operating systems, hypervisors, and / or other resources, to virtualize physical resources (e.g., via virtual machines), provide application deployment (e.g., via containers), and other resources. In PaaS, a “public” cloud platform can be considered, where services can be provided to clients on demand via the Internet. For example, resources at a public cloud platform can be operated and / or owned by a cloud service provider (e.g., Amazon Web Services, Azure, etc.), allowing physical resources at the cloud service provider to be shared by multiple clients. Alternatively or additionally, PaaS can be considered a “private” cloud platform, in which case the cloud platform’s resources can be hosted on the entity’s own private servers (e.g., dedicated corporate servers operated and / or owned by the entity). Alternatively or additionally, PaaS can be considered a “hybrid” cloud platform, which includes a combination of on-premises resources and resources hosted by public or private cloud platforms. For example, a hybrid cloud service may include a web server running in a public cloud, while application servers and / or databases are hosted on-premises (e.g., in a region controlled or operated by an entity such as a corporate entity).

[0020] exist Figure 1AIn the example, database server 160A includes an in-memory database, an example of which is SAP's HANA database (although other types of databases can also be used). Alternatively or additionally, database server 160A may include a column-stored database. The phrase "in-memory database" refers to a database management system where main memory provides primary computer data storage for transactional data, such as data used to respond to queries, so that most (if not all) of the data typically used for transactions can be retained in main memory rather than stored on slower types of storage devices, such as disk-based storage. The use of in-memory databases provides faster queries when compared to more disk-based storage solutions. A column-stored database (or column-stored database management system) is a database management system that indexes data in columns of database tables, allowing column indexes to be stored instead of row data. In a column-stored database (also known as a column-oriented database), dictionaries can be used to compress values ​​in columns, allowing the values ​​of column indexes to be decoded into the original data using a dictionary. Because values ​​in columns may be of similar types (e.g., a column reference, a column of countries, a column of quantities), the use of dictionaries can provide some compression when compared to row-oriented databases where values ​​in rows may be dissimilar.

[0021] Figure 1B This illustrates an example of a columnar, value-ID-based storage layout used in a columnstore database. Data vector 110 represents value 105 encoded using dictionary 115 and then stored in the columnstore database. For example, the value "Rome" 102A is encoded using dictionary value ID "4" 102B. Other values, such as Berlin, NULL, Istanbul, Hamburg, etc., are also encoded using dictionary 115. Therefore, column 105 can be stored as data vector 110 in encoded form (e.g., based on dictionary 115 encoding). Figure 1B In the example, dictionary 115 is used to encode null values, but these may not be stored, although they could be. Inverted index 148 can be used to map each value ID of the data vector to a list of row IDs (representing row positions) in the data vector 110 that contain the value ID.

[0022] Since the database server 160A is implemented at the PaaS provider, multiple database servers 160B-160N can be provided to serve the same or different client user devices. For various reasons, such as increased or decreased demand, the number of database servers 160A-160N can be scaled up (e.g., increased) or scaled down (e.g., decreased).

[0023] Refer again Figure 1ADatabase server 160A may include one or more components, such as session manager 162A, optimizer 162B, execution engine 162C, directory 162D, transaction manager 162E, and table updater 162F.

[0024] Session Manager 162A handles the tracking of each session associated with client user devices 102A-102N (e.g., by tracking sessions associated with requests such as queries from clients). Optimizer 162B provides a query optimizer for database queries provided by client user devices. Execution Engine 162C is the query execution engine that handles the execution of database queries (or simply queries). Catalog 162D maintains metadata for database server 160 (e.g., metadata about tables, users, etc.). Transaction Manager 162E is responsible for the atomicity, consistency, isolation, and durability (ACID) of transactions in a multi-user system. Table Updater 162F is responsible for data manipulation in the database, such as inserting data into tables, deleting data from tables, updating data in tables, etc.

[0025] exist Figure 1A In the example, database server 160A stores its data in a storage location labeled cloud storage 180. For example, database server 160A may contain most (if not all) of the transaction data in its own storage, but may periodically save some data to cloud storage. The cloud storage may be in-memory cloud storage. Alternatively or additionally, cloud storage may also include other types of storage devices. In some implementations, data is persistently stored in cloud storage and available to the in-memory database as a temporary storage. CaaS, as an example of a data processing service, not only requires read-only table data for processing but also modifies the data, requiring write access to the data store. The newly compressed data will then be available to the in-memory component. To some extent, the in-memory component can be a data processor similar to that of CaaS.

[0026] In some embodiments, system 100 also includes Compression as a Service (CaaS) 190A. In some embodiments, CaaS can provide compression for data used by one or more database servers. For example, a table (or column) can be identified for compression by CaaS. In this case, CaaS compresses the table (or column) data to save memory size (and usage) of database server 160A. As mentioned above, the data can be user data, or it can be as described above regarding... Figure 1B The index described in vector 110. In the case of a column-store database, the column can correspond to the index (also called a data vector) as described above with respect to 110.

[0027] In some embodiments, CaaS 190A includes a hardware accelerator that performs compression of column data. In some embodiments, the hardware accelerator is an FPGA circuit (or other type of hardware accelerator). In some embodiments, a single CaaS 190A may be shared among multiple database servers 160A-160N. Furthermore, as shown in Figure 1, the number of CaaS can be scaled up (added) or scaled down (removed) such that the scaling is based on compression requirements. For example, based on requirements, a set of CaaS 190B can be designated to serve database servers 190A-190N. Alternatively or additionally, a given CaaS, such as CaaS 190B, can be designated to serve a specific database server (or multiple specific database servers).

[0028] System 100 may also include a CaaS orchestrator 164A-164B. The CaaS orchestrator can be used to interface with CaaS. Alternatively or additionally, the CaaS orchestrator can be used to trigger CaaS to identify one or more candidate columns for compression and / or trigger CaaS to compress the identified columns. The orchestrator provides an intelligent component for determining when and by whom an action is performed. In the case of CaaS, the orchestrator examines table metrics in the cloud storage, determines (or determines) a compression scheme, selects a data processor for the hardware accelerator, schedules the job, and / or consumes the results after compression to determine further actions and evaluate the quality of the results (e.g., performance improvements, etc.).

[0029] exist Figure 1A In the example of System 100, Database Server 160A, Cloud Storage 180, and CaaS 190A are all implemented on the same virtual network within the data center of a given PaaS. When implemented at a single data center of a PaaS provider, the system implementation can be simplified in terms of communication and other issues associated with cross-data center and cross-PaaS implementations. Nevertheless, Database Server 160A, Cloud Storage 180, and CaaS 190A can be implemented across data centers (i.e., different data centers of the same PaaS provider) or across PaaS providers (different PaaS providers). Figure 2 System 200 is depicted as similar in some respects to System 1 in Figure 1, but when compared to Database Management System 160A, System 200 implements CaaS in a different data center of the same PaaS (e.g., another Amazon Web Services data center) or using a different PaaS provider (e.g., another web service provider).

[0030] Figure 3A A block diagram, including a process flow, is depicted according to some embodiments. Figure 3ADatabase server 160A, cloud storage 180, and CaaS 190A (which implement hardware accelerators, such as FPGAs) are described.

[0031] At 310, database operations can be performed on one or more database tables (or columns of database tables). For example, database server 160A can perform a delta merge. An delta merge refers to a merge that moves data (corresponding to changes) to the read-optimized main storage of database server 160A. The changed data may be stored in one or more database tables (or columns) and moved to one or more other database tables (or columns) in the main storage. In some embodiments, the database operation may trigger metric collection at 320, metric polling at 330, and / or identification and compression of certain tables or columns at 340-350.

[0032] although Figure 3A The example describes the operation as an incremental merge, but other database operations can also occur in 310. These operations require access to tables (or columns) in database server 160A. Thus, using CaaS compression can save memory usage for the memory database.

[0033] In the case of a column-oriented database, it can consist of read-optimized main fragments (which can hold most of the data) and small write-optimized incremental fragments. In this scenario, values ​​can initially be inserted (or updated) at the end of the write-optimized incremental fragment. However, queries may need to be executed on both the main and incremental fragments, in which case the corresponding results are merged as an incremental merge. Figure 3B Examples of incremental fragment 162 and main fragment 164 of a columnstore database are depicted, both of which are related to (in Figure 1B The column store master segment 164 is associated with column 105 (as depicted in the diagram). It may include most (if not all) of the data in column 105 (or, if partitioned, the column partition itself), and the incremental segment 162 contains newly written data that has not yet been merged into the column store master segment 164. Although only a single column of the incremental and master segments is shown for simplicity, a column store database may include additional segments and columns.

[0034] Refer again Figure 3ACloud storage 180 can collect metrics at 320 regarding database operations performed by database server 160A at 310. For example, cloud storage can collect metrics such as the length (e.g., size) of tables (or columns of database tables) accessed as part of the operations at 310. To further illustrate, a first database column (a long string containing values) will provide more storage space savings when compressed than a compressed second database column of the same length without values, allowing the benefits of compression to be weighed against the processing costs of performing compression. As an additional example, cloud storage can collect metrics such as the access frequency of tables (or columns of database tables) accessed as part of the operations at 310. For example, access frequency can indicate how often database server 160A accesses a table (or column) within a given time period or session. Thus, a table (or column) that is rarely accessed may not be a good candidate for compression compared to frequently accessed columns. In some embodiments, metrics collected at 320 may be processed to determine size thresholds (e.g., length in bytes or string length) for one or more tables (or columns) of the database server 160, such that a given column with a length (or string length) less than the size threshold is not compressed. In this way, the memory-saving benefits of compression are more likely to outweigh the processing costs of performing compression.

[0035] In some embodiments, the column includes data representing an index (or column index) as described above with respect to vector 110. In this case, a first compression scheme can be used when the column size is less than the size threshold described above. However, when the column size is greater than (or equal to) the size threshold, a second compression can be applied to the column. As illustrated by example, if the column size is less than the 3-byte size threshold, no compression (or lightweight compression, such as prefix compression) is applied to the column, but if the column size is greater than or equal to the 3-byte size threshold, a re-pairing compression algorithm is applied to the column by the hardware accelerator of the CaaS 190A. In the context of string dictionary compression, prefix compression refers to encoding where, for a string sequence, subsequent strings can share a common prefix. The first string of a block is stored uncompressed, and for subsequent strings, the prefix length shared with the original string is stored along with the suffix. The term "re-pairing" refers to recursive pairing. Re-pairing is a grammar-based compressor that operates by recursively replacing high-frequency character pairs with new grammar symbols.

[0036] Figure 3CThis illustrates an example of re-pairing compression applied to the text “tobeornottobeortobeornot” (see line 1). When the grammar rules are applied, the text after the pairing replacement is “0beornot0beor0beornot”. The grammar rules are applied until compression produces “747” (see line 8), which is the re-pairing compressed form of “tobeornottobeortobeornot”.

[0037] Refer again Figure 3A CaaS 190A can obtain one or more metrics collected by cloud storage 180 at 330. For example, CaaS 190A can check (e.g., poll) or request metrics about one or more database tables (or columns) of database server 160A from cloud storage 180. Alternatively, cloud storage 180 can push metrics for one or more database tables (or columns) without explicit request from CaaS 190A. These metrics may include one or more of the following for each table (or column) of database server 160A: table (or column) size in bytes, table (or column) access frequency, access frequency threshold, size threshold, and / or other metrics.

[0038] At 340, CaaS 190A can identify one or more tables (or columns) for compression in CaaS. For example, CaaS can process the metric information obtained at 330 and then identify which tables (or columns) have a size exceeding a threshold size. Referring to the previous example, CaaS can identify one or more tables (or columns) larger than a size threshold (e.g., such as 3 bytes) as candidates for compression by CaaS.

[0039] Alternatively or additionally, CaaS 190A can also identify tables (or columns) based on the frequency of access. For example, if the frequency of access to a table (or column) (e.g., read, write, delete, or other operations) is less than an access frequency threshold, CaaS can determine that the column is not a good candidate for compression by CaaS, even if the column size exceeds a size threshold. If the frequency of access to a table (or column) is greater than an access frequency threshold, CaaS can determine that the table (or column) is a good candidate for compression by CaaS, assuming the column size exceeds a size threshold.

[0040] At 350, the CaaS 190A can compress one or more tables (or columns) identified at 340. As described above, the CaaS 190A may include hardware accelerators to provide faster compression compared to CPU-implemented compression. In some embodiments, the CaaS 190A may apply grammar-based compression schemes, such as re-pairing compression of the table(s)(s)(s)(s) identified at 340.

[0041] To illustrate compression 350 during operation, if the size of the first column exceeds a size threshold, the first column can be identified as a candidate for compression at 340, and then compressed by the compression hardware of CaaS 190 at 350. In another example, if the first column has a size exceeding a size threshold and an access frequency exceeding an access frequency threshold, the first column can be identified as a candidate for compression at 340, and then compressed by the compression hardware of CaaS 190 at 350. In yet another example, if the first column has a size exceeding a size threshold and an access frequency not exceeding an access frequency threshold, the first column may not be identified as a candidate for compression at 340, and therefore not compressed by the compression hardware of CaaS 190 at 350. Furthermore, if the size of the first column does not exceed a size threshold, the first column will not be identified as a candidate for compression at 340, and therefore not compressed by the compression hardware of CaaS 190 at 350. CaaS can evaluate one or more tables (or columns) in a database server 160A to identify candidate tables (or columns) for compression, and then compress the identified candidate tables (or columns).

[0042] While some examples involve the CaaS 190A providing re-paired compression, the CaaS 190A can perform other types of compression, and re-paired compression can advantageously provide enhanced compression ratios when compared to other technologies.

[0043] At 360, CaaS 190A can provide database server 160A with notifications of columns (or tables) that have been optimized through compression by CaaS 190A. For example, CaaS 190A can provide a list of one or more columns (or tables) that have been compressed by CaaS using, for example, re-pair compression. This allows DB server 160A to reload the compressed columns (or tables)(s) into memory (thus saving memory). Alternatively or additionally, CaaS can provide compressed tables(or columns)(s) at 360. For example, compressed tables are streamed to cloud storage and updated if consistent. As described above, the in-memory portion is subsequently updated from storage.

[0044] In 370, the database server can reload (multiple) compressed tables (or columns) in response to notifications that they have been compressed.

[0045] While some examples involve in-memory and / or column-store databases such as HANA, CaaS 190A can also be implemented in other types of databases.

[0046] Figure 4 An example of process flow 400 according to some embodiments is described.

[0047] At 410, CaaS 190A can receive metrics about one or more columns (or one or more tables) of database server 160A. In some embodiments, as described above, the database server can be configured as an in-memory, column-store database. CaaS 190A can be provided by a Platform as a Service (PaaS) provider that also provides database server 160A and / or cloud storage 180, allowing CaaS, database server, and cloud server to operate on the same virtual private network without needing to communicate across data centers or across PaaS providers. Alternatively, CaaS 190A can be provided at a first data center of PaaS, while database server 160A and / or cloud storage 180 are hosted at a second data center of PaaS (thus requiring cross-data center communication between the two data centers across PaaS). Alternatively, CaaS 190A can be provided at a first PaaS, and database server 160A can be provided at a different second PaaS provider (thus requiring cross-PaaS communication between the two PaaS providers).

[0048] At 410, metrics may include information about columns (or tables) of database server 160A, enabling CaaS to determine which columns (or tables) are candidates for compression. Metrics may include one or more of the following for each table (or column) of database server 160A: table (or column) size in bytes, table (or column) access frequency, access frequency threshold, size threshold, etc.

[0049] In response to receiving metric information, the CaaS 190A can process the metric information at 420 to identify at least one column (or at least one table) as a candidate for compression by the CaaS 190A. In some embodiments, this processing can be based on a size threshold and / or an access frequency threshold. For example, a size threshold can be used to identify columns (or tables) that exceed the size threshold and similarly justify the processing resources required to compress the column (or table). An access frequency threshold can identify tables (or columns) that are rarely accessed and therefore not worth compressing when compared to other tables / columns that are accessed more frequently.

[0050] At 430, the CaaS 190A can use a hardware accelerator to compress at least one column (or table) identified at 420. As described above, the CaaS 190A may include a hardware accelerator, such as an FPGA, to compress at least one table (or at least one column) identified at 420.

[0051] At 440, CaaS can notify the DB server that at least one column (or table) has been compressed. CaaS can provide the database server 160A and / or cloud storage 180 with a list of compressed columns (or tables) and / or provide (multiple) compressed columns (or tables) to enable the use or reloading of compressed columns or tables.

[0052] Figure 5 An example implementation from the perspective of hardware-based cloud services supporting CaaS 190A is described. Figure 5 In this implementation, CaaS 190A is implemented at the PaaS host node 550. The host node processes (or controls) attached FPGAs 510A-510N based on workload monitoring provided by a monitoring component 502 outside the hardware service, which amplifies (e.g., adds) or reduces (e.g., removes) the workload. The hardware service can process synchronization requests, including, for example, a unique ID, a problem (mapped to a hardware function, such as compression), data (e.g., for synchronization requests) / src-data-id, and trg-data-id (for streaming large datasets), and return the processed data (e.g., compressed in the case of CaaS). Depending on the trigger (e.g., the monitoring component) or itself, the application may need to perform hardware functions on potentially large amounts of data. The data may arrive with the request or be streamed from an external data store 520A, which is received via a data controller. From the hardware service's perspective, when hardware resources 510A-510N are busy, requests are buffered (e.g., queued) to employ flow control (e.g., backpressure). Hardware resources are partitioned, with each partition containing logic for data processing functions (e.g., compression, predictive analytics, or data processing as a custom function). In the host portion, each resource partition can be addressed by a worker, which accepts new requests based on its hardware capabilities and / or the specific problem being addressed (e.g., compression) and streams data from the data controller to its attached hardware resources for execution. Results are streamed back to the worker and forwarded to the destination data store 520B. Once a result has been forwarded, the worker asynchronously reports the processing status (e.g., id, trg-id, status = success, statistics / metrics, such as runtime) to the local controller and will accept new compression requests. The controller reports the status to a monitor 502 or application 515, decoupled via a publish-subscribe channel, on which the controller and application can listen using the id.

[0053] In some implementations, the current topic can be configured to be implemented in system 600, such as Figure 6As shown. For example, a DB server 160A can be physically embedded in system 600. To further illustrate, system 600 may also include an operating system, hypervisor, and / or other resources to provide virtualized physical resources (e.g., via virtual machines). System 600 may include a processor 610, memory 620, storage device 630, and input / output device 640. Each of components 610, 620, 630, and 640 may be interconnected using a system bus 650. Processor 610 may be configured to process instructions for execution within system 600. In some embodiments, processor 610 may be a single-threaded processor. In alternative embodiments, processor 610 may be a multi-threaded processor. In some embodiments, the system may be coupled to a hardware accelerator, such as an FPGA, that performs compression algorithms such as re-pairing to compress certain tables (or columns).

[0054] Processor 610 may also be configured to process instructions stored in memory 620 or on storage device 630, including receiving or sending information via input / output device 640. Memory 620 may store information within system 600. In some embodiments, memory 620 may be a computer-readable medium. In alternative embodiments, memory 620 may be a volatile memory cell. In still other embodiments, memory 620 may be a non-volatile memory cell. Storage device 630 is capable of providing mass storage for system 600. In some embodiments, storage device 630 may be a computer-readable medium. In alternative embodiments, storage device 630 may be a floppy disk device, hard disk device, optical disk device, magnetic tape device, non-volatile solid-state memory, or any other type of storage device. Input / output device 640 may be configured to provide input / output operations for system 600. In some embodiments, input / output device 640 may include a keyboard and / or a pointing device. In alternative embodiments, input / output device 640 may include a display unit for displaying a graphical user interface.

[0055] The systems and methods disclosed herein can be embodied in various forms, including, for example, data processors, computers that also include databases, digital electronic circuits, firmware, software, or combinations thereof. Furthermore, the aforementioned features, as well as other aspects and principles of the embodiments of this disclosure, can be implemented in various environments. Such environments and related applications can be specifically constructed to perform various processes and operations according to the disclosed embodiments, or they can include general-purpose computers or computing platforms that are selectively activated or reconfigured by code to provide the necessary functionality. The processes disclosed herein are not inherently associated with any particular computer, network, architecture, environment, or other apparatus, and can be implemented through appropriate combinations of hardware, software, and / or firmware. For example, various general-purpose machines can be used with programs written according to the teachings of the disclosed embodiments, or it may be more convenient to construct dedicated apparatus or systems to perform the required methods and techniques.

[0056] Although in some cases, ordinal numbers such as first, second, etc., relate to order, as used in this document, ordinal numbers do not necessarily imply order. For example, ordinal numbers can only be used to distinguish one item from another. For example, to distinguish the first event from the second event, but do not need to imply any chronological order or a fixed system of reference (such that the first event in one paragraph may differ from the first event in another paragraph).

[0057] The foregoing description is intended to illustrate, and not limit, the scope of the invention, which is defined by the appended claims. Other embodiments are within the scope of the appended claims.

[0058] These computer programs may also be referred to as programs, software, software applications, applications, components, or code, including machine instructions for programmable processors, and may be implemented using high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the term "machine-readable medium" means any computer program product, apparatus, and / or device, such as a disk, optical disk, memory, and programmable logic device (PLD), for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" means any signal used to provide machine instructions and / or data to a programmable processor. Machine-readable media may store such machine instructions non-transitory, such as non-transitory solid-state memory or magnetic hard disk drives or any equivalent storage medium. Machine-readable media may alternatively or additionally store such machine instructions transiently, such as processor caches or other random access memory associated with one or more physical processor cores.

[0059] To provide interaction with the user, the subjects described herein can be implemented on a computer having a display device for showing information to the user, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including but not limited to sound, speech, or tactile input.

[0060] The subject matter described herein can be implemented in a computing system that includes backend components, such as one or more data servers, or middleware components, such as one or more application servers, or frontend components, such as one or more client computers having a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described herein, or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium, such as a communication network. Examples of communication networks include, but are not limited to, local area networks (“LANs”), wide area networks (“WANs”), and the Internet.

[0061] A computing system may include clients and servers. Clients and servers are typically, but not exclusively, geographically separated and usually interact through a communication network. The client-server relationship arises from computer programs running on their respective computers, and they have a client-server relationship with each other.

[0062] In view of the above-described embodiments of the subject matter, this application discloses the following list of examples, wherein a single feature of an example, or a combination of more than one feature of the example, and optionally a combination with one or more features of one or more other examples, are other examples that also fall within the scope of the disclosure of this application:

[0063] Example 1: A computer-implemented method comprising: receiving, at a compression service, measurement information about one or more columns of a database server; in response to receiving the measurement information, processing the measurement information by the compression service to identify at least one column as a candidate for compression by the compression service; in response to identifying the at least one column, compressing the at least one column by the compression service, wherein the compression service uses a hardware accelerator to perform a compression algorithm for compressing the at least one column; and notifying the database server of the at least one column compressed by the compression service.

[0064] Example 2: A computer-implemented method according to Example 1, wherein the compression service includes a hardware data processor that provides hardware acceleration for compression provided by the compression service.

[0065] Example 3: A computer-implemented method according to any one of Examples 1 to 2, wherein the hardware data processor includes one or more of the following: at least one field-programmable gate array (FPGA), at least one graphics processing unit, and at least one tensor processing unit.

[0066] Example 4: A computer-implemented method according to any one of Examples 1 to 3, wherein the compression service includes compression-as-a-service provided by a platform as a service provider.

[0067] Example 5: A computer-implemented method according to any one of Examples 1 to 4, wherein the service provider providing the compression service is the same platform as the service provider providing the database server.

[0068] Example 6: A computer-implemented method according to any one of Examples 1 to 5, wherein the same data center of the platform as a service provider provides compression services and a database server.

[0069] Example 7: A computer-implemented method according to any one of Examples 1 to 6, wherein different data centers of the platform, acting as a service provider, provide compression services and database servers.

[0070] Example 8: A computer-implemented method according to any one of Examples 1 to 7, wherein the platform that provides the compression service is a different platform from the platform that provides the database server.

[0071] Example 9: A computer-implemented method according to any one of Examples 1 to 8, wherein the database server includes an in-memory columnstore database.

[0072] Example 10: A computer-implemented method according to any one of Examples 1 to 9, wherein the measurement information includes one or more of the following: the size of at least one column in bytes, the access frequency of the size of at least one column, an access frequency threshold, and a size threshold.

[0073] Example 11: A computer-implemented method according to any one of Examples 1 to 10, wherein the process further includes identifying at least one column in response to the size of at least one column exceeding a size threshold.

[0074] Example 12: A computer-implemented method according to any one of Examples 1 to 11, wherein the process further includes processing at least one other column, and in response to a corresponding size of at least one other column being less than a size threshold, at least one other column is not identified as being compressed by a compression service.

[0075] Example 13: A computer-implemented method according to any one of Examples 1 to 12, wherein the hardware accelerator includes a floating-point gate array that provides a recursive pairing compression algorithm.

[0076] Example 14: A computer-implemented method according to any one of Examples 1 to 14, wherein the notification further includes a list of at least one table provided by a compression service and / or at least one column provided by a compression service.

[0077] Example 15: A system comprising: at least one processor; and

[0078] At least one memory includes program code that, when executed by at least one processor, causes operations including: receiving measurement information about one or more columns of a database server at a compression service; processing the measurement information by the compression service in response to receiving the measurement information to identify at least one column as a candidate for compression by the compression service; compressing at least one column by the compression service in response to identifying the at least one column, wherein the compression service uses a hardware accelerator to execute a compression algorithm for compressing at least one column; and notifying the database server of the at least one column compressed by the compression service.

[0079] Example 16: The system according to Example 15, wherein the compression service includes a hardware data processor that provides hardware acceleration for compression provided by the compression service.

[0080] Example 17: A system according to any one of Examples 16 to 16, wherein the hardware data processor includes one or more of the following: at least one field-programmable gate array (FPGA), at least one graphics processing unit, and at least one tensor processing unit.

[0081] Example 18: A system according to any one of Examples 16 to 18, wherein the compression service includes compression-as-a-service provided by a platform as a service provider.

[0082] Example 19: A system according to any one of Examples 16 to 19, wherein the database server includes an in-memory columnstore database.

[0083] Example 20: A non-transitory computer-readable storage medium including program code that, when executed by at least one processor, causes operations including: receiving measurement information about one or more columns of a database server at a compression service; in response to receiving the measurement information, processing the measurement information by the compression service to identify at least one column as a candidate for compression by the compression service; in response to identifying the at least one column, compressing the at least one column by the compression service, wherein the compression service uses a hardware accelerator to perform a compression algorithm for compressing the at least one column; and notifying the database server of the at least one column compressed by the compression service.

[0084] The embodiments described above do not represent all embodiments consistent with the subject matter described herein. Rather, they are merely examples of aspects consistent with the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. Specifically, further features and / or variations may be provided in addition to those set forth herein. For example, the above embodiments may be applicable to various combinations and sub-combinations of the disclosed features, and / or combinations and sub-combinations of several further features disclosed above. Furthermore, the logical flows depicted in the drawings and / or described herein do not necessarily require the specific order or sequential order shown to achieve the desired results. Other embodiments are within the scope of the appended claims.

Claims

1. A computer-implemented method, comprising: Receive measurement information about one or more columns of the database server from the compression service; In response to receiving the measurement information, the compression service processes the measurement information to identify at least one column as a candidate for compression by the compression service, wherein the at least one column is identified as a candidate for compression based on the access frequency of the at least one column being greater than an access frequency threshold. In response to identifying the at least one column, the compression service compresses the at least one column, wherein the compression service uses a hardware accelerator to perform a compression algorithm to compress the at least one column; and The compression service notifies the database server of the at least one column that has been compressed by the compression service.

2. The computer-implemented method according to claim 1, wherein, The compression service includes a hardware data processor that provides hardware acceleration for compression provided by the compression service.

3. The computer-implemented method according to claim 2, wherein, The hardware data processor includes one or more of the following: at least one field-programmable gate array (FPGA), at least one graphics processing unit, and at least one tensor processing unit.

4. The computer-implemented method according to claim 1, wherein, The compression service includes compression-as-a-service provided by the platform as a service provider.

5. The computer-implemented method according to claim 4, wherein, The platform used by the service provider offering compression services is the same platform used by the service provider offering database servers.

6. The computer-implemented method according to claim 5, wherein, The same data center on the platform, which acts as a service provider, offers compression services and database servers.

7. The computer-implemented method according to claim 5, wherein, The platform, acting as a service provider, offers compression services and database servers in its various data centers.

8. The computer-implemented method according to claim 4, wherein, The platform that provides compression services is different from the platform that provides database servers.

9. The computer-implemented method according to claim 1, wherein, The database server includes an in-memory column-store database.

10. The computer-implemented method according to claim 1, wherein, The metric information includes one or more of the following: the size of at least one column in bytes, the access frequency of the size of at least one column, an access frequency threshold, and a size threshold.

11. The computer-implemented method according to claim 1, wherein, The process also includes identifying at least one column in response to the size of at least one column exceeding a size threshold.

12. The computer-implemented method according to claim 9, wherein, The processing also includes processing at least one other column, and in response to the corresponding size of at least one other column being less than a size threshold, at least one other column is not identified as being compressed by the compression service.

13. The computer-implemented method according to claim 1, wherein, The hardware accelerator includes an FPGA that provides a recursive pairing compression algorithm.

14. The computer-implemented method according to claim 1, wherein, The notification also includes a list of at least one table that the compression service provides compression for and / or at least one column that provides compression for.

15. A system comprising: At least one processor; as well as At least one memory, including program code that, when executed by at least one processor, causes operations including: Receive measurement information about one or more columns of the database server from the compression service; In response to receiving the measurement information, the compression service processes the measurement information to identify at least one column as a candidate for compression by the compression service, wherein the at least one column is identified as a candidate for compression based on the access frequency of the at least one column being greater than an access frequency threshold. In response to identifying the at least one column, the compression service compresses the at least one column, wherein the compression service uses a hardware accelerator to perform a compression algorithm to compress the at least one column; and The compression service notifies the database server of the at least one column that has been compressed by the compression service.

16. The system according to claim 15, wherein, The compression service includes a hardware data processor that provides hardware acceleration for compression provided by the compression service.

17. The system according to claim 16, wherein, The hardware data processor includes one or more of the following: at least one field-programmable gate array (FPGA), at least one graphics processing unit, and at least one tensor processing unit.

18. The system according to claim 15, wherein, The compression service includes compression-as-a-service provided by the platform as a service provider.

19. The system according to claim 15, wherein, The database server includes an in-memory column-store database.

20. A non-transitory computer-readable storage medium comprising program code that, when executed by at least one processor, causes operations including: Receive measurement information about one or more columns of the database server from the compression service; In response to receiving the measurement information, the compression service processes the measurement information to identify at least one column as a candidate for compression by the compression service, wherein the at least one column is identified as a candidate for compression based on the access frequency of the at least one column being greater than an access frequency threshold. In response to identifying the at least one column, the compression service compresses the at least one column, wherein the compression service uses a hardware accelerator to perform a compression algorithm to compress the at least one column; and The compression service notifies the database server of the at least one column that has been compressed by the compression service.

Images