FPGA-accelerated database query methods, devices, equipment and media

Abstract

This invention provides an FPGA-accelerated database query method, apparatus, device, and medium, comprising: inputting a query SQL into a row parsing module in the FPGA hardware core to obtain a first data stream; inputting the first data stream into a filtering module to obtain a second data stream; inputting the second data stream into a calculation module to obtain a third data stream; and inputting the third data stream into a projection output module to obtain a target data table. The row parsing module is used to split the data table into column data based on the configuration corresponding to the query SQL and put it into the stream; the filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream; the calculation module is used to execute the function expression calculation corresponding to the query SQL on the input data stream; and the projection output module is used to encapsulate the input data stream into row data. This invention aims to improve the versatility of FPGA-accelerated database SQL queries.

Description

Technical Field

[0001] This invention relates to the field of database query technology, and in particular to an FPGA-accelerated database query method, apparatus, device, and medium. Background Technology

[0002] FPGAs are semi-custom circuits, requiring the computational logic to be programmed into the chip to complete a full function. After a database SQL query is translated into hardware, the hardware can only execute that query. However, if the query target clause or filter condition expression is changed, the hardware cannot perform the function of the modified SQL query. The hardware needs to be redeveloped and customized, meaning that FPGA hardware circuits are difficult to make universal.

[0003] Therefore, there is an urgent need for a method in the field of database querying that aims to improve the general ability of FPGA to accelerate database SQL queries. Summary of the Invention

[0004] This invention provides an FPGA-accelerated database query method, apparatus, device, and medium to address the shortcomings of existing technologies in FPGA's poor general-purpose capability for SQL database queries, thereby improving the general-purpose capability of FPGA in accelerating SQL database queries.

[0005] This invention provides an FPGA-accelerated database query method, comprising:

[0006] When a query SQL is received, the query SQL is input to the row parsing module in the FPGA hardware core to obtain the first data stream output by the row parsing module;

[0007] The first data stream is input to the filtering module in the FPGA hardware core to obtain the second data stream output by the filtering module;

[0008] The second data stream is input to the computing module in the FPGA hardware core to obtain the third data stream output by the computing module;

[0009] The third data stream is input to the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module;

[0010] The row parsing module is used to split the data table into column data and put it into the stream based on the configuration corresponding to the query SQL. The filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream. The operation module is used to perform the function expression operation corresponding to the query SQL on the input data stream. The projection output module is used to encapsulate the input data stream into row data and output it.

[0011] According to the FPGA-accelerated database query method provided by the present invention, the computing module includes a non-aggregate computing module and an aggregate computing module;

[0012] The second data stream is input to the computing module in the FPGA hardware core to obtain the third data stream output by the computing module, including:

[0013] The second data stream is input to the non-aggregation operation module to obtain the fourth data stream output by the non-aggregation operation module;

[0014] The fourth data stream is input to the aggregation operation module to obtain the third data stream output by the aggregation operation module;

[0015] The non-aggregate operation module is used to perform non-aggregate function expression operations on the input data stream, and the aggregate operation module is used to perform aggregate function expression operations on the input data stream.

[0016] According to the FPGA-accelerated database query method provided by the present invention, before the step of inputting the second data stream into the computing module in the FPGA hardware core to obtain the third data stream output by the computing module, the method includes:

[0017] The second data stream is input to the grouping module in the FPGA hardware core to obtain the fifth data stream output by the grouping module;

[0018] The fifth data stream is used as the second data stream;

[0019] The grouping module is used to execute a grouping clause on the input data stream when the query SQL contains a grouping clause.

[0020] According to the FPGA-accelerated database query method provided by the present invention, before inputting the third data stream to the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module, the method includes:

[0021] The third data stream is input to the group filtering module in the FPGA hardware core to obtain the sixth data stream output by the group filtering module;

[0022] The sixth data stream is used as the third data stream;

[0023] The group filtering module is used to execute the filtering condition clause on multiple input data streams.

[0024] According to an FPGA-accelerated database query method provided by the present invention, determining the configuration of each hardware kernel module of the FPGA hardware core includes:

[0025] Generate the query plan tree corresponding to the query SQL;

[0026] The query plan tree is traversed to determine the configuration of each hardware kernel module of the FPGA hardware core based on the query plan tree, and the configuration corresponds one-to-one with the hardware kernel module;

[0027] The query plan tree is used to represent the SQL expression corresponding to the query SQL and the execution order of the SQL expression, and the configuration is used to instruct each hardware kernel module to cooperate in executing the query SQL according to the corresponding configuration.

[0028] According to an FPGA-accelerated database query method provided by the present invention, the configuration of each hardware kernel module of the FPGA hardware core is determined based on the query plan tree, including:

[0029] Extract the filter clauses from the query plan tree, and generate the configuration required by the filtering module based on the filter clauses;

[0030] Based on the target list of the query plan tree, the configuration of the operation module is generated, wherein the target list includes the non-aggregate function expression operation and the aggregate function expression operation.

[0031] The present invention also provides an FPGA-accelerated database query device, comprising:

[0032] The first input module is used to input the query SQL into the row parsing module in the FPGA hardware core when a query SQL is received, so as to obtain the first data stream output by the row parsing module;

[0033] The second input module is used to input the first data stream to the filtering module in the FPGA hardware core to obtain the second data stream output by the filtering module.

[0034] The third input module is used to input the second data stream to the computing module in the FPGA hardware core to obtain the third data stream output by the computing module;

[0035] The fourth input module is used to input the third data stream to the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module;

[0036] The row parsing module is used to split the data table into column data and put it into the stream based on the configuration corresponding to the query SQL. The filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream. The operation module is used to perform the function expression operation corresponding to the query SQL on the input data stream. The projection output module is used to encapsulate the input data stream into row data and output it.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the FPGA-accelerated database query method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the FPGA-accelerated database query method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the FPGA-accelerated database query method as described above.

[0040] The present invention provides an FPGA-accelerated database query method, apparatus, device, and medium. During the SQL query process, the data table is first split into column data and placed into a stream. Then, filtering condition clauses are executed, along with non-aggregate function expression operations and aggregate function expression operations. Finally, the data is encapsulated into row data and output. By proposing an FPGA hardware kernel framework including a row parsing module, a filtering module, an operation module, and a projection output module, the SQL query is executed in the FPGA hardware circuit as a data stream between various modules. This is suitable for specific aggregation and scan queries. Even if the query target clause or filtering condition expression in the SQL query is changed, the hardware circuit can still complete the function of the changed SQL query, thereby improving the versatility of FPGA-accelerated database SQL queries. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts illustrating the FPGA-accelerated database query method provided by the present invention;

[0043] Figure 2This invention provides a database query SQL example;

[0044] Figure 3 This is the second flowchart of the FPGA-accelerated database query method provided by the present invention;

[0045] Figure 4 This is the third flowchart of the FPGA-accelerated database query method provided by the present invention;

[0046] Figure 5 This is the fourth flowchart of the FPGA-accelerated database query method provided by the present invention;

[0047] Figure 6 This is the fifth flowchart of the FPGA-accelerated database query method provided by the present invention;

[0048] Figure 7 This is the sixth flowchart of the FPGA-accelerated database query method provided by the present invention;

[0049] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] The following is combined with Figures 1-7 The present invention describes an FPGA-accelerated database query method.

[0052] This invention proposes an FPGA-accelerated database query method, comprising:

[0053] Step 10: When a query SQL is received, the query SQL is input into the row parsing module in the FPGA hardware core to obtain the first data stream output by the row parsing module;

[0054] The technology described in this invention is divided into host-side and device-side. The host-side runs on the CPU and mainly includes database software and acceleration system adaptation software (including FPGA device management, host-side and device-side memory migration, and device kernel execution functions). The device-side runs on the FPGA hardware, which mainly includes device kernel adaptation programs and device kernel programs. Figure 2 As shown.

[0055] This invention designs and implements an FPGA hardware kernel in the form of a configuration library, consisting of multiple modules. The FPGA hardware kernel framework includes a row parsing module, a filtering module, a computation module, and a projection output module. The hardware circuitry of the FPGA hardware kernel framework executes data streams between the modules, suitable for specific aggregation and scanning queries. The FPGA hardware kernel framework is deployed on FPGA hardware.

[0056] The row parsing module is used to split the data table into column data and put it into the stream based on the configuration corresponding to the query SQL. In other words, for row-stored data tables, it is responsible for splitting each row of data in the data table into column data according to the externally passed metadata configuration and putting it into the stream; for column-stored data tables, it is responsible for grabbing one row of each column of data according to its own length and putting it into the stream.

[0057] For example, the emp table actually uses 8 data stream channels c1-c8, and the correspondence between each channel and the data columns is as follows: Figure 2 As shown in the example, there is a maximum limit to the number of channels in a stream. The program configures the channels and the required data columns by passing in certain rules. Therefore, the channels of the stream and the columns of the entire data table may not necessarily have a one-to-one correspondence as shown in the example (the left side of the colon represents the channel, and the right side represents the column).

[0058] Step 20: Input the first data stream into the filtering module in the FPGA hardware core to obtain the second data stream output by the filtering module;

[0059] The filtering module executes filtering conditional clauses on its input data stream. Within the filtering module, only streams that meet the filtering conditions corresponding to the clause can proceed to the next module; streams that do not meet the conditions are discarded. These filtering conditional clauses can be either `quals` clauses or `where` clauses.

[0060] For example, taking the SQL: select deptno,avg(sal)avgs from emp where hiredate>'2022-09-01'group by deptnohaving avgs>2000 as an example, the SQL shown in the example executes the expression "c5>?", which determines whether the number in channel 5 is greater than the variable passed through external storage (in this example, the variable passed is '2022-09-01').

[0061] Step 30: Input the second data stream into the computing module in the FPGA hardware core to obtain the third data stream output by the computing module;

[0062] The computation module is used to sequentially perform expression calculations on the input data stream. This module is responsible for querying the target list corresponding to the SQL query, retrieving the data from the corresponding channel, executing the target expression specified in the target list, and then placing the result into the channel with the corresponding target number in the stream and passing it to the next module. If the target list does not specify a target expression, no calculation is performed, and the data from the original channel is directly placed into the channel with the corresponding target number in the stream and passed to the next module.

[0063] Step 40: Input the third data stream into the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module;

[0064] The row parsing module is used to split the data table into column data and put it into the stream based on the configuration corresponding to the query SQL. The filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream. The operation module is used to perform the function expression operation corresponding to the query SQL on the input data stream. The projection output module is used to encapsulate the input data stream into row data and output it.

[0065] Projection output module: Responsible for encapsulating the data in the data stream into a single line according to the input configuration and outputting it to external memory. It should be noted that whether this projection output module needs to output to external memory every time can be specified using configuration according to functional requirements.

[0066] The FPGA-accelerated database query method provided by this invention first splits the data table into column data and puts it into a stream during the SQL query process. Then, it executes the filtering condition clause, performs non-aggregate function expression operations and aggregate function expression operations, and finally encapsulates it into row data and outputs it. By proposing an FPGA hardware kernel framework including a row parsing module, a filtering module, an operation module and a projection output module, the SQL query is executed in the form of a data stream between various modules in the FPGA hardware circuit. It is suitable for specific aggregation and scan queries. Even if the query target clause or filtering condition expression in the query SQL is changed, the hardware circuit can still complete the function of the changed query SQL, thereby improving the versatility of FPGA-accelerated database SQL query.

[0067] In one embodiment, please refer to Figure 3 The computation module includes a non-aggregate computation module and an aggregate computation module;

[0068] Step 30: Input the second data stream into the arithmetic module in the FPGA hardware core to obtain the third data stream output by the arithmetic module, including:

[0069] Step 301: Input the second data stream into the non-aggregation operation module to obtain the fourth data stream output by the non-aggregation operation module;

[0070] Step 302: Input the fourth data stream into the aggregation operation module to obtain the third data stream output by the aggregation operation module;

[0071] The operation module is used to perform non-aggregate function expression operations on the input data stream, and the aggregation operation module is used to perform aggregate function expression operations on the input data stream.

[0072] The SQL query statement includes both non-aggregate function expression operations and aggregate function expression operations. The operation module is divided into a non-aggregate operation module and an aggregate operation module, which are used to execute the non-aggregate function expression operations and aggregate function expression operations respectively.

[0073] The non-aggregate operation module is used to perform non-aggregate function expression operations on its input data stream. This module is responsible for calculating the non-aggregate function expressions in the target list. It takes the corresponding channel number to execute the non-aggregate function expressions in the target list, and then puts the result into the channel corresponding to the target number in the stream and passes it to the next module.

[0074] If the target list does not contain any non-aggregate function expressions, no calculation is performed; the data from the original channel is directly placed into the channel corresponding to the target number in the stream and passed to the next module. In the query SQL, the clause between `SELECT` and `FROM` is the non-aggregate function expression. If the target list requires aggregation operations, additional intermediate calculations may need to be performed before the expressions in the target list are evaluated to ensure correct query execution. The results of these intermediate expression calculations are then used as input to perform further calculations on the non-aggregate expressions in the target list.

[0075] For example, if the target list contains a summation operation sum(saly*1.2), then the non-aggregation operation module only performs the intermediate operation saly*1.2, and then passes the result of saly*1.2 to the next module.

[0076] Aggregation Operation Module: This module is responsible for executing the corresponding aggregation operation process according to the input configuration. Supported aggregation operations include SUM, COUNT, AVG, and MAX. In particular, the aggregation operations actually executed by this module may need to be converted into other aggregation operations to achieve the purpose. For example, in the SQL example above, according to the hardware execution process, the hardware needs to calculate the cumulative sum of saly before obtaining the average, that is, the average aggregation operation needs to be converted into a sum aggregation operation and a counting aggregation operation. In the SQL example above, the target for number 2 is 'avg(saly)avgs'. The executor needs to convert the target for number 2 to 'sum(saly)' and the target for number 3 to 'count(saly)'. Number 2 requires an accumulation operation, and number 3 requires a counting operation.

[0077] In this embodiment, the computation module includes a non-aggregate computation module and an aggregate computation module, which are used to perform non-aggregate function expression operations and aggregate function expression operations respectively, thereby improving the efficiency of FPGA-accelerated database query.

[0078] In one embodiment, please refer to Figure 4 Before step 30, which involves inputting the second data stream into the computing module in the FPGA hardware core to obtain the third data stream output by the computing module, the following steps are included:

[0079] Step 50: Input the second data stream into the packet module in the FPGA hardware core to obtain the fifth data stream output by the packet module;

[0080] Step 60: Use the fifth data stream as the second data stream;

[0081] The grouping module is used to execute a grouping clause on the input data stream when the query SQL contains a grouping clause.

[0082] In this embodiment, the grouping module is used to execute the grouping clause on the input data stream when the query SQL contains a grouping clause. The grouping module is responsible for executing the grouping clause, which can be a group clause, a group by clause, etc.

[0083] For SQL queries containing GROUP BY clauses, this module splits the data stream into multiple groups after execution, adds group numbers as identifiers, and then passes the data stream to the next module. For SQL queries without GROUP BY clauses, it's equivalent to one group of data flowing to the next module. Internally, the grouping channels are configured by an external program and passed to the grouping module from external storage.

[0084] In this embodiment, a grouping module is set before the calculation module. For substatements in the query SQL that contain grouping clauses, the data stream is first grouped and processed by the added grouping module before the calculation, which further improves the efficiency of FPGA-accelerated database query.

[0085] In one embodiment, please refer to Figure 5 Before step 40, inputting the third data stream to the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module, the following steps are included:

[0086] Step 70: Input the third data stream into the group filtering module in the FPGA hardware core to obtain the sixth data stream output by the group filtering module;

[0087] Step 80: Use the sixth data stream as the third data stream;

[0088] The group filtering module is used to execute filtering conditional clauses on multiple input data streams.

[0089] In this embodiment, when the grouping module is enabled, the grouping filtering module executes filtering condition clauses on the multiple input data streams. These filtering condition clauses can be HAVING clauses. Generally, this module is configurable and selectable; that is, it is only enabled when the grouping module is used for grouping calculations.

[0090] In the example SQL above, there are two channels flowing into the group filtering module: channel c1 stores the deptno column, and channel c2 stores the avgs column. The group filtering module will execute the expression "c2>?" to filter, that is, to determine whether the number in channel c2 is greater than the variable passed in through external storage (in this example, the variable passed in is 2000).

[0091] In this embodiment, a grouping module is set before the projection output module and after the calculation module. When there is a grouping operation in the query SQL, the data stream is filtered separately by the corresponding added grouping filtering module, which improves the efficiency of FPGA-accelerated database query.

[0092] In one embodiment, please refer to Figure 6 Determine the configuration of each hardware kernel module of the FPGA hardware core, including:

[0093] Step 100: Generate the query plan tree corresponding to the query SQL;

[0094] Step 200: Traverse the query plan tree to determine the configuration of each hardware kernel module of the FPGA hardware core based on the query plan tree, wherein the configuration corresponds one-to-one with the hardware kernel module;

[0095] The query plan tree is used to represent the SQL expression corresponding to the query SQL and the execution order of the SQL expression, and the configuration is used to instruct each hardware kernel module to cooperate in executing the query SQL according to the corresponding configuration.

[0096] The specific process for configuring each hardware kernel module of the FPGA hardware core is as follows:

[0097] During the software initialization phase, the acceleration system adaptation software on the host side will register with the database to establish communication, request a memory pool, execute the device management module initialization program to obtain the FPGA device and the device-side kernel program, and perform device memory allocation, etc.

[0098] The acceleration system adaptation software on the host side communicates with the database. When a query SQL is received, the acceleration system adaptation software on the host side will attempt to generate a query plan tree that can be offloaded to the FPGA for acceleration. The database determines whether to use the FPGA hardware system to execute the query based on the cost of the query plan tree.

[0099] When FPGA-accelerated querying is deemed suitable, the query execution module of the acceleration system adaptation software will execute the query. Specifically, if the query cost corresponding to the query plan tree meets preset conditions, these preset conditions are used to determine whether FPGA-accelerated querying is suitable; if FPGA-accelerated querying is suitable, the query plan tree will be traversed.

[0100] During the query initialization phase, the host-side query execution module traverses the query plan tree and determines the configuration of each hardware kernel module of the FPGA hardware core based on the query plan tree. The generated configuration corresponds one-to-one with the hardware kernel module.

[0101] Based on the query plan tree, after determining the configuration of each hardware kernel module, the process includes: encapsulating the configuration into a host memory block, and mapping and migrating the configuration to the device memory block of the FPGA hardware kernel based on the host memory block; wherein, the device memory block is used by the hardware kernel module to obtain the corresponding configuration. Specifically, a memory block of a specific size is requested from the memory pool, the query data table is scanned and read to obtain the table tuples, and encapsulated into the memory block in row or column format; a memory block for storing the execution results is requested from the memory pool; host memory is mapped and migrated to device memory; the device kernel execution management module interface is called to start the hardware kernel execution, and the execution completion event is waited for; after the hardware kernel event returns, the memory migration module interface is called to migrate the memory block storing the execution results from the device side back to the host side; the query execution module parses the result data structure and returns the result to the database.

[0102] In this embodiment, the various hardware kernel modules of the FPGA hardware core are configured through a query plan tree, so that each hardware kernel module of the FPGA hardware core can perform SQL queries according to the corresponding configuration, thereby improving the versatility of FPGA-accelerated database queries.

[0103] In one embodiment, please refer to Figure 7 Step 200: Based on the query plan tree, determine the configuration of each hardware kernel module of the FPGA hardware core, including:

[0104] Step 201: Extract the filter condition clauses of the query plan tree, and generate the configuration required by the filtering module based on the filter condition clauses;

[0105] Step 202: Based on the target list of the query plan tree, generate the configuration of the operation module. The target list includes the non-aggregate function expression operation and the aggregate function expression operation.

[0106] During the query initialization phase, the query execution module traverses the query plan tree and, based on the tree, performs configuration generation operations for each hardware kernel module. Specifically, it extracts the filter clauses from the query plan tree and generates the Tree structure configuration file required by the filtering module. Filter conditions can be qual clause statements. For queries containing grouping clauses, it generates the configuration for the grouping module. For queries containing grouping clauses with corresponding filter conditions, it generates the configuration for the grouping filtering module. For queries containing computational expressions, it generates the configuration for the computational module based on the target list of the query plan tree. The target list contains computational expressions, which can include non-aggregate function expressions and aggregate function expressions. Finally, it reads the database system tables and, based on the system table metadata, generates the configuration required by the row parsing module.

[0107] It should be noted that the query plan tree is a data structure defined by the database. All the information required for the configuration of each FPGA hardware kernel module can be obtained from the query plan tree structure. That is, each clause is obtained from the query plan tree data structure and converted into the configuration of each hardware kernel module.

[0108] Based on the query SQL, a query tree (also a data structure) is first obtained, and then a plan tree is generated from the query tree. The plan tree is a hierarchical structure that stores the plan nodes (also a data structure) required to execute the query SQL. In addition, each plan node contains an expression field, a projection list (the target list mentioned is also a projection list), a node type, pointers to the left / right subtrees, and several extended fields.

[0109] In this embodiment, a query plan tree is used to generate configurations for each hardware kernel module of the FPGA hardware core, so that each hardware kernel module of the FPGA hardware core can perform SQL queries according to the corresponding configuration, thereby improving the versatility of FPGA-accelerated database queries.

[0110] The configuration generation function is handled by the configuration management program provided in the device kernel execution management module. This program provides interfaces for generating the configurations required by various modules of the hardware kernel. To better guide the operation of these modules, specific data structures and message formats are used.

[0111] The format used in this invention pre-allocates 4096 bytes (1 page) to each hardware kernel module. The memory between the hardware kernel modules is contiguous. Therefore, the entire configuration is passed to the external memory of the hardware kernel as a single parameter and is actively read by the hardware kernel during execution.

[0112] The configuration message format consists of three parts: a mask, reserved fields, and the body content. The mask occupies 8 bytes; the purpose of the mask varies between different hardware kernel modules, and the actual number of bits used also differs. The body content occupies 4064 bytes; different hardware kernel modules have different body message formats, and the information they transmit also differs.

[0113] This invention guides the various hardware kernel modules to cooperate and complete their tasks through configuration; therefore, the configuration message format has significant design requirements. The following lists the main message content and functions of each hardware kernel module:

[0114] (1) Line parsing module:

[0115] Mask: Each bit represents a column of the table, with a maximum support of 64 columns. A bit of 1 indicates the column number to be used in query operations or projections.

[0116] Target column count: The total number of columns required for query operations or projections. By specifying the target column count, you can stop parsing the remaining columns once all target columns have been parsed and retrieved.

[0117] Column metadata includes the length of each column (the number of bytes occupied by each column, variable length is -1), the type ID of each column, column alignment, etc. Column metadata indicates how to parse the columns needed to obtain one row. It is suitable for both row-oriented and column-oriented storage.

[0118] Channel number: Used to indicate which channel (channel number or channel ID) in the data stream each column should be handed over to the next module after parsing the columns required for a row.

[0119] (2) Filtering module:

[0120] Mask: Indicates whether filtering should be performed. 0x0 indicates no filtering, other values ​​indicate filtering.

[0121] The Tree header includes the number, offset, and size of allocated Tree structures. It indicates how to read Tree entities. Tree is a private, undocumented binary tree-like structure that allows for the conversion between complex expressions and Tree structure forms, guiding the kernel on how to flexibly execute expressions.

[0122] Tree body: Stores the contents of the Tree structure, which is used to represent a set of expressions (chain).

[0123] (3) Grouping module:

[0124] Mask: Each bit represents a channel of the table. Theoretically, it supports 64 channels, but in practice, depending on the hardware design, it generally supports a maximum of 8 to 16 channels. A bit of 1 indicates that the data from this channel needs to be grouped.

[0125] Grouping Mode: Passes a grouping mode identifier. Supported modes include hash grouping and value grouping modes. The purpose is to improve performance by selecting an appropriate grouping method based on the type of the grouping column.

[0126] Matching pattern: Passes the matching pattern identifier. Supported patterns include full match, 1-character prefix match, 2-character prefix match, 3-character prefix match, 1-character suffix match, etc., with the aim of improving hardware kernel performance.

[0127] Partition table: Indicates whether partitioning is enabled. 0 indicates disabled, otherwise enabled. When the number of groups is large, partitioning is used to quickly find the mapping relationship between channels and group numbers.

[0128] Partition mode: Indicates how partitions are used. Effective when partitioning is enabled.

[0129] Partition matching mode: Indicates how to match partitions. Effective when partitions are enabled.

[0130] (4) Non-aggregate operation module:

[0131] Mask: Indicates whether basic operations are required. 0x0 indicates no operation is required, and other values ​​indicate they are required.

[0132] Target header: Includes the number of Target structures allocated, offset, size, etc. It indicates how to read the Target body.

[0133] Target body: may contain multiple Target structures, each Target structure identifies what operation needs to be performed (represented by a Tree structure) and which channel the result of the operation is placed in.

[0134] (5) Aggregation operation module:

[0135] Mask: Indicates whether aggregation operations are needed. 0x0 indicates no aggregation is needed, and other values ​​indicate they are needed.

[0136] Target header: Includes the number of Target structures allocated, offset, size, etc. It indicates how to read the Target body.

[0137] Target body: May contain multiple Target structures. Each Target structure identifies the aggregation operation to be performed on the input channel and which channel the result of the aggregation operation is placed into.

[0138] (6) Group filtering module:

[0139] Mask: Indicates whether post-group filtering should be performed. 0x0 indicates no filtering is needed, and other values ​​indicate yes.

[0140] Tree header: Includes the number of tree structures allocated, offsets, size, etc. It indicates how to read tree entities.

[0141] Tree body: Stores the contents of the Tree structure, which is used to represent a set of expressions (chain).

[0142] (7) Projection output module:

[0143] Mask: Indicates whether projection is needed. 0x0 indicates no projection is needed; otherwise, projection is needed.

[0144] Write Enable: Indicates whether writing to external memory is required. 0x0 indicates no write is required, other values ​​indicate write is required.

[0145] Storage format: Indicates the projection encapsulation format, such as columnar storage or row storage. Valid only when projection is required.

[0146] Number of columns: The number of columns per row, specifying the number of elements in the sequential sequence of columns.

[0147] The column sequence is an array where each element stores a channel number. During encapsulation, the data of the channels specified by the element order must be encapsulated.

[0148] Alignment method: The method of encapsulation alignment.

[0149] The FPGA-accelerated database query device provided by the present invention is described below. The FPGA-accelerated database query device described below and the FPGA-accelerated database query method described above can be referred to in correspondence.

[0150] The FPGA-accelerated database query device proposed in this invention includes:

[0151] The first input module is used to input the query SQL into the row parsing module in the FPGA hardware core when a query SQL is received, so as to obtain the first data stream output by the row parsing module;

[0152] The second input module is used to input the first data stream to the filtering module in the FPGA hardware core to obtain the second data stream output by the filtering module.

[0153] The third input module is used to input the second data stream to the computing module in the FPGA hardware core to obtain the third data stream output by the computing module;

[0154] The fourth input module is used to input the third data stream to the projection output module in the FPGA hardware core to obtain the target data table output by the projection output module;

[0155] The row parsing module is used to split the data table into column data and put it into the stream based on the configuration corresponding to the query SQL. The filtering module is used to execute filtering condition clauses on the input data stream. The operation module is used to perform non-aggregate function expression operation and aggregate function expression operation on the input data stream in sequence. The projection output module is used to encapsulate the input data stream into row data and output it.

[0156] Furthermore, the computation module includes a non-aggregate computation module and an aggregate computation module;

[0157] The third input module is also used for:

[0158] The second data stream is input to the non-aggregation operation module to obtain the fourth data stream output by the non-aggregation operation module;

[0159] The fourth data stream is input to the aggregation operation module to obtain the third data stream output by the aggregation operation module;

[0160] The non-aggregate operation module is used to perform non-aggregate function expression operations on the input data stream, and the aggregate operation module is used to perform aggregate function expression operations on the input data stream.

[0161] Furthermore, the FPGA-accelerated database query device also includes a fifth input module, used for:

[0162] The second data stream is input to the grouping module in the FPGA hardware core to obtain the fifth data stream output by the grouping module;

[0163] The fifth data stream is used as the second data stream;

[0164] The grouping module is used to execute a grouping clause on the input data stream when the query SQL contains a grouping clause.

[0165] Furthermore, the FPGA-accelerated database query device also includes a sixth input module, used for:

[0166] The third data stream is input to the group filtering module in the FPGA hardware core to obtain the sixth data stream output by the group filtering module;

[0167] The sixth data stream is used as the third data stream;

[0168] The group filtering module is used to execute the filtering condition clause on multiple input data streams.

[0169] Furthermore, the FPGA-accelerated database query device also includes a configuration module for:

[0170] Generate the query plan tree corresponding to the query SQL;

[0171] The query plan tree is traversed to determine the configuration of each hardware kernel module of the FPGA hardware core based on the query plan tree, and the configuration corresponds one-to-one with the hardware kernel module;

[0172] The query plan tree is used to represent the SQL expression corresponding to the query SQL and the execution order of the SQL expression, and the configuration is used to instruct each hardware kernel module to cooperate in executing the query SQL according to the corresponding configuration.

[0173] Furthermore, the configuration module is also used for:

[0174] Extract the filter clauses from the query plan tree, and generate the configuration required by the filtering module based on the filter clauses;

[0175] Based on the target list of the query plan tree, the configuration of the operation module is generated, wherein the target list includes the non-aggregate function expression operation and the aggregate function expression operation.

[0176] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logic instructions in the memory 830 to execute an FPGA-accelerated database query method. This method includes: when a query SQL is received, inputting the query SQL to a row parsing module in the FPGA hardware core to obtain a first data stream output by the row parsing module; inputting the first data stream to a filtering module in the FPGA hardware core to obtain a second data stream output by the filtering module; inputting the second data stream to a computation module in the FPGA hardware core to obtain a third data stream output by the computation module; and inputting the third data stream to a projection output module in the FPGA hardware core to obtain a target data table output by the projection output module. The row parsing module is used to split the data table into column data based on the configuration corresponding to the query SQL and place it into a stream; the filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream; the computation module is used to execute the function expression operation corresponding to the query SQL on the input data stream; and the projection output module is used to encapsulate the input data stream into row data and output it.

[0177] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the FPGA-accelerated database query method provided by the above methods. The method includes: when a query SQL is received, inputting the query SQL into a row parsing module in the FPGA hardware core to obtain a first data stream output by the row parsing module; inputting the first data stream into a filtering module in the FPGA hardware core to obtain a second data stream output by the filtering module; inputting the second data stream into a computation module in the FPGA hardware core to obtain a third data stream output by the computation module; and inputting the third data stream into a projection output module in the FPGA hardware core to obtain a target data table output by the projection output module. The row parsing module is used to split the data table into column data based on the configuration corresponding to the query SQL and put it into a stream; the filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream; the computation module is used to execute the function expression operation corresponding to the query SQL on the input data stream; and the projection output module is used to encapsulate the input data stream into row data and output it.

[0179] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the FPGA-accelerated database query method provided by the above methods. The method includes: when a query SQL is received, inputting the query SQL to a row parsing module in the FPGA hardware core to obtain a first data stream output by the row parsing module; inputting the first data stream to a filtering module in the FPGA hardware core to obtain a second data stream output by the filtering module; inputting the second data stream to a computation module in the FPGA hardware core to obtain a third data stream output by the computation module; and inputting the third data stream to a projection output module in the FPGA hardware core to obtain a target data table output by the projection output module. The row parsing module is used to split the data table into column data based on the configuration corresponding to the query SQL and place it into a stream; the filtering module is used to execute the filtering condition clause corresponding to the query SQL on the input data stream; the computation module is used to execute the function expression operation corresponding to the query SQL on the input data stream; and the projection output module is used to encapsulate the input data stream into row data and output it.

[0180] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for FPGA-accelerated database query, the method comprising: The method comprises the following steps: when a query SQL is received, inputting the query SQL into a row parsing module in an FPGA hardware kernel to obtain a first data stream output by the row parsing module; inputting the first data stream into a filtering module in the FPGA hardware kernel to obtain a second data stream output by the filtering module; inputting the second data stream into an operation module in the FPGA hardware kernel to obtain a third data stream output by the operation module; inputting the third data stream into a projection output module in the FPGA hardware kernel to obtain a target data table output by the projection output module; wherein the row parsing module is configured to split a data table into column data form based on a configuration corresponding to the query SQL and put it into a stream, the filtering module is configured to execute a filter condition clause corresponding to the query SQL on an input data stream, the operation module is configured to execute a function expression operation corresponding to the query SQL on the input data stream, and the projection output module is configured to encapsulate the input data stream as row data and output; determining configurations of each hardware kernel module of the FPGA hardware kernel, comprising: generating a query plan tree corresponding to the query SQL; traversing the query plan tree to determine, based on the query plan tree, the configurations of each hardware kernel module of the FPGA hardware kernel, which correspond one-to-one to the hardware kernel modules; wherein the query plan tree is configured to represent a SQL expression corresponding to the query SQL and an execution order of the SQL expression, and the configurations are configured to instruct the hardware kernel modules to execute the query SQL in cooperation according to the corresponding configurations; determining the configurations of each hardware kernel module of the FPGA hardware kernel based on the query plan tree, comprising: extracting a filter condition clause of the query plan tree and generating a configuration required by the filtering module based on the filter condition clause; generating a configuration of the operation module based on a target list of the query plan tree, the target list comprising non-aggregated function expression operations and aggregated function expression operations.

2. The FPGA-accelerated database query method of claim 1, wherein, The operation module comprises a non-aggregated operation module and an aggregated operation module; inputting the second data stream into the operation module in the FPGA hardware kernel to obtain a third data stream output by the operation module, comprising: inputting the second data stream into the non-aggregated operation module to obtain a fourth data stream output by the non-aggregated operation module; inputting the fourth data stream into the aggregated operation module to obtain a third data stream output by the aggregated operation module; wherein the non-aggregated operation module is configured to execute non-aggregated function expression operations on an input data stream, and the aggregated operation module is configured to execute aggregated function expression operations on the input data stream.

3. The FPGA-accelerated database query method of claim 1, wherein, Before the step of inputting the second data stream into the operation module in the FPGA hardware kernel to obtain a third data stream output by the operation module, comprising: inputting the second data stream into a grouping module in the FPGA hardware kernel to obtain a fifth data stream output by the grouping module; taking the fifth data stream as the second data stream; The grouping module is configured to execute a grouping clause on the input data stream when the query SQL contains the grouping clause.

4. The FPGA-accelerated database query method of claim 3, wherein, Before the third data stream is input into a projection output module in the FPGA hardware kernel to obtain a target data table output by the projection output module, the method comprises: inputting the third data stream into a grouping filtering module in the FPGA hardware kernel to obtain a sixth data stream output by the grouping filtering module; regarding the sixth data stream as the third data stream; The grouping filtering module is configured to execute the filtering condition clause on the input multiple groups of data streams.

5. An FPGA-accelerated database query apparatus, comprising: The method comprises: a first input module configured to input the query SQL into a row parsing module in the FPGA hardware kernel to obtain a first data stream output by the row parsing module when the query SQL is received; a second input module configured to input the first data stream into a filtering module in the FPGA hardware kernel to obtain a second data stream output by the filtering module; a third input module configured to input the second data stream into an operation module in the FPGA hardware kernel to obtain a third data stream output by the operation module; a fourth input module configured to input the third data stream into a projection output module in the FPGA hardware kernel to obtain a target data table output by the projection output module; The row parsing module is configured to split a data table into column data form based on a configuration corresponding to the query SQL and put the column data into a stream, the filtering module is configured to execute a filtering condition clause corresponding to the query SQL on the input data stream, the operation module is configured to execute a function expression operation corresponding to the query SQL on the input data stream, and the projection output module is configured to encapsulate the input data stream into row data and output the row data. The configurations of the hardware kernel modules of the FPGA hardware kernel are determined, comprising: generating a query plan tree corresponding to the query SQL; traversing the query plan tree to determine the configurations of the hardware kernel modules of the FPGA hardware kernel based on the query plan tree, the configurations corresponding to the hardware kernel modules one by one; The query plan tree is configured to represent a SQL expression corresponding to the query SQL and an execution order of the SQL expression, and the configurations are configured to instruct the hardware kernel modules to execute the query SQL in cooperation according to the corresponding configurations. The configurations of the hardware kernel modules of the FPGA hardware kernel are determined based on the query plan tree, comprising: extracting a filtering condition clause of the query plan tree, and generating a configuration required by the filtering module based on the filtering condition clause; generating a configuration of the operation module based on a target list of the query plan tree, the target list comprising non-aggregation function expression operations and aggregation function expression operations.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the FPGA-accelerated database query method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the FPGA-accelerated database query method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the FPGA-accelerated database query method according to any one of claims 1 to 4.

Images