Database query method and device, equipment, medium and product

Abstract

The invention discloses a database query method and device, equipment, a medium and a product, and the method comprises the steps: analyzing a received database query instruction to obtain a corresponding query operator, and judging whether the query operator meets a preset grouping aggregation row limiting condition or not, the query operators at least comprise a grouping aggregation operator, a row limitation correlation operator and a projection operator; if the query operator meets the grouping aggregation row limiting condition, obtaining the to-be-queried data in batches through the projection operator, and completing the construction and data storage calculation of a preset row number of groups in the batch access process through the grouping aggregation operator; and continuing to store and calculate the to-be-queried data newly acquired by the projection operator based on the existing groups, and uploading the stored grouped data to the line limitation related operator through the group aggregation operator. Through the technical features, resource waste is reduced in the database query process, the execution time is shortened, and efficient execution of the SQL statement is achieved.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a database query method, apparatus, device, medium, and product. Background Technology

[0002] In the execution mechanism of Structured Query Language (SQL), the GROUP BY clause can group and aggregate query results by specified columns. If the grouping item or aggregate function parameter is not a base table column, the grouping column needs to be preprocessed by the projection operator (PRJT). The TOPN clause can filter the first N rows of data in the result set. When the two are combined, the semantics are to return the first N groups of data after grouping.

[0003] In existing technologies, when GROUP BY and TOPN clauses coexist, the database first scans the base table through the storage engine to obtain the full data. This data is then projected to the subquery join module to complete the join calculation. Next, the full data is fed into the Grouping Aggregation Operator (HAGR) to perform set function operations on all groups. Finally, the TOPN operator selects the top N groups from all grouping results and outputs them. However, even if only the top N groups are needed, existing solutions still require performing join calculations and full grouping aggregation on the full base table data. This not only generates a large number of meaningless calculations, resulting in a serious waste of computing resources, but also significantly reduces the execution efficiency of SQL statements due to the extended execution time caused by the full data processing flow, making it unsuitable for query performance requirements in scenarios with large data volumes. Summary of the Invention

[0004] This invention provides a database query method, apparatus, device, medium, and product that reduces resource waste and execution time during database querying, thereby achieving efficient execution of SQL statements.

[0005] In a first aspect, embodiments of this disclosure provide a database query method, including:

[0006] The received database query instruction is parsed to obtain the corresponding query operator, and it is determined whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least grouping and aggregation operators, row restriction related operators, and projection operators.

[0007] If the query operator satisfies the grouping and aggregation row restriction conditions, the projection operator obtains the data to be queried in batches. The grouping and aggregation operator completes the construction and data storage calculation of a preset number of groups during the batch data retrieval process. Then, based on the existing groups, the newly obtained data to be queried by the projection operator is stored and calculated. Finally, the grouping and aggregation operator uploads the stored group data to the row restriction related operator.

[0008] Secondly, embodiments of this disclosure provide a database query apparatus, comprising:

[0009] The condition determination module is used to parse the received database query instruction, obtain the corresponding query operator, and determine whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least the grouping and aggregation operator, the row restriction related operator, and the projection operator.

[0010] The data query module is used to obtain the data to be queried in batches through the projection operator if the query operator meets the grouping and aggregation row restriction conditions, complete the construction and data storage calculation of a preset number of groups during the batch data retrieval process through the grouping and aggregation operator, and then continue to store and calculate the newly obtained data to be queried by the projection operator based on the existing groups, and upload the stored group data to the row restriction related operator through the grouping and aggregation operator.

[0011] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0012] At least one processor; and

[0013] A memory that is communicatively connected to at least one processor; wherein,

[0014] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform a database query method provided in the first aspect embodiment described above.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement a database query method provided in the first aspect of the embodiments described above.

[0016] Fifthly, this disclosure provides a computer program product, which includes a computer program that, when executed by a processor, implements a database query method provided in the first aspect of the embodiment.

[0017] The technical solution of this invention involves parsing a received database query instruction to obtain the corresponding query operator, and determining whether the query operator meets preset grouping and aggregation row restriction conditions. The query operator includes at least a grouping and aggregation operator, row restriction-related operators, and a projection operator. If the query operator meets the grouping and aggregation row restriction conditions, the projection operator acquires the data to be queried in batches. During the batch data retrieval process, the grouping and aggregation operator completes the construction and data storage calculation of several preset rows of groups. Then, based on the existing groups, the newly acquired data to be queried by the projection operator is stored and calculated. Finally, the grouping and aggregation operator uploads the stored grouped data to the row restriction-related operators. Through these technical features, resource waste is reduced and execution time is shortened during database queries, achieving efficient execution of SQL statements.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a database query method provided in an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of another database query method provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of a database query device provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] In Structured Query Language (SQL), using the GROUP BY clause groups all query results by one or more values, with equal values ​​grouped together. If the GROUP BY terms or aggregate function parameters are not base table columns, a projection operator (PRJT) is generated below the GROUP operator to calculate the value of the grouping column. Furthermore, when using the TOP N or LIMIT N row restriction clauses (hereinafter collectively referred to as the TOPN clause) to filter results, the database returns the first N rows of the result set.

[0027] When both the GROUP BY clause and the TOPN clause appear in a single SQL statement, the semantics are: after grouping the result set according to the GROUP BY clause, return the first N groups of data in the grouped result set. Since only the first N groups are ultimately needed, the data beyond the first N groups does not actually require further grouping and calculation.

[0028] For example:

[0029] CREATE TABLE T1 AS SELECT LEVEL C1, LEVEL C2 FROM DUAL CONNECT BYLEVEL <= 1000;

[0030] CREATE TABLE T2 AS SELECT LEVEL D1,LEVEL D2 FROM DUAL CONNECT BYLEVEL <= 100;

[0031] Execute the following SQL statement with a GROUP BY clause and a TOPN clause:

[0032] SELECT TOP 3

[0033] SUM(SELECT D1 FROM T2 WHERE T2.D2 = T1.C1) ,C2

[0034] FROM

[0035] T1 GROUP BY C2;

[0036] The plan for this statement is as follows:

[0037] 1 #NSET2: [1, 3->3, 8];

[0038] 2 #PIPE2: [1, 3->3, 8];

[0039] 3 #PRJT2: [1, 3->3, 8]; exp_num(2), is_atom(FALSE);

[0040] 4 #TOPN2: [1, 3->3, 8];

[0041] 5 #HAGR2: [1, 3->3, 8]; grp_num(1), sfun_num(1);

[0042] 6 #PRJT2: [1, 1000->1000, 8]; exp_num(2), is_atom(FALSE);

[0043] 7 #CSCN2: [1, 1000->1000, 8]; INDEX33555472(T1); btr_scan(1);

[0044] 8 #SPL2: [1, 1->0, 8]; key_num(1), spool_num(0), has_variable(1), sites(-);

[0045] 9 #PRJT2: [1, 1->100, 8]; exp_num(1), is_atom(TRUE);

[0046] 10 #SLCT2: [1, 2->100, 8]; T2.D2 = var1;

[0047] 11 #CSCN2: [1, 100->100000, 8]; INDEX33555474(T2); btr_scan(1).

[0048] Among them, CSCN means scanning the table (for example, CSCN[7] means "scanning table T1"); PRJT means performing calculations / transferring data (for example, PRJT[6] means transferring the data of T1 to the subquery); SPL means processing the subquery (for example, using C1 of T1 to find the matching D1 in T2); HAGR means grouping and aggregation calculation (the operation corresponding to the GROUP BY clause and the aggregate function, calculating the sum of each group); TOPN means taking the first N groups (for example, taking the first 3 groups). T1 and T2 are the source data tables of the data to be queried, C1 and C2 are the base table columns of the T1 data table, and D1 and D2 are the base table columns of the T2 data table.

[0049] The execution logic of this plan is as follows:

[0050] 1. The clustered index scan operator CSCN[7] (for ease of understanding, it is explained here that the numbers in square brackets are the row numbers of the execution plan, and CSCN[7] means that the operator is CSCN in the 7th row) retrieves the data of columns C1 and C2 in batches from table T1 through the primary key scan and passes them to the projection operator PRJT[6] to perform step 2.

[0051] 2.PRJT[6] passes the C1 column of each row of data as a variable to the temporary table operator SPL[8] for subquery related calculations, and then passes it to the grouping operator HAGR[5] for step 3.

[0052] 3. HAGR[5] performs group calculations, saves the group values ​​to a hash table, and calculates the results of set function operations on each group of data. Then, it continues to return to step 1 to scan the data until all the data in table T1 is retrieved, and then proceeds to step 4.

[0053] 4. The TOPN[4] operator takes the first 3 groups of data from all the groups calculated by HAGR[5] and outputs them upwards.

[0054] According to the processing logic of the above conventional method, table T1 needs to transfer all 1,000 rows of data to SPL[8], and then perform equality matching calculations with each row of table T2. This requires 1,000 x 100 calculations, totaling 100,000 times. In addition, HAGR[5] needs to calculate the set function results of all groups, resulting in extremely high computational costs. In fact, after the first batch of data is calculated, the grouping column C2 of table T1 has already obtained the number of groups (3 groups) that meet the TOPN clause conditions. Even if subsequent data undergoes grouping operations and set function operations, it will be filtered out by the TOPN[4] operator, which will not only greatly increase the time consumption and reduce performance, but also cause resource waste.

[0055] Based on the above, this invention proposes an optimization method to improve the performance of grouping execution of database SQL statements. Once the grouping operator corresponding to the GROUP BY clause has obtained the number of groups satisfying the TOPN clause, it no longer creates new groups, thus improving the execution performance of the grouping operator. If there is a projection operator (PRJT) below the grouping operator, for data that does not require grouping, there is no need to perform related instructions on the projection operator.

[0056] In one embodiment, Figure 1 This is a flowchart of a database query method provided by an embodiment of the present invention. This embodiment can be applied to generating an execution plan and execution phase during the database query process. The method can be executed by a database query device, which can be implemented in hardware and / or software.

[0057] like Figure 1 As shown, the method includes:

[0058] S101. Parse the received database query instruction to obtain the corresponding query operator, and determine whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least the grouping and aggregation operator, the row restriction related operator, and the projection operator.

[0059] In this embodiment, a database query command refers to a query request initiated by a user or application to the database, typically in the form of an SQL statement (e.g., "SELECT grouping column, SUM(statistical column) FROM table GROUP BY grouping column LIMIT N"), which is the input command that triggers subsequent data processing. Query operators are core components generated after the database executes a database query command, used to complete specific data processing functions. These include grouping aggregation operators, row restriction operators, and projection operators. Specifically, the grouping aggregation operator HAGR is used to group data according to a specified grouping column and perform aggregation calculations; the row restriction operators TOPN or TOPN-PRJT are used to limit the number of rows in the query results; and the projection operator PRJT is used to preprocess the obtained data, such as selecting specified columns or performing simple calculations. The preset grouping and aggregation row restriction conditions can be understood as the trigger conditions pre-set by the database to determine whether to start the optimization logic of this scheme. They are: the parent operator of the grouping and aggregation operator is a row restriction related operator and the parent operator of the projection operator is a grouping and aggregation operator, and the grouping item of the grouping and aggregation operator is a base table column.

[0060] Specifically, the database receives database query commands initiated by users, parses the commands, transforms them into a set of executable query operators, and determines the type and hierarchical relationship of each operator. Based on preset grouping and aggregation row constraints, the parsed query operators are validated. First, it checks whether the parent operator of the grouping and aggregation operator HAGR is a row constraint-related operator TOPN or TOPN-PRJT. If so, it checks whether the parent operator of the projection operator PRJT is the grouping and aggregation operator HAGR and whether the grouping item of HAGR is a base table column. If the preset grouping and aggregation row constraints are met, the optimization process in S102 is executed; otherwise, the regular query process is executed.

[0061] S102. If the query operator meets the grouping and aggregation row restriction conditions, the data to be queried is obtained in batches through the projection operator. During the batch data retrieval process, the grouping and aggregation operator completes the construction and data storage calculation of the preset number of groups. Then, based on the existing groups, the newly obtained data to be queried by the projection operator is stored and calculated. The stored group data is uploaded to the row restriction related operator through the grouping and aggregation operator.

[0062] In this embodiment, the data to be queried refers to the raw or pre-processed dataset obtained by the projection operator from the underlying data source (such as a base table, storage engine, or underlying operator) during the execution of the query command. Grouped data refers to the complete grouped business results, composed of the values ​​of the base table columns and the corresponding aggregation calculation results.

[0063] Specifically, after confirming that the query operator meets the grouping and aggregation row limit conditions, the projection operator sequentially retrieves the raw or pre-processed query data from the lower-level data source (base table, storage engine, or lower-level operator CSCN) according to a preset batch size. After retrieving each batch of query data, the projection operator first checks if the batch is empty. If it is empty, it means that all query data has been processed, and the result upload process of the grouping and aggregation operator is triggered. If it is not empty, it enters the grouping number threshold determination stage, executing different processing strategies based on whether the current grouping and aggregation operator's grouping number has reached the preset row number. If the current grouping number of the grouping and aggregation operator has not reached the preset row number, it is in the grouping construction stage. The projection operator performs full instruction calculations (such as field operations, column filtering, and other preprocessing) on ​​the current batch of query data, and then passes all calculation results to the grouping and aggregation operator. These calculation results of the query data will be used as the construction material for new groups. The grouping and aggregation operator extracts the base table column values, generates group identifiers, and creates new groups, while simultaneously completing the storage of the corresponding aggregation calculation results. If the current number of groups in the grouping aggregation operator has reached the preset number of rows, it enters the grouping maintenance phase. At this time, the projection operator first calculates the grouping index (such as a hash value) based on the values ​​of the base table columns in the data to be queried, and matches it with the grouping hash table maintained by the grouping aggregation operator to verify the grouping affiliation of the data to be queried. It only executes the instruction calculation on the data to be queried that belongs to an existing group, skipping the data to be queried that does not belong to an existing group to avoid invalid calculations. Then, the filtered calculation results are passed to the grouping aggregation operator to update the aggregation calculation results of the existing groups. Finally, after all the data to be queried has been traversed, the grouping aggregation operator uploads the stored grouped data to the row limit related operators.

[0064] This invention provides a database query method that includes parsing a received database query instruction to obtain the corresponding query operator, and determining whether the query operator meets preset grouping and aggregation row restriction conditions. The query operator includes at least a grouping and aggregation operator, row restriction related operators, and a projection operator. If the query operator meets the grouping and aggregation row restriction conditions, the projection operator obtains the data to be queried in batches. During the batch data retrieval process, the grouping and aggregation operator completes the construction and data storage calculation of a preset number of groups. Then, based on the existing groups, the newly obtained data to be queried by the projection operator is stored and calculated. Finally, the grouping and aggregation operator uploads the stored grouped data to the row restriction related operators. The database processes user-input SQL statements through several stages: syntax analysis, semantic analysis, query optimization, execution plan generation, and execution. The grouping performance optimization method proposed in this invention operates on the execution plan generation and execution stages. The method determines whether to use the method of this invention for optimization based on the structure of the execution plan. When the optimization method of this invention can be used, the HAGR and PRJT operators can use unconventional and general methods to execute the plan. The above technical solution reduces resource waste and execution time during database queries, thereby achieving efficient execution of SQL statements.

[0065] Optionally, determine whether the query operator meets the preset grouping and aggregation row restrictions, including:

[0066] S1011. Determine whether the parent operator of the grouping aggregation operator is a row restriction related operator. If not, determine that the current grouping aggregation row restriction condition is not met. If so, save the row restriction condition of the row restriction related operator to the grouping aggregation operator.

[0067] In this embodiment, the upper-level operator can be understood as an operation unit that follows the execution order of a certain operator in the operator chain of the database query execution plan and receives its output data. That is, data flows from the lower-level operator to the upper-level operator for subsequent processing. The upper-level operator is the parent operator, and the upper-level operator is the parent node of the lower-level operator. Row constraint conditions can be understood as the core constraint parameters carried by the row constraint-related operators, such as N in TOPN (e.g., 3 in TOP 3 or LIMIT 3), used to limit the number of groups or data rows that can be output in the end.

[0068] Specifically, in the parsed set of operators, the grouping aggregation operator is located. It is then determined whether the parent operator of the grouping aggregation operator HAGR is a row-limiting related operator TOPN or TOPN-PRJT. If not, it means that the current query has no row-limiting requirement or the row-limiting operator has no direct upstream or downstream relationship with the grouping aggregation operator, and the optimization triggering premise of this solution is not met. Therefore, it is directly determined that the row-limiting condition for grouping aggregation is not met. If so, the row-limiting condition (such as the upper limit of the number of rows N) in the row-limiting related operator is synchronously saved to the grouping aggregation operator to provide a basis for the subsequent threshold determination of the number of groups.

[0069] S1012. Determine whether the parent operator of the projection operator in the query operator is a grouping aggregation operator, and whether the grouping item of the grouping aggregation operator is a base table column. If not, determine that the current grouping aggregation row restriction condition is not met. If yes, determine that the current grouping aggregation row restriction condition is met.

[0070] In this embodiment, the grouping item is the field used by the grouping aggregation operator to divide data into groups, that is, the field specified in the GROUP BY clause of the SQL statement. The value of the grouping item determines the grouping of the data. The base table column is an inherent field of the original data table (base table) in the database, which is a native column defined when the database table is created.

[0071] Specifically, after confirming that S1011 is satisfied, the judgment logic of S1012 is executed. The projection operator PRJT is located in the operator set. It is determined whether the parent operator of PRJT is the grouping aggregation operator HAGR matched in S1011, ensuring that the preprocessed data of the projection operator can be directly transferred to the grouping aggregation operator for subsequent grouping calculations. Simultaneously, it is determined whether the grouping item of the grouping aggregation operator HAGR is a base table column. If the grouping item is a derived field, efficient grouping index verification based on the original data cannot be achieved. If either of the two conditions—that the parent operator of PRJT is the grouping aggregation operator HAGR and that the grouping item of HAGR is a base table column—is not met, it is determined that the current query does not meet the grouping aggregation row restriction conditions, and the optimization logic of this scheme cannot be initiated. If both conditions are met, it is ultimately determined that the current query meets the grouping aggregation row restriction conditions.

[0072] Optionally, if the query operator satisfies the grouping and aggregation row restriction conditions, the projection operator retrieves the data to be queried in batches. During the batch retrieval process, the grouping and aggregation operator completes the construction and data storage calculation of a preset number of groups. Then, based on the existing groups, the newly retrieved data to be queried by the projection operator is stored and calculated. Finally, the grouping and aggregation operator uploads the stored grouped data to the row restriction-related operators, including:

[0073] S1021. Obtain the data to be queried in the current batch through the projection operator, and determine whether the obtained data to be queried is empty.

[0074] In this embodiment, the current batch can be understood as the batch of query data obtained by the projection operator from the lower-level data source in the current iteration.

[0075] Specifically, the projection operator retrieves the current batch of data to be queried from the lower-level data source according to preset batch rules (such as fixed number of records, fixed memory size); then it checks for null values ​​in the batch of data. If the data is null, it means that all the data to be queried has been obtained and processed, triggering S1022 to upload the results; if the data is not null, it means that there is still unprocessed data to be queried, triggering the grouping threshold determination in S1023, and continuing to perform the optimization of grouping aggregation.

[0076] S1022. If the data to be queried is empty, use the grouping aggregation operator to take the stored grouped data as the query result and upload it to the row limit related operator.

[0077] In this embodiment, the query result is the data corresponding to the data query instruction returned to the user after the database has completed the entire query processing flow. It is grouped data composed of base table column values ​​and corresponding aggregate calculation results.

[0078] Specifically, when the projection operator determines that the current batch of data to be queried is empty, it sends a signal to the grouping aggregation operator indicating that data processing is complete. Upon receiving this signal, the grouping aggregation operator extracts all stored grouped data from its maintained grouped hash table, transforms the internal grouping index and aggregation results into user-recognizable standard grouped business results containing base table column values, and uploads this grouped data as the final query result to the row restriction related operators, which then execute the subsequent result return.

[0079] S1023. If the data to be queried is not empty, then determine whether the current number of groups of the grouping aggregation operator has reached the preset number of rows.

[0080] In this embodiment, the current number of groups can be understood as the number of groups successfully constructed by the group aggregation operator, which is the same as the number of group indexes stored in the group hash table. The preset number of rows can be understood as the row limit condition synchronized from the row limit related operators to the group aggregation operator, that is, the upper limit of the final number of groups to be output.

[0081] Specifically, when the current batch of data to be queried obtained by the projection operator is not empty, the status is synchronized to the group aggregation operator. The group aggregation operator counts the number of groups in its own group hash table to obtain the current number of groups. Then, the current number of groups is compared with the preset number of rows. If the current number of groups has not reached the preset number of rows, the group construction in step S1024 is executed to continue creating new groups; if the current number of groups has reached the preset number of rows, the group maintenance in step S1025 is executed to update the data of existing groups only.

[0082] S1024. If not reached, the projection operator performs full instruction calculation on the current batch of data to be queried and passes the calculation result to the grouping aggregation operator. The grouping aggregation operator generates grouped data based on the grouping column of the data to be queried in the calculation result and stores the grouped data in the grouping hash table. After synchronously updating the current number of groups, the projection operator obtains the next batch of data to be queried.

[0083] In this embodiment, full instruction computation refers to the preprocessing operation performed indiscriminately by the projection operator on the current batch of data to be queried. It serves as a prerequisite for providing effective computational material for grouping and aggregation. The preprocessing method is a preset instruction computation method, which is not limited in this embodiment. The group hash table is a key-value pair structure table maintained by the grouping aggregation operator to store grouped data. The key is the index identifier (such as a hash value) corresponding to the grouping column, and the value is the grouped data composed of the values ​​from the base table columns and the aggregation calculation results. The next batch can be understood as the batch of data to be queried obtained by the projection operator from the lower-level data source in the iteration following the current iteration.

[0084] Specifically, the projection operator performs a full instruction calculation on all the data to be queried in the current batch without any data filtering, and passes all calculation results to the grouping aggregation operator. After receiving the calculation results, the grouping aggregation operator extracts the values ​​of the grouping column (base table column) in each data, generates a grouping index identifier, and constructs grouped data containing the base table column values ​​and the aggregation calculation results. If the grouping index identifier of the grouped data is an existing hash value in the grouping hash table, the grouped data is directly stored under the corresponding hash value in the grouping hash table. If there is no hash value in the grouping hash table that is equal to the grouping index identifier of the grouped data, the grouping index identifier is used as the new hash value, a new group is created in the grouping hash table, and the current group number is incremented by 1. After the current batch of data is processed, the projection operator is triggered to retrieve the next batch of data to be queried and to re-execute step S1021.

[0085] Furthermore, S1024 includes:

[0086] a1. The projection operator performs full calculations on the current batch of data to be queried and passes the calculation results to the grouping aggregation operator.

[0087] In this embodiment, the calculation result is a structured dataset obtained after the data to be queried is calculated by the full instruction of the projection operator. It retains the base table column values ​​required for grouping and aggregation, as well as the preprocessed field values, and is the direct input data for the grouping and aggregation operator to perform grouping and construction.

[0088] Specifically, after the projection operator obtains the data to be queried in the current batch, it performs full instruction calculations on all the data to be queried in the batch according to the preprocessing requirements of the database query instruction, without performing any data filtering; then, the completed calculation results are completely passed to the grouping aggregation operator, providing a data source with a unified format for subsequent grouping hash value calculations and aggregation operations.

[0089] a2. Calculate the hash value of the grouping column of each row of data to be queried in the calculation result using the grouping aggregation operator. Perform aggregation operation on the calculation results corresponding to the same hash value to obtain the grouping aggregation result. Create a new group and store the hash value as the key and the grouping aggregation result as the value in the grouping hash table maintained by the grouping aggregation operator. Simultaneously increase the current group number of the grouping aggregation operator.

[0090] In this embodiment, the hash value of the grouping column is a unique index identifier obtained by performing a hash operation on the grouping column (base table column) of each row of data in the calculation result using the grouping aggregation operator. It is used to quickly distinguish different groups and is the core key value of the grouping hash table. The aggregation operation is a statistical operation performed by the grouping aggregation operator on multiple calculation results corresponding to the same hash value. The result of the operation is the core business statistical data of that group.

[0091] Specifically, the grouping aggregation operator receives the calculation results passed by the projection operator, extracts the values ​​of the grouping columns (base table columns) corresponding to the data to be queried from the data row by row, and calculates the corresponding hash values; then, it performs aggregation operations on the calculation results with the same hash values ​​to obtain the grouping aggregation result; then, it creates a new group based on the hash value and the aggregation result, and stores the hash value as the key and the grouping aggregation result as the value in the grouping hash table; each time a new group is successfully created, the current group number is incremented by 1, completing the grouping construction of the current batch of data.

[0092] a3. After processing the current batch of data to be queried, obtain the next batch of data to be queried through the projection operator.

[0093] In this embodiment, after the grouping aggregation operator completes the grouping construction and data storage of the current batch of data to be queried, it sends a signal to the projection operator indicating that the batch processing is complete. After receiving the signal, the projection operator returns to the batch acquisition process of the data to be queried, obtains the next batch of data to be queried, and then re-executes step S1021.

[0094] S1025. If the target has been reached, the projection operator pre-verifies the group index belonging of the current batch of data to be queried, executes the instruction calculation only for the data to be queried that belongs to an existing group, and passes the calculation result to the group aggregation operator. The group aggregation operator updates the group data of the existing group in the group hash table without creating a new group. After the data update is completed, the next batch of data to be queried is obtained through the projection operator.

[0095] In this embodiment, group index attribution refers to the matching relationship between the group index corresponding to the value of the group column in the data to be queried and the index already stored in the group hash table maintained by the group aggregation operator. It is used to determine whether a single piece of data to be queried belongs to an established group. The group index is a unique identifier (such as a hash value) generated based on the value of the base table column (group column).

[0096] Specifically, the projection operator iterates through the current batch of data to be queried. For each data record, it first calculates the hash value of its grouping column as an index identifier, then matches it with the grouping hash table of the grouping aggregation operator to verify the grouping index affiliation. It only executes the instruction calculation on data whose index matches successfully (belonging to an existing group), skipping invalid data whose index does not match, and passes the filtered calculation result to the grouping aggregation operator. After receiving the result, the grouping aggregation operator only uses it to update the aggregation calculation result of the corresponding existing group in the grouping hash table, without creating a new group. After the current batch of data is updated, the projection operator is triggered to re-execute step S1021 to obtain the next batch of data to be queried, until no new data to be queried can be obtained.

[0097] Furthermore, S1025 includes:

[0098] b1. Iterate through the current batch of data to be queried using the projection operator and calculate the hash value of the grouping column for each row of data to be queried.

[0099] In this embodiment, after the projection operator obtains the data to be queried in the current batch, it starts the row-by-row traversal process, reads the complete field information of each data to be queried in turn, extracts the original value of its grouping column (base table column) for each data to be queried, and then performs a calculation on the value using a preset hash algorithm to generate the corresponding grouping column hash value.

[0100] b2. For each row of data to be queried, if the hash value of the grouping column of the data to be queried exists in the grouping hash table, execute the instruction to calculate the data to be queried and retain the calculation result; if the hash value of the grouping column of the data to be queried does not exist in the grouping hash table, skip the data to be queried in this row and do not execute the instruction to calculate.

[0101] In this embodiment, the projection operator iterates through each piece of data to be queried that has generated a hash value, and matches its grouping column hash value with the key in the grouping hash table maintained by the grouping aggregation operator. If the hash value exists in the grouping hash table, it means that the data belongs to an existing group, and the instruction calculation is performed on it and the calculation result is retained; if the hash value does not exist in the grouping hash table, it means that the data has no corresponding existing group, and the data in this row is skipped directly without performing any instruction calculation, so as to avoid wasting computing resources and reduce processing time.

[0102] b3. Pass the calculation results retained in the current batch to the group aggregation operator. The group aggregation operator updates the group aggregation results corresponding to the existing hash values ​​in the group hash table based on the calculation results, without creating new groups.

[0103] In this embodiment, the group aggregation result can be understood as the statistical result obtained after the group aggregation operator performs aggregation operation on multiple batches of data under the same group. It is the core component of the group data and is stored in the group hash table.

[0104] Specifically, the projection operator completely passes the calculation results retained in the current batch (only the calculation results corresponding to data in existing groups) to the group aggregation operator; after receiving the data, the group aggregation operator locates the corresponding existing group in the group hash table based on the group column hash value in the calculation result, and then performs an incremental update on the aggregation result of that group based on the new calculation result. No new groups are created throughout the process.

[0105] b4. After processing the current batch of data to be queried, obtain the next batch of data to be queried through the projection operator.

[0106] In this embodiment, after the grouping aggregation operator completes the aggregation result update of the current batch of data to be queried, it sends a signal to the projection operator indicating that the batch processing is complete. After receiving the signal, the projection operator returns to the batch acquisition process of the data to be queried, re-executes step S1021, and acquires the next batch of data to be queried until there is no new data to be queried.

[0107] Optionally, the method further includes:

[0108] c1. Using row restriction operators, return the grouped data as the query result corresponding to the database query command to the upper-level operator.

[0109] In this embodiment, the row restriction related operator receives all the group data uploaded by the group aggregation operator. Then, the row restriction related operator filters out the result set that meets the row count requirement from all the group data according to the preset row restriction conditions (such as LIMIT 3) (based on the optimized process of this scheme, all group data meet the row restriction conditions). Finally, the filtered group data is passed to the upper-level operator of the row restriction related operator for uploading layer by layer and subsequent processing.

[0110] In one embodiment, Figure 2 This is a flowchart of another database query method provided in an embodiment of the present invention, such as... Figure 2 As shown, it includes:

[0111] First, during the execution plan generation phase, when the parent operator of the HAGR operator is TOPN or TOPN-PRJT, the HAGR operator is parsed, the condition N of TOPN is saved to the HAGR operator, and step two is executed for further judgment.

[0112] 2. If the parent operator of the PRJT operator in the plan is the HAGR operator from step 1, and the grouping item of HAGR is a base table column, then the current plan can be optimized using the method of this invention, and step 3 is executed. If the plan does not meet the requirements of steps 1 and 2, then the optimization is terminated.

[0113] Third, during the execution phase, the PRJT operator retrieves a batch of data from the operator below. If the batch of data is empty, it means that all data has been processed, and step eight is executed. If the batch of data is not empty, it is determined whether the number of groups of the HAGR operator above has reached N in the TOP condition. If the condition N is not met, step four is executed; otherwise, step five is executed.

[0114] 4. If the number of groups for the HAGR operator has not yet reached the TOP condition N, the PRJT operator executes the corresponding instruction calculation on all data in the current batch. Then, all calculation results are passed to the HAGR operator, and step 6 is executed.

[0115] 5. If the number of groups for the HAGR operator has reached the TOP condition N, the PRJT operator, for each row of data in the current batch, must first calculate the hash value of the grouping column in that row before executing the instruction calculation. Then, it searches for that value in the HAGR grouping hash table. If it can be found in the hash table, it means that the row of data belongs to an existing group, and the instruction calculation continues. If it is not found, it means that it does not belong to an existing group, and the row of data is skipped without calculation. Finally, the calculation result of the data belonging to the existing group is passed to the HAGR operator, and step seven is executed.

[0116] 6. After the HAGR operator receives the data from the operator below, it calculates the hash value of the grouping column in each row of data. When the number of groups has not yet reached the TOP condition N, new groups are allowed to be created, the number of groups increases, and the corresponding hash value is stored in the group hash table of the HAGR operator; after processing the current batch of data, it jumps to step 3.

[0117] 7. When the number of groups meets the TOP condition N, PRJT will only pass data belonging to existing groups to HAGR, and no new groups will be created in the hash table recording group values. The HAGR operator processes the current batch of data according to the hash value of the grouping column and then jumps to step three.

[0118] 8. The PRJT operator has finished processing all the data and notifies the HAGR operator that there is no more data to process. The HAGR operator then passes the grouped data to the TOPN operator, and the optimized execution of the method of this invention ends.

[0119] The execution logic of the plan in the above example, after optimization using this method, is as follows:

[0120] 1. The clustered index scan operator CSCN[7] retrieves data from columns C1 and C2 in batches from table T1 by scanning the primary key and passes it to the projection operator PRJT[6].

[0121] 2. When the number of groups calculated by the grouping operator HAGR[5] is less than 3 (i.e. the grouping value required by the TOPN operator), PRJT[6] passes the C1 column of each row of data as a variable to the temporary table operator SPL[8] for subquery related calculations, and then passes it to the grouping operator HAGR[5].

[0122] When the number of groups calculated by HAGR[5] is 3, the data in column C2 of each row will first be searched in the group hash table recorded by HAGR[5]. If the data in the row belongs to an existing group, the data in the row will be calculated using a subquery. Data rows that do not belong to an existing group do not need to be calculated and are skipped directly. Then the data belonging to an existing group is passed to HAGR[5], and the data that does not belong to an existing group is discarded.

[0123] 3. HAGR[5] performs grouping calculations. When the number of groups in HAGR[5] is less than 3 (the grouping value required by the TOPN operator), the new grouping value is saved to the hash table, and the result of the set function operation on each group of data is calculated. After HAGR[5] obtains 3 groups, HAGR[5] no longer adds new groups, but only calculates the data belonging to the existing grouping value. Then, it continues to scan the data from the first step.

[0124] 4. The TOPN[4] operator takes the first 3 groups of data from the 3 groups calculated by HAGR[5] and outputs them upwards.

[0125] Execute the following SQL statement with the GROUP BY and TOPN clauses:

[0126] SELECT TOP 3

[0127] SUM(SELECT D1 FROM T2 WHERE T2.D2 = T1.C1) ,C2

[0128] FROM

[0129] T1 GROUP BY C2;

[0130] The optimized plan for this statement is as follows:

[0131] 1 #NSET2: [1, 3->3, 8]

[0132] 2 #PIPE2: [1, 3->3, 8]

[0133] 3 #PRJT2: [1, 3->3, 8]; exp_num(2), is_atom(FALSE)

[0134] 4 #TOPN2: [1, 3->3, 8];

[0135] 5 #HAGR2: [1, 3->3, 8]; grp_num(1), sfun_num(1)

[0136] 6 #PRJT2: [1, 1000->3, 8]; exp_num(2), is_atom(FALSE)

[0137] 7 #CSCN2: [1, 1000->1000, 8]; INDEX33555472(T1); btr_scan(1)

[0138] 8 #SPL2: [1, 1->0, 8]; key_num(1), spool_num(0), has_variable(1), sites(-)

[0139] 9 #PRJT2: [1, 1->3, 8]; exp_num(1), is_atom(TRUE)

[0140] 10 #SLCT2: [1, 2->3, 8]; T2.D2 = var1

[0141] 11 #CSCN2: [1, 100->300, 8]; INDEX33555474(T2); btr_scan(1)

[0142] After optimization, table T1 only needs to pass 3 rows of data to the temporary table operator SPL, performing 3 x 100 (300) calculations. The grouping operator also only needs to calculate 3 groups of data, which can greatly improve the execution performance of SQL statements containing grouping and TOP clauses. The more rows of data that can skip grouping, and the more complex the instructions to be calculated for each row, the more significant the optimization effect provided by this invention becomes.

[0143] In one embodiment, Figure 3 This is a schematic diagram of the structure of a database query device provided in an embodiment of the present invention. Figure 3 As shown, the device includes:

[0144] The condition determination module 31 is used to parse the received database query instruction, obtain the corresponding query operator, and determine whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least the grouping and aggregation operator, the row restriction related operator, and the projection operator.

[0145] The data query module 32 is used to obtain the data to be queried in batches through the projection operator if the query operator meets the grouping and aggregation row restriction conditions, complete the construction and data storage calculation of a preset number of groups during the batch data retrieval process through the grouping and aggregation operator, and then continue to store and calculate the newly obtained data to be queried by the projection operator based on the existing groups, and upload the stored group data to the row restriction related operator through the grouping and aggregation operator.

[0146] The database query device used in this technical solution reduces resource waste and execution time during the database query process, thereby achieving efficient execution of SQL statements.

[0147] Optionally, the condition determination module 31 is specifically used for:

[0148] Determine whether the parent operator of the grouping aggregation operator is a row restriction related operator. If not, determine that the current grouping aggregation row restriction condition is not met. If so, save the row restriction condition of the row restriction related operator to the grouping aggregation operator.

[0149] Determine whether the parent operator of the projection operator in the query operator is the grouping aggregation operator, and whether the grouping item of the grouping aggregation operator is a base table column. If not, determine that the current grouping aggregation row restriction condition is not met; if yes, determine that the current grouping aggregation row restriction condition is met.

[0150] Optionally, the data query module 32 includes:

[0151] The data null value judgment unit is used to obtain the data to be queried in the current batch through the projection operator and to determine whether the obtained data to be queried is null.

[0152] The query result uploading unit is used to upload the stored grouped data as the query result to the row restriction related operator if the data to be queried is empty;

[0153] A preset row count determination unit is used to determine whether the current group count of the grouping aggregation operator has reached the preset row count if the data to be queried is not empty.

[0154] The first processing unit is configured to, if not, perform full instruction calculation on the current batch of data to be queried by the projection operator and pass the calculation result to the grouping aggregation operator. The grouping aggregation operator generates group data based on the grouping column of the data to be queried in the calculation result and stores the group data in the grouping hash table. After synchronously updating the current number of groups, the next batch of data to be queried is obtained through the projection operator.

[0155] The second processing unit is used to, if the target has been reached, pre-verify the group index belonging of the current batch of data to be queried by the projection operator, execute the instruction calculation only for the data to be queried that belongs to the existing group and pass the calculation result to the group aggregation operator, the group aggregation operator updates the group data of the existing group in the group hash table without creating a new group, and after the data update is completed, obtain the next batch of data to be queried through the projection operator.

[0156] Optionally, the first processing unit is specifically used for:

[0157] The projection operator performs a full instruction calculation on the current batch of data to be queried, and then passes the calculation result to the grouping and aggregation operator.

[0158] The hash value of the grouping column of each row of data to be queried in the calculation result is calculated by the grouping aggregation operator. The aggregation operation is performed on the calculation results corresponding to the same hash value to obtain the grouping aggregation result. A new group is created and the hash value is used as the key and the grouping aggregation result is used as the value to store in the grouping hash table maintained by the grouping aggregation operator. The current group number of the grouping aggregation operator is increased synchronously.

[0159] After processing the current batch of data to be queried, the next batch of data to be queried is obtained through the projection operator.

[0160] Optionally, the second processing unit is specifically used for:

[0161] The projection operator is used to traverse the current batch of data to be queried, and the hash value of the grouping column of each row of the data to be queried is calculated.

[0162] For each row of data to be queried, if the hash value of the grouping column of the data to be queried exists in the grouping hash table, the instruction to calculate the data to be queried is executed and the calculation result is retained; if the hash value of the grouping column of the data to be queried does not exist in the grouping hash table, the data to be queried in this row is skipped and the instruction to calculate is not executed.

[0163] The calculation results retained in the current batch are passed to the group aggregation operator, which updates the group aggregation results corresponding to the existing hash values ​​in the group hash table based on the calculation results, without creating new groups;

[0164] After processing the current batch of data to be queried, the next batch of data to be queried is obtained through the projection operator.

[0165] Optionally, the method further includes:

[0166] The result return module is used to return the grouped data as the query result corresponding to the database query instruction to the upper-level operator by using the row restriction related operators.

[0167] The database query device provided in this embodiment of the invention can execute the database query method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0168] In one embodiment, Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 4 The diagram illustrates a schematic representation of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0169] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0170] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0171] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as database query methods.

[0172] In some embodiments, the database query method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the database query method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the database query method by any other suitable means (e.g., by means of firmware).

[0173] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0178] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0179] This invention also provides a computer program product, including a computer program that, when executed by a processor, can implement the database query method provided in any embodiment of this application.

[0180] In the implementation of the computer program product, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0181] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A database query method, characterized by, include: The received database query instruction is parsed to obtain the corresponding query operator, and it is determined whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least grouping and aggregation operators, row restriction related operators, and projection operators. If the query operator satisfies the grouping and aggregation row restriction conditions, the projection operator obtains the data to be queried in batches. The grouping and aggregation operator completes the construction and data storage calculation of a preset number of groups during the batch data retrieval process. Then, based on the existing groups, the newly obtained data to be queried by the projection operator is stored and calculated. Finally, the grouping and aggregation operator uploads the stored group data to the row restriction related operator.

2. The method of claim 1, wherein, The step of determining whether the query operator satisfies the preset grouping and aggregation row restriction conditions includes: Determine whether the parent operator of the grouping aggregation operator is a row restriction related operator. If not, determine that the current grouping aggregation row restriction condition is not met. If so, save the row restriction condition of the row restriction related operator to the grouping aggregation operator. Determine whether the parent operator of the projection operator in the query operator is the grouping aggregation operator, and whether the grouping item of the grouping aggregation operator is a base table column. If not, determine that the current grouping aggregation row restriction condition is not met; if yes, determine that the current grouping aggregation row restriction condition is met.

3. The method of claim 1, wherein, If the query operator satisfies the grouping and aggregation row restriction condition, the data to be queried is obtained in batches through the projection operator. During the batch data retrieval process, the grouping and aggregation operator completes the construction and data storage calculation of a preset number of groups. Then, based on the existing groups, the newly obtained data to be queried by the projection operator is stored and calculated. Finally, the stored grouped data is uploaded to the row restriction-related operator through the grouping and aggregation operator. The projection operator is used to obtain the data to be queried in the current batch, and it is determined whether the obtained data to be queried is empty. If the data to be queried is empty, the stored grouped data is used as the query result by the grouping aggregation operator and uploaded to the row restriction related operator; If the data to be queried is not empty, then determine whether the current number of groups of the grouping aggregation operator has reached the preset number of rows; If the target is not reached, the projection operator performs a full instruction calculation on the current batch of data to be queried and passes the calculation result to the grouping aggregation operator. The grouping aggregation operator generates grouped data based on the grouping column of the data to be queried in the calculation result and stores the grouped data in the grouping hash table. After synchronously updating the current number of groups, the next batch of data to be queried is obtained through the projection operator. If the target has been reached, the projection operator pre-verifies the group index belonging of the current batch of data to be queried, executes the instruction calculation only for the data to be queried that belongs to an existing group, and passes the calculation result to the group aggregation operator. The group aggregation operator updates the group data of the existing groups in the group hash table without creating new groups. After the data update is completed, the next batch of data to be queried is obtained through the projection operator.

4. The method of claim 3, wherein, The projection operator performs a full instruction calculation on the current batch of data to be queried and passes the calculation result to the grouping aggregation operator. The grouping aggregation operator generates grouped data based on the grouping column of the data to be queried in the calculation result and stores the grouped data in a grouping hash table. After synchronously updating the current group count, it obtains the next batch of data to be queried through the projection operator, including: The projection operator performs a full instruction calculation on the current batch of data to be queried, and then passes the calculation result to the grouping and aggregation operator. The hash value of the grouping column of each row of data to be queried in the calculation result is calculated by the grouping aggregation operator. The aggregation operation is performed on the calculation results corresponding to the same hash value to obtain the grouping aggregation result. A new group is created and the hash value is used as the key and the grouping aggregation result is used as the value to store in the grouping hash table maintained by the grouping aggregation operator. The current group number of the grouping aggregation operator is increased synchronously. After processing the current batch of data to be queried, the next batch of data to be queried is obtained through the projection operator.

5. The method of claim 3, wherein, The projection operator pre-verifies the group index affiliation of the current batch of data to be queried, only performs instruction calculations on data belonging to existing groups, and passes the calculation results to the group aggregation operator. The group aggregation operator updates the group data of existing groups in the group hash table without creating new groups. After completing the data update, the projection operator obtains the next batch of data to be queried, including: The projection operator is used to traverse the current batch of data to be queried, and the hash value of the grouping column of each row of the data to be queried is calculated. For each row of data to be queried, if the hash value of the grouping column of the data to be queried exists in the grouping hash table, the instruction to calculate the data to be queried is executed and the calculation result is retained; if the hash value of the grouping column of the data to be queried does not exist in the grouping hash table, the data to be queried in this row is skipped and the instruction to calculate is not executed. The calculation results retained in the current batch are passed to the group aggregation operator, which updates the group aggregation results corresponding to the existing hash values ​​in the group hash table based on the calculation results, without creating new groups; After processing the current batch of data to be queried, the next batch of data to be queried is obtained through the projection operator.

6. The method of claim 1, wherein, Also includes: By using the row restriction related operators, the grouped data is returned to the upper-level operator as the query result corresponding to the database query instruction.

7. A database query apparatus, characterized by, include: The condition determination module is used to parse the received database query instruction, obtain the corresponding query operator, and determine whether the query operator meets the preset grouping and aggregation row restriction conditions. The query operator includes at least the grouping and aggregation operator, the row restriction related operator, and the projection operator. The data query module is used to obtain the data to be queried in batches through the projection operator if the query operator meets the grouping and aggregation row restriction conditions, complete the construction and data storage calculation of a preset number of groups during the batch data retrieval process through the grouping and aggregation operator, and then continue to store and calculate the newly obtained data to be queried by the projection operator based on the existing groups, and upload the stored group data to the row restriction related operator through the grouping and aggregation operator.

8. An electronic device, comprising: include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a database query method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a database query method according to any one of claims 1-6.

10. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, implements a database query method according to any one of claims 1-6.

Images