Data query method, query table construction method, computing device and storage medium

Through batch processing and switching join algorithms, the construction of database query tables is optimized, which solves the problem of low efficiency in query table construction in the existing technology and improves the efficiency of data query.

WO2024239805A1PCT designated stage expired Publication Date: 2024-11-28ALIBABA CLOUD COMPUTING CO LTD

Patent Information

Application Number
PCT/CN2024/085593
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-19
Filing Date
2024-04-02
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

When querying data in existing databases, the use of optimizer to select the query table construction method is not accurate enough, resulting in low efficiency in query table construction, which in turn affects the data query efficiency.

Method used

By obtaining query statements, processing data tables in batches, building target query tables, and detecting query attribute information, when the switching conditions are met, switch the join algorithm, from lookup join to hash join, and optimize the query table construction method.

Benefits of technology

It improves the efficiency of query table construction, improves the efficiency of data query execution, and avoids waste of resources caused by switching of query table construction methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024085593_28112024_PF_FP_ABST
    Figure CN2024085593_28112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the embodiments of the present application are a data query method, the data query method comprising: acquiring a query statement, and determining at least two data tables corresponding to the query statement; invoking a first operator to perform batch processing on the data tables, and, according to data of a target batch in the at least two data tables, constructing a first target query table, the target batch being any batch; checking query attribute information of the first target query table; when the query attribute information meets a switching condition, constructing a second target query table on the basis of the remaining data in the at least two data tables, the remaining data referring to data excepting the data contained in the first target query table; and obtaining a data query result based on the first target query table and the second target query table. Constructing the second target query table on the basis of the remaining data in the at least two data tables achieves switching of the query table generation mode, thus improving the efficiency of query table generation.
Need to check novelty before this filing date? Find Prior Art

Description

Data query method, query table construction method, computing device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 19, 2023, with application number 202310576506.6 and invention name “Data query method, query table construction method, computing device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data query method, a query table construction method, a computing device, and a storage medium. Background Art

[0003] Currently, when querying data in a database, there may be situations where two or more data tables need to be used to implement data queries. In order to query data, a join algorithm is usually used to connect the data in the data tables, construct a query table, and obtain a query table. Furthermore, the query of the data to be queried is completed based on the query table.

[0004] However, the current method of using the optimizer to select the query table construction method is not accurate enough, which will lead to low query table construction efficiency and further reduce data query efficiency.

[0005] Therefore, it is necessary to provide a faster or more reliable solution to determine the query table construction method.

[0006] Summary of the Invention

[0007] In view of this, embodiments of the present application provide a data query method. One or more embodiments of the present application also involve a data query device, a query table construction method, a query table construction device, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0008] According to a first aspect of an embodiment of the present application, a data query method is provided, comprising:

[0009] Obtain a query statement, and determine at least two data tables corresponding to the query statement;

[0010] Invoking a first operator to process the data table in batches, and constructing a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0011] detecting query attribute information of the first target query table;

[0012] In the case where the query attribute information satisfies the switching condition, constructing a second target query table based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table;

[0013] A data query result based on the first target query table and the second target query table is obtained.

[0014] According to a second aspect of an embodiment of the present application, a data query device is provided, including:

[0015] An acquisition module is configured to acquire a query statement and determine at least two data tables corresponding to the query statement;

[0016] a first construction module configured to call a first operator to perform batch processing on the data table, and to construct a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0017] a detection module, configured to detect query attribute information of the first target query table;

[0018] a second constructing module configured to construct a second target query table based on the remaining data in the at least two data tables if the query attribute information satisfies a switching condition, wherein the remaining data refers to data other than the data included in the first target query table;

[0019] The obtaining module is configured to obtain data query results based on the first target query table and the second target query table.

[0020] According to a third aspect of an embodiment of the present application, a query table construction method is provided, comprising:

[0021] Obtain a query statement, and determine at least two data tables corresponding to the query statement;

[0022] Invoking a first operator to process the data table in batches, and constructing a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0023] detecting query attribute information of the first target query table;

[0024] In a case where the query attribute information satisfies a switching condition, a second target query table is constructed based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table.

[0025] According to a fourth aspect of an embodiment of the present application, a query table construction apparatus is provided, comprising:

[0026] An acquisition module is configured to acquire a query statement and determine at least two data tables corresponding to the query statement;

[0027] a first construction module configured to call a first operator to perform batch processing on the data table, and to construct a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0028] a detection module, configured to detect query attribute information of the first target query table;

[0029] The second construction module is configured to construct a second target query table based on the remaining data in the at least two data tables when the query attribute information meets the switching condition, wherein the remaining data refers to data other than the data included in the first target query table.

[0030] According to a fifth aspect of an embodiment of the present application, there is provided a computing device, including:

[0031] memory and processor;

[0032] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above method are implemented.

[0033] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above method are implemented.

[0034] According to a seventh aspect of an embodiment of the present application, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above method.

[0035] An embodiment of the present application implements the following steps: obtaining a query statement, determining at least two data tables corresponding to the query statement; calling a first operator to perform batch processing on the data tables, and constructing a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch; detecting query attribute information of the first target query table; constructing a second target query table based on the remaining data in the at least two data tables when the query attribute information meets a switching condition, wherein the remaining data refers to data other than the data contained in the first target query table; and obtaining a data query result based on the first target query table and the second target query table.

[0036] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; query attribute information corresponding to the first target query table is detected, and when the query attribute information meets the switching condition, a second target query table is constructed based on the remaining data in at least two data tables, that is, the method of generating the query table is switched, thereby improving the efficiency of generating the query table; based on the first target query table and the second target query table, a data query result corresponding to the query statement is obtained, thereby avoiding the waste of the generated first target query table due to the switching of the query table construction method, thereby ensuring the execution efficiency of the data query. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of a scenario of a data query method provided by an embodiment of the present application;

[0038] FIG2 is a flow chart of a data query method provided by one embodiment of the present application;

[0039] FIG3 is a flowchart of a method for constructing a query table provided by one embodiment of the present application;

[0040] FIG4 is a flowchart of a processing process of a data query method provided by an embodiment of the present application;

[0041] FIG5 is a schematic structural diagram of a data query device provided by an embodiment of the present application;

[0042] FIG6 is a schematic diagram of the structure of a query table construction device provided by one embodiment of the present application;

[0043] FIG7 is a structural block diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0045] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0046] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0047] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0048] First, the terms involved in one or more embodiments of the present application are explained.

[0049] Lookup join algorithm: A join algorithm that pulls data from one end of a probe hash table in batches. For each batch of data pulled, a query condition is constructed for another table based on the batch of data. The corresponding data is then pulled to the upper layer to construct a hash table. The batch of data at one end of the probe hash table is used to probe the hash table and output the result.

[0050] Hash join algorithm: A join algorithm that uses one end to build a hash table and the other end to probe the hash table and output the result.

[0051] CAS: compare and swap, can be used to implement optimistic locking. There is no thread context switching in CAS, which can reduce unnecessary overhead.

[0052] Optimizer: This module estimates the query table construction method for the computing node.

[0053] Currently, join algorithms are typically selected through an optimizer. However, optimizers use statistical information and cost models to determine the optimal execution method, which can lead to errors. Based on the optimizer's principles, the accuracy of the join algorithm cannot be guaranteed. Therefore, the data query method provided in this application further optimizes the join algorithm selection.

[0054] In this application, a data query method is provided. This application also relates to a data query device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0055] Referring to FIG1 , FIG1 shows a scenario diagram of a data query method provided according to an embodiment of the present application, specifically comprising:

[0056] The computing node receives a query statement; a lookup join algorithm is assigned to the computing node by the optimizer; a lookup join operator is created on the computing node based on the lookup join algorithm; at least two data tables corresponding to the query statement are obtained; the lookup join operator is called to process the target data table in at least two data tables in batches, and the data corresponding to each batch is obtained from the data table; based on the data corresponding to each batch, the data tables are connected in batches, that is, a private hash table 1 is created based on the data corresponding to the first batch, a private hash table 2 is created based on the data corresponding to the second batch, and a private hash table 3 is created based on the data corresponding to the third batch, and then the connection processing between the data of each batch and other data tables is completed based on the hash table to obtain the first target query table corresponding to each batch for data query; when the content of the data table is large, the process of generating the query table based on the lookup join algorithm may have low execution efficiency. In the process of generating the query table in batches, the method of the present application detects the query attribute information corresponding to the query table, and when the query attribute information meets the switching condition, the current lookup join is switched to the hash join algorithm, thereby improving the execution efficiency of the algorithm, thereby improving the efficiency of generating the hash table, and thus improving the generation efficiency of the query table based on the hash table; specifically, when creating the lookup join, When the join operator is used, a shared hash table is created, which does not contain any data. When the query attribute information generated by the lookup join algorithm meets the switching conditions, a second target query table is constructed based on the remaining data in the data table not processed by the lookup join algorithm. That is, the empty shared hash table is filled with the remaining data, and then the second target query table is constructed based on the filled shared hash table. In actual applications, after switching the lookup join algorithm to the hash join algorithm, the IN expression is no longer constructed based on the data in the data table. Instead, the IN expression is replaced with a true expression. The data results are queried in the first target query table and the second query table based on the query statement.

[0057] The data query method of the present application obtains a query statement and determines at least two data tables corresponding to the query statement; calls a first operator to perform batch processing on the data tables, and constructs a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch; detects query attribute information of the first target query table; if the query attribute information meets a switching condition, constructs a second target query table based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data contained in the first target query table; and obtains a data query result based on the first target query table and the second target query table.

[0058] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; query attribute information corresponding to the first target query table is detected, and when the query attribute information meets the switching condition, a second target query table is constructed based on the remaining data in at least two data tables, that is, the method of generating the query table is switched, thereby improving the efficiency of generating the query table; based on the first target query table and the second target query table, a data query result corresponding to the query statement is obtained, thereby avoiding the waste of the generated first target query table due to the switching of the query table construction method, thereby ensuring the execution efficiency of the data query.

[0059] Referring to FIG. 2 , FIG. 2 shows a flow chart of a data query method provided according to an embodiment of the present application, which specifically includes the following steps.

[0060] Step 202: Obtain a query statement and determine at least two data tables corresponding to the query statement.

[0061] Among them, the query statement refers to the statement used to query the required data in the database. For example, the query statement SELECT name FROM student table that can be used to query data in the database means querying the name information in the name column of the student table; the data table refers to the table that stores data in the database, for example, the database contains student information table, member information table, order table, etc.

[0062] Specifically, in response to a data query request, a query statement is obtained; the query statement can be input by the user, or generated based on the question text of user data, etc., and the present application does not specifically limit the method for obtaining the query statement; the obtained query statement is parsed to determine at least two data tables corresponding to the query statement; in actual applications, the query statement can correspond to only one data table, but in the case of one data table, there is no need to perform a data table connection operation, so the method of the present application is only used when the query statement corresponds to two or more data tables.

[0063] In a specific embodiment of the present application, in response to an order query request, a query statement for querying order data is determined; and based on the query statement, data table 1 and data table 2 required in the order query process are determined.

[0064] In practical applications, the method for determining at least two data tables corresponding to the query statement may include:

[0065] Determining the query fields included in the query statement;

[0066] Search the database for a data table containing the query field as the data table corresponding to the query statement.

[0067] Specifically, the query field refers to the field parsed in the query statement, for example, the fields "membership level", "order number", etc. are parsed in the query statement; based on the query field, at least two data tables required for subsequent data query are determined in the database.

[0068] At least two corresponding data tables are determined through the obtained query statement, so as to subsequently perform data query based on the at least two data tables.

[0069] Step 204: calling a first operator to process the data table in batches, and constructing a first target query table based on the target batch data in the at least two data tables.

[0070] Among them, the first operator refers to a computing unit that generates a query table based on the data in the data table. Multiple operators can be included on a computing node, and parallel processing of data can be achieved based on multiple operators; batch processing refers to dividing the data in the data table into at least two parts; the first target query table refers to the data table obtained by connecting the data of the target batch in the data table; the target batch is any batch corresponding to the data table.

[0071] Specifically, the first operator responds to the query table construction request, selects one or more data tables from at least two data tables corresponding to the query statement for batch processing; obtains data of a target batch from at least two data tables, and constructs a first target query table based on the data of the target batch.

[0072] In practical applications, the method for batch processing the data table may include:

[0073] determining a target data table among the at least two data tables;

[0074] The data in the target data table are processed in batches according to a processing quantity threshold to obtain data sets corresponding to each batch.

[0075] Among them, the target data table refers to any one of the at least two data tables corresponding to the query statement; the processing quantity threshold refers to the pre-set upper limit of data processing; in actual applications, the processing quantity threshold can be set based on the processing upper limit of the operator. For example, if the processing upper limit of the operator is 20 data, the processing quantity threshold can be set to 20, so as to ensure that the data corresponding to each target batch can be efficiently processed subsequently; the data set refers to the data set composed of the data of the target batch in the data table. For example, batch 1 corresponds to data 1, data 2 and data 3 in the data table, then the data set corresponding to batch 1 can be composed of data 1, data 2 and data 3.

[0076] Specifically, a target data table is selected from at least two data tables corresponding to the query statement; data in the target data table are processed in batches according to a processing quantity threshold, and data sets corresponding to each batch are formed from the data in each batch in the data table.

[0077] In a specific embodiment of the present application, data table 1 is selected as the target data table among data table 1, data table 2 and data table 3, and the target data table contains 10 data items 1-10; the processing quantity threshold is determined to be 5, and the data in the target data table is divided into two batches based on the processing quantity threshold, that is, the data items 1-5 in the target data table can be divided into the first batch, and the data items 6-10 in the target data table can be divided into the second batch.

[0078] The data in the data table is processed in batches by using the processing quantity threshold to determine the data sets corresponding to each batch in the data table so that the data sets corresponding to each batch can be processed in batches subsequently, thereby reducing the processing pressure of the computing unit and ensuring the real-time performance of data processing.

[0079] In practical applications, the method of constructing a first target query table based on the target batch data in the at least two data tables may include:

[0080] Determine the data set corresponding to the target batch, and perform deduplication processing on the data in the data set;

[0081] Querying first data to be connected in data tables other than the target data table according to the deduplicated data, and generating a hash table based on the first data to be connected;

[0082] searching the data set for matching fields based on the hash table;

[0083] Acquire second data to be connected from the data set according to the matching field;

[0084] A first target query table is constructed according to the first data to be connected and the second data to be connected.

[0085] Among them, deduplication processing refers to deduplication of data in the data set; the first data to be connected refers to the data collected in the data table other than the target data table corresponding to the query statement based on the deduplicated data; the hash table refers to the data table composed of the first data to be connected and the hash value corresponding to the first data to be connected; the matching field refers to the field contained in the data set that is consistent with the hash table. For example, the hash table contains the field "Zhang San" and the corresponding hash value "123". If it is determined that the data set contains the field "Zhang San", then the field "Zhang San" can be used as the matching field; the second data to be connected refers to the data obtained in the data set based on the matching field and related to the matching field. For example, if the matching field is determined to be "Zhang San", and the data set contains data related to the field "Zhang San" such as "18 years old" and "Level 2 member", then the data related to the field "Zhang San" such as "18 years old" and "Level 2 member" are the second data to be connected.

[0086] Specifically, a lookup join algorithm is used to implement batch connection of data tables to obtain a query table corresponding to each batch, that is, to determine the data set corresponding to the target batch; the data in the data set is deduplicated to obtain deduplicated data in order to improve subsequent processing efficiency; based on the deduplicated data, the first data to be connected is queried in data tables other than the target data table, that is, data consistent with the deduplicated data is obtained in data tables other than the target data table; a hash table is generated based on the first data to be connected obtained from the query; based on the hash table, a matching field is queried in the target data set of the target batch; the second data to be connected is obtained in the data set according to the queried matching field; and a first target query table is generated by connecting the first data to be connected and the second data to be connected.

[0087] In a specific embodiment of the present application, it is determined that the data set contains data "a, b, c", and the data in the data set is deduplicated to obtain the deduplicated data "a, b"; based on the deduplicated data "a, b", the first data to be connected is queried as "a" in data tables other than the target data table; a hash table is generated based on the first data to be connected "a", and the hash table contains the field "a" and the hash value "124" corresponding to the field "a"; according to the hash table, a matching field "a" is queried in the data set, and based on the matching field, data related to the matching field is obtained in the data set as the second data to be connected; the first data to be connected and the second data to be connected are connected based on the matching field to obtain a first target query table.

[0088] By constructing the first target query table based on the target batch data in at least two data tables, it is avoided to construct the first target query table based on all the data in the data tables at the same time, thereby improving the construction efficiency of the first target query table and ensuring the real-time performance of the first target query table.

[0089] Step 206: Detect query attribute information of the first target query table.

[0090] The query attribute information refers to the attribute information obtained in the process of constructing the first target query table.

[0091] Specifically, since the first operator processes the data table in batches, it is necessary to generate the corresponding first target query table in batches based on the data sets corresponding to each batch; in the process of generating the first target query table of the current batch, the query attribute information of the current batch is detected so that it can be judged whether it is necessary to switch the query table construction method based on the query attribute information.

[0092] In practical applications, the query attribute information includes at least one of the total number of query tables, query execution time, and target connection number, where the target connection number refers to the number of connections between the query unit and the storage unit.

[0093] The total number of query tables refers to the total number of first target query tables generated. In actual applications, query tables can be generated based on the first target query tables constructed in each batch. By counting the number of first target query tables, the number of batches for executing queries can be determined; the query execution time refers to the time it takes to generate the first target query table in any batch; the number of target connections refers to the number of transmission connections between the query unit and the storage unit in any batch.

[0094] Specifically, the query attribute information can be based on one or more of the total number of query tables, query execution time and target connection number. For example, only the total number of query tables is used as the query attribute information of the first target query table, or the total number of query tables and query execution time are used as query attribute information, etc.

[0095] In a specific embodiment of the present application, in the process of constructing a first target query table based on target batch data in at least two data tables, the total number of query tables of the generated first target query table is counted as query attribute information of the first target query table.

[0096] By detecting the query attribute information of the first target query table, it is subsequently determined whether the construction method of the query table needs to be adjusted based on the query attribute information.

[0097] Step 208: When the query attribute information satisfies the switching condition, a second target query table is constructed based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table.

[0098] Among them, the switching condition refers to the condition for switching the query table construction method, and the switching condition is set based on the construction information of the target query table. Specifically, the switching condition can be that the total number of query tables of the first target query table is greater than or equal to the preset number threshold, and the query execution time of the first target query table is greater than or equal to one or more of the preset time thresholds, that is, the switching condition is the upper limit condition for using the current query table construction method to construct the target query table. If this condition is exceeded, it is necessary to switch the query table construction method, thereby improving the query table construction efficiency; the remaining data refers to the data of the unprocessed data table in the process of constructing the first target query table, that is, the data other than the data contained in the first target query table; the second query table refers to the query table constructed based on the remaining data.

[0099] Specifically, before constructing the first target query table based on the data of each batch, determine whether the query attribute information corresponding to the first target query table meets the switching conditions; if the query attribute information does not meet the switching conditions, construct the first target query table based on the data corresponding to the batch; if the query attribute information meets the switching conditions, it is necessary to switch the query table construction method, that is, construct the second target query table based on the remaining data in the at least two data tables.

[0100] In practical applications, the method of constructing the second target query table based on the remaining data in the at least two data tables may include:

[0101] Set the state of the pre-built shared hash table;

[0102] A second target query table is constructed based on the result of the state setting and the remaining data in the at least two data tables.

[0103] Specifically, the solution of the present application is to switch from a lookup join query table construction algorithm to a hash join query table construction algorithm; since the lookup join query table construction algorithm does not have the concept of a shared hash table, an empty shared hash table is created in advance to enable switching; before switching from lookup join to hash join, the state of the pre-constructed shared hash table is obtained; if the state of the shared hash table can be set, it is necessary to further determine whether the shared hash table is filled. If not, the shared hash table state is set in the form of CAS (compare and swap), and the shared hash table needs to be filled based on the generated data. If it is filled, a second target query table can be constructed based on the shared hash table; if the state of the shared hash table cannot be set, it is necessary to wait until the shared hash table is converted to a settable state before performing subsequent processing.

[0104] In a specific embodiment of the present application, a state setting is performed on a pre-built shared hash table A to obtain a state setting result; and a second target query table is constructed based on the state setting result and the remaining data in at least two data tables.

[0105] By setting the shared hash table and then constructing the second target query table based on the setting state, the processing efficiency of the second target query table is improved.

[0106] In practical applications, based on the result of the state setting and the remaining data in the at least two data tables, the method of constructing the second target query table may include:

[0107] If the state setting result of the shared hash table is successful, determining a target data table among the at least two data tables, and target remaining data of the target data table;

[0108] Based on the target remaining data, the shared hash table is filled with data to obtain a second target query table.

[0109] The target data table refers to any one of the at least two data tables; and the target remaining data refers to the data in the target data table that is not used to generate the first target query table.

[0110] Specifically, when the status setting result of the shared hash table is successful, it is further determined whether the shared hash table has been filled; if not, a target data table is selected from at least two data tables, for example, a data table with the smallest amount of data is selected from at least two data tables as the target data table; the target remaining data in the target data table is determined; the shared hash table is filled based on the target remaining data, and then a second target query table is obtained based on the filled shared hash table; if it has been filled, the second target query table is directly constructed based on the shared hash table.

[0111] In a specific embodiment of the present application, based on a state setting request for a shared hash table, the state of the shared hash table is set; when the state setting for the shared hash table is successful, it is determined whether the shared hash table has been filled; if it is determined that the shared hash table is not filled, a target data table in at least two data tables is determined, and target remaining data corresponding to the target data table is determined; the shared hash table is filled based on the target generation data to obtain a second target query table.

[0112] When the state setting result is determined to be successful, the shared hash table is filled based on the target remaining data, so as to obtain the second target lookup table based on the filled shared hash table.

[0113] Furthermore, based on the target remaining data, the shared hash table is filled with data to obtain a second target query table, including:

[0114] Calculating a hash value corresponding to the target remaining data, and filling the shared hash table based on the target remaining data and the hash value;

[0115] A second target lookup table is constructed based on the populated shared hash table.

[0116] Specifically, determine the target remaining data in the target data table; calculate the hash value corresponding to each target remaining data, for example, calculate the hash value corresponding to the target remaining data based on a hash function; fill each target remaining data and the hash value corresponding to each target remaining data into the shared hash table, thereby completing the filling of the shared hash table; construct a second target query table based on the filled shared hash table.

[0117] Furthermore, the method of constructing the second target query table based on the populated shared hash table may include:

[0118] According to the populated shared hash table, querying a matching field in data tables other than the target data table, and obtaining first data to be connected in the data tables other than the target data table based on the matching field;

[0119] Acquire second data to be connected in the target data table based on the matching field;

[0120] A second target query table is constructed according to the first data to be connected and the second data to be connected.

[0121] The first data to be connected refers to data obtained from a data table other than the target data table according to the matching field; the second data to be connected refers to data obtained from the target data table according to the matching field.

[0122] Specifically, according to the filled shared hash table, the corresponding matching field is queried in the data table other than the target data table, and the second data to be connected is obtained based on the matching field; the second data to be connected is obtained in the target data table based on the matching field; and a second target query table is constructed based on the first data to be connected and the second data to be connected.

[0123] In a specific embodiment of the present application, a populated shared hash table is obtained, and a matching field is queried in a data table other than a target data table based on the shared hash table; first data to be connected is obtained in a data table other than the target data table according to the matching field; second data to be connected is obtained in the target data table according to the matching field, and a second target query table is obtained by splicing the first data to be connected and the second data to be connected.

[0124] The second target query table is constructed by the filled shared hash table, thereby realizing the switching of the query table construction mode, and then the second target query table is constructed based on the switched query table construction mode, so that data can be queried based on the second target query table subsequently.

[0125] In practical applications, to improve processing efficiency, multiple operators can be set up on a node to perform query table generation processing. When switching from lookup join to hash join, if a lookup join operator on the node completes filling the shared hash table during the switch, the remaining lookup join operators on the node do not need to fill the shared hash table again when switching and can directly use the filled shared data table.

[0126] Specifically, the method of the present application also includes:

[0127] Calling a second operator to process the data table in batches, and constructing a third target query table based on the target batch data in the at least two data tables;

[0128] When the state setting result of the shared hash table is failure and the query attribute information of the third target query table meets the switching condition, a fourth target query table is constructed based on the shared hash table.

[0129] Among them, the second operator refers to a calculation unit that generates a query table based on the data in the data table, which is different from the first operator; the third target query table refers to a query table constructed by the second operator based on the data in at least two data tables; the fourth target query table refers to a query table constructed based on the filled shared hash table.

[0130] Specifically, the second operator processes the data table in batches, and then constructs a third target query table based on the data of the target batch in at least two data tables; the state of the shared hash table is set based on the switching request. If the state setting fails, it is further determined whether the shared hash table is filled; if so, after the shared hash table can be set, and when the query attribute information of the third target query table meets the switching conditions, a fourth target query table is constructed based on the shared hash table, thereby completing the switching of the query table construction method.

[0131] In a specific embodiment of the present application, a second operator is called to process the data table in batches; a third target query table is constructed based on the data of the target batch in at least two data tables; the query attribute information of the third target query table is detected, and it is determined that the query attribute information of the current third target query table meets the switching condition, and then the state of the shared hash table is set; if the current state setting result of the shared hash table is a setting failure, the shared hash table is set to a blocked state; and while waiting for the shared hash table to be set, a fourth target query table is constructed based on the shared hash table.

[0132] By performing a switching judgment on the second operator, the detection of each operator is achieved; when it is determined that the switch is possible, it is further determined whether the shared hash table is filled. If it is filled, a query table is constructed based on the filled shared hash table to improve the efficiency of query table construction.

[0133] Step 210: Obtain data query results based on the first target query table and the second target query table.

[0134] The data query result refers to the query result corresponding to the query statement.

[0135] In practical applications, in order to ensure data query efficiency and real-time performance, after building the first target query table, the following steps are also required:

[0136] The first target query table is searched for a first query result corresponding to the query statement.

[0137] Among them, the first query result refers to the result queried in the first query table based on the query statement; specifically, after the first target query table is generated, data can be queried in the first query table based on the query statement, and the current first query result can also be returned to the user, thereby reducing the time the user waits for the query result.

[0138] The first query result is not a complete query result. You need to build the remaining query tables and perform result queries based on the remaining query tables.

[0139] Specifically, the method for obtaining the data query result based on the first target query table and the second target query table may include:

[0140] Querying the second target query table for a second query result corresponding to the query statement;

[0141] A data query result corresponding to the query statement is determined based on the first query result of the first query table and the second query result of the second query table.

[0142] Specifically, the second query result refers to the result queried in the second query table; after obtaining the first query result and the second query result, the first query result and the second query result are merged to obtain the total query result corresponding to the query statement, that is, the first query result.

[0143] In a specific embodiment of the present application, after generating a first target query table, a first query result is queried in the first target query table based on a query statement; after generating a second target query table, the first query result is queried in the second target query table based on the query statement; the first query result and the second query result are merged to obtain a data query result corresponding to the query statement.

[0144] By completing the data query while generating a query table, partial data query results are obtained; then, based on the partial query results, the data query results are merged to achieve efficient data query.

[0145] The data query method of the present application obtains a query statement and determines at least two data tables corresponding to the query statement; calls a first operator to perform batch processing on the data tables, and constructs a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch; detects query attribute information of the first target query table; if the query attribute information meets a switching condition, constructs a second target query table based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data contained in the first target query table; and obtains a data query result based on the first target query table and the second target query table.

[0146] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; detecting query attribute information corresponding to the first target query table, and constructing a second target query table based on the remaining data in at least two data tables when the query attribute information meets a switching condition, that is, switching the method of generating the query table, thereby improving the efficiency of generating the query table; obtaining the data query result corresponding to the query statement based on the first target query table and the second target query table, avoiding the waste of the generated first target query table due to the switching of the query table construction method, thereby ensuring the execution efficiency of the data query

[0147] Referring to FIG3 , FIG3 shows a flow chart of a query table construction method provided according to an embodiment of the present application, which specifically includes the following steps.

[0148] Step 302: Obtain a query statement and determine at least two data tables corresponding to the query statement.

[0149] Step 304: calling a first operator to process the data table in batches, and constructing a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch.

[0150] Step 306: Detect query attribute information of the first target query table.

[0151] Step 308: When the query attribute information satisfies the switching condition, a second target query table is constructed based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table.

[0152] The query table construction method of the present application obtains a query statement and determines at least two data tables corresponding to the query statement; calls a first operator to perform batch processing on the data tables, and constructs a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch; detects query attribute information of the first target query table; and constructs a second target query table based on the remaining data in the at least two data tables when the query attribute information meets a switching condition, wherein the remaining data refers to data other than the data contained in the first target query table.

[0153] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; the query attribute information corresponding to the first target query table is detected, and when the query attribute information meets the switching condition, a second target query table is constructed based on the remaining data in at least two data tables, that is, the method of generating the query table is switched, thereby improving the efficiency of generating the query table.

[0154] The following further illustrates the data query method provided by the present application by taking its application in student data query as an example, in conjunction with FIG4 . FIG4 shows a flow chart of a processing process of a data query method provided by an embodiment of the present application, which specifically includes the following steps.

[0155] Step 402: Obtain a query statement and determine at least two student information tables corresponding to the query statement.

[0156] Step 404: Call the first operator to process the target student information table in batches, and construct a first target query table based on the data of the target batches.

[0157] Specifically, determine the data set corresponding to the target batch and deduplicate the data in the data set; query the first data to be connected in the student information table other than the target student information table based on the deduplicated data, and generate a hash table based on the first data to be connected; query the matching field in the data set based on the hash table; obtain the second data to be connected in the data set based on the matching field; and construct a first target query table based on the first data to be connected and the second data to be connected.

[0158] Step 406: Detect query attribute information of the first target query table.

[0159] Step 408: When the query attribute information satisfies the switching condition, a second target query table is constructed based on the remaining data in at least two student information tables, wherein the remaining data refers to data other than the data contained in the at least two student information tables.

[0160] Specifically, the state of the pre-built shared hash table is set; when the state setting result of the shared hash table is successful, the target student information table in at least two student information tables and the target remaining data of the target student information table are determined; the hash value corresponding to the target remaining data is calculated, and the shared hash table is filled based on the target remaining data and the hash value; according to the filled shared hash table, the matching field is queried in the student information tables other than the target student information table, and the first data to be connected is obtained in the student information tables other than the target student information table based on the matching field; the second data to be connected is obtained in the target student information table based on the matching field; and a second target query table is constructed based on the first data to be connected and the second data to be connected.

[0161] Step 410: Generate data query results based on the first target query table and the second target query table.

[0162] The data query method in the embodiment of the present application constructs a first target query table based on a query statement, thereby realizing construction of a first target query table for data query based on the student information table corresponding to the query statement; detecting query attribute information corresponding to the first target query table, and constructing a second target query table based on the remaining data in at least two student information tables when the query attribute information meets the switching condition, that is, switching the method of generating the query table, thereby improving the efficiency of generating the query table; obtaining the data query result corresponding to the query statement based on the first target query table and the second target query table, avoiding the waste of the generated first target query table due to the switching of the query table construction method, thereby ensuring the execution efficiency of the student information data query.

[0163] Corresponding to the above method embodiment, the present application also provides a data query device embodiment. FIG5 shows a schematic diagram of the structure of a data query device provided by an embodiment of the present application. As shown in FIG5, the device includes:

[0164] An acquisition module 502 is configured to acquire a query statement and determine at least two data tables corresponding to the query statement;

[0165] A first construction module 504 is configured to call a first operator to perform batch processing on the data table and construct a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0166] A detection module 506 is configured to detect query attribute information of the first target query table;

[0167] A second constructing module 508 is configured to construct a second target query table based on the remaining data in the at least two data tables if the query attribute information satisfies the switching condition, wherein the remaining data refers to data other than the data included in the first target query table;

[0168] The obtaining module 510 is configured to obtain data query results based on the first target query table and the second target query table.

[0169] Optionally, the second building module 508 is further configured to:

[0170] Set the state of the pre-built shared hash table;

[0171] A second target query table is constructed based on the result of the state setting and the remaining data in the at least two data tables.

[0172] Optionally, the second building module 508 is further configured to:

[0173] If the state setting result of the shared hash table is successful, determining a target data table among the at least two data tables, and target remaining data of the target data table;

[0174] Based on the target remaining data, the shared hash table is filled with data to obtain a second target query table.

[0175] Optionally, the second building module 508 is further configured to:

[0176] Calculating a hash value corresponding to the target remaining data, and filling the shared hash table based on the target remaining data and the hash value;

[0177] A second target lookup table is constructed based on the populated shared hash table.

[0178] Optionally, the second building module 508 is further configured to:

[0179] According to the populated shared hash table, querying a matching field in data tables other than the target data table, and obtaining first data to be connected in the data tables other than the target data table based on the matching field;

[0180] Acquire second data to be connected in the target data table based on the matching field;

[0181] A second target query table is constructed according to the first data to be connected and the second data to be connected.

[0182] Optionally, the device further includes a construction submodule configured to:

[0183] Calling a second operator to process the data table in batches, and constructing a third target query table based on the target batch data in the at least two data tables;

[0184] When the state setting result of the shared hash table is failure and the query attribute information of the third target query table meets the switching condition, a fourth target query table is constructed based on the shared hash table.

[0185] Optionally, the query attribute information includes at least one of the total number of query tables, query execution time, and target connection number, wherein the target connection number refers to the number of connections between the query unit and the storage unit.

[0186] Optionally, the obtaining module 510 is further configured to:

[0187] Querying the first target query table for a first query result corresponding to the query statement;

[0188] Accordingly, obtaining data query results based on the first target query table and the second target query table includes:

[0189] Querying the second target query table for a second query result corresponding to the query statement;

[0190] A data query result corresponding to the query statement is determined based on the first query result of the first query table and the second query result of the second query table.

[0191] Optionally, the first building module 504 is further configured to:

[0192] Determine the data set corresponding to the target batch, and perform deduplication processing on the data in the data set;

[0193] Querying first data to be connected in data tables other than the target data table according to the deduplicated data, and generating a hash table based on the first data to be connected;

[0194] searching the data set for matching fields based on the hash table;

[0195] Acquire second data to be connected from the data set according to the matching field;

[0196] A first target query table is constructed according to the first data to be connected and the second data to be connected.

[0197] Optionally, the acquisition module 502 is further configured to:

[0198] Determining the query fields included in the query statement;

[0199] Search the database for a data table containing the query field as the data table corresponding to the query statement.

[0200] Optionally, the switching condition includes at least one of the following: a total number of query tables in the first target query table is greater than or equal to a preset number threshold, and a query execution time of the first target query table is greater than or equal to a preset time threshold.

[0201] The data query device of the present application obtains a query statement and determines at least two data tables corresponding to the query statement; calls a first operator to perform batch processing on the data tables, and constructs a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch; detects query attribute information of the first target query table; and constructs a second target query table based on the remaining data in the at least two data tables when the query attribute information meets a switching condition, wherein the remaining data refers to data other than the data contained in the first target query table; and obtains a data query result based on the first target query table and the second target query table.

[0202] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; detecting query attribute information corresponding to the first target query table, and constructing a second target query table based on the remaining data in at least two data tables when the query attribute information meets a switching condition, that is, switching the method of generating the query table, thereby improving the efficiency of generating the query table; obtaining the data query result corresponding to the query statement based on the first target query table and the second target query table, avoiding the waste of the generated first target query table due to the switching of the query table construction method, thereby ensuring the execution efficiency of the data query

[0203] Corresponding to the above method embodiment, the present application also provides an embodiment of a query table construction device. FIG6 shows a schematic diagram of the structure of a query table construction device provided by one embodiment of the present application. As shown in FIG6 , the device includes:

[0204] An acquisition module 602 is configured to acquire a query statement and determine at least two data tables corresponding to the query statement;

[0205] A first construction module 604 is configured to call a first operator to perform batch processing on the data table and construct a first target query table based on the data of a target batch in the at least two data tables, wherein the target batch is any batch;

[0206] A detection module 606 is configured to detect query attribute information of the first target query table;

[0207] The second construction module 608 is configured to construct a second target query table based on the remaining data in the at least two data tables when the query attribute information meets the switching condition, wherein the remaining data refers to data other than the data included in the first target query table.

[0208] The query table construction device of the present application obtains a query statement and determines at least two data tables corresponding to the query statement; calls a first operator to perform batch processing on the data tables, and constructs a first target query table based on the data of the target batch in the at least two data tables, wherein the target batch is any batch; detects query attribute information of the first target query table; and constructs a second target query table based on the remaining data in the at least two data tables when the query attribute information meets a switching condition, wherein the remaining data refers to data other than the data contained in the first target query table.

[0209] By constructing a first target query table based on a query statement, a first target query table for data query is constructed based on the data table corresponding to the query statement; the query attribute information corresponding to the first target query table is detected, and when the query attribute information meets the switching condition, a second target query table is constructed based on the remaining data in at least two data tables, that is, the method of generating the query table is switched, thereby improving the efficiency of generating the query table.

[0210] The above is a schematic diagram of the data query device of this embodiment. It should be noted that the technical solution of the data query device and the technical solution of the above-mentioned data query method are of the same concept. For details not described in detail in the technical solution of the data query device, please refer to the description of the technical solution of the above-mentioned data query method.

[0211] Figure 7 shows a block diagram of a computing device 700 according to one embodiment of the present application. Components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0212] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0213] In one embodiment of the present application, the aforementioned components of the computing device 700 and other components not shown in FIG7 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG7 is for illustrative purposes only and does not limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0214] Computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 700 may also be a mobile or stationary server.

[0215] The processor 720 is configured to execute the following computer-executable instructions, which implement the steps of the above-mentioned data query method when executed by the processor.

[0216] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data query method are of the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned data query method.

[0217] An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the above-mentioned data query method when executed by a processor.

[0218] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned data query method are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-mentioned data query method.

[0219] An embodiment of the present application further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data query method.

[0220] The above is an illustrative solution of a computer program of this embodiment. It should be noted that the technical solution of the computer program and the technical solution of the above-mentioned data query method are of the same concept. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the above-mentioned data query method.

[0221] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0222] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0223] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present application.

[0224] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0225] The preferred embodiments of the present application disclosed above are intended only to help illustrate the present application. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of the present application. This application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present application, so that those skilled in the art can better understand and utilize the present application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A data query method, comprising: Obtain a query statement, and determine at least two data tables corresponding to the query statement; Calling a first operator to process the data table in batches, and constructing a first target query table according to the target batch data in the at least two data tables; Detecting query attribute information of the first target query table; In the case where the query attribute information satisfies the switching condition, constructing a second target query table based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table; A data query result based on the first target query table and the second target query table is obtained.

2. The method according to claim 1, constructing a second target query table based on the remaining data in the at least two data tables, comprising: Set the state of the pre-built shared hash table; A second target query table is constructed based on the result of the state setting and the remaining data in the at least two data tables.

3. The method according to claim 2, constructing a second target query table based on the result of the state setting and the remaining data in the at least two data tables, comprising: In a case where the state setting result of the shared hash table is successful, determining a target data table among the at least two data tables, and target remaining data of the target data table; Based on the target remaining data, the shared hash table is filled with data to obtain a second target query table.

4. The method according to claim 3, based on the target remaining data, filling the shared hash table with data to obtain a second target query table, comprising: Calculating a hash value corresponding to the target remaining data, and filling the shared hash table based on the target remaining data and the hash value; A second target lookup table is constructed based on the populated shared hash table.

5. The method of claim 4, constructing a second target query table based on the populated shared hash table, comprising: According to the filled shared hash table, in data tables other than the target data table, query a matching field, and based on the matching field, obtain first data to be connected in the data tables other than the target data table; Acquire second data to be connected in the target data table based on the matching field; A second target query table is constructed according to the first data to be connected and the second data to be connected.

6. The method of claim 2, further comprising: Calling a second operator to process the data table in batches, and constructing a third target query table according to the target batch data in the at least two data tables; When the state setting result of the shared hash table is failure and the query attribute information of the third target query table meets the switching condition, a fourth target query table is constructed based on the shared hash table.

7. The method of claim 1, wherein the query attribute information includes at least one of the total number of query tables, query execution time, and the number of target connections, wherein: The target number of connections refers to the number of connections between the query unit and the storage unit.

8. The method according to claim 1, after constructing the first target query table, further comprises: Searching the first target query table for a first query result corresponding to the query statement; Accordingly, obtaining data query results based on the first target query table and the second target query table includes: Searching the second target query table for a second query result corresponding to the query statement; Based on the first query result of the first query table and the second query result of the second query table, a data query result corresponding to the query statement is determined.

9. The method of claim 1, constructing a first target query table based on the target batch data in the at least two data tables, comprising: Determine a data set corresponding to the target batch, and perform deduplication processing on the data in the data set; Querying first data to be connected in data tables other than the target data table according to the deduplicated data, and generating a hash table based on the first data to be connected; searching the data set for matching fields based on the hash table; Acquire second data to be connected in the data set according to the matching field; A first target query table is constructed according to the first data to be connected and the second data to be connected.

10. The method according to claim 1, determining at least two data tables corresponding to the query statement comprises: Determining the query fields included in the query statement; A data table containing the query field is searched in the database as the data table corresponding to the query statement.

11. The method according to claim 1, wherein the switching condition comprises at least one of the following: the total number of query tables of the first target query table is greater than or equal to a preset number threshold, and the query execution time of the first target query table is greater than or equal to a preset time threshold.

12. A method for constructing a query table, comprising: Obtain a query statement, and determine at least two data tables corresponding to the query statement; Calling a first operator to process the data table in batches, and constructing a first target query table according to the data of the target batch in the at least two data tables, wherein the target batch is any batch; Detecting query attribute information of the first target query table; In the case that the query attribute information satisfies the switching condition, a second target query table is constructed based on the remaining data in the at least two data tables, wherein the remaining data refers to data other than the data included in the first target query table.

13. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. The computer executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 12.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Citation Information

Patent Citations

  • Multi-table connection query method and device, equipment and storage medium

    CN114780589A

  • Data table switching method, device and equipment

    CN115185975A

  • Automatic integration method and device for external prompt expression, generation and application

    CN115794826A

  • Query method and device of distributed database, storage medium and electronic equipment

    CN116089450A

  • Data query method, query table construction method, computing device and storage medium

    CN116775645A

Cited By

  • Hive metadata-based total data amount auditing method and device, equipment and storage medium

    CN121166778A