A database query method and apparatus

Abstract

Embodiments of the present specification provide a database query method and device. The database includes a driver table and a driven table. When a computing device receives a join query request, repeated data detection is performed on a plurality of data of the driver table to obtain repeated information of each data in the plurality of data. Then, for any one data in the plurality of data of the driver table, the data is judged based on the repeated information of the data. When the data has repeated data, and the data is the first data in the repeated data, data having a connection relationship with the data is determined from a plurality of data of the driven table to obtain a connection result, and a correspondence relationship between the data and the connection result is recorded. When the data has repeated data, and the data is not the first data in the repeated data, the data of the driven table does not need to be repeatedly scanned, and the connection result corresponding to the data can be obtained from the recorded correspondence relationship. When the data does not have repeated data, the connection result is directly determined from the driven table.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of database, and particularly relate to a database query method and device. BACKGROUND

[0002] A database usually includes multiple data tables. When querying data in the database, a table join can be established between the multiple data tables through certain join conditions, so that the data tables are associated, and data can be obtained from the multiple data tables. Nested loop join (NLJ) is a common method of database join query, and databases such as MySQL, Dream, OceanBase, and PingCap support using this join query method. At present, it is desirable to have an improved solution to improve the query efficiency of the database. SUMMARY

[0003] One or more embodiments of the present specification describe a database query method and device to improve the query efficiency of the database. The specific technical solutions are as follows.

[0004] In a first aspect, an embodiment provides a database query method, wherein the database includes a driving table and a driven table, and the driving table and the driven table include multiple data; the method comprises:

[0005] When the join query request is received, repeated data detection is performed on the multiple data of the driving table to obtain repeated information of a single data in the multiple data;

[0006] For any first data in the multiple data of the driving table, it is determined whether the first data has repeated data and whether the first data is the first data in the repeated data based on the repeated information of the first data;

[0007] When it is determined that the first data has repeated data and the first data is the first data, data having a connection relationship with the first data is determined from the multiple data of the driven table to obtain a connection result, and a correspondence relationship between the first data and the connection result is recorded;

[0008] When it is determined that the first data has repeated data and the first data is not the first data, the connection result corresponding to the first data is obtained from the recorded correspondence relationship.

[0009] In an embodiment, the method further comprises:

[0010] When it is determined that the first data does not have repeated data, data having a connection relationship with the first data is directly determined from the multiple data of the driven table to obtain a connection result.

[0011] In an embodiment, the step of detecting the plurality of data of the driving table for duplicate data comprises:

[0012] reading batch data from the driving table in a batch reading manner;

[0013] detecting the batch data for duplicate data to obtain a plurality of non-identical numerical data;

[0014] when the number of the plurality of non-identical numerical data is less than a preset threshold, returning to execute the step of reading batch data from the driving table in a batch reading manner;

[0015] when the number of the plurality of non-identical numerical data is not less than the preset threshold, stopping reading data from the driving table, and determining the duplicate information of individual data in the read batch data based on the obtained non-identical numerical data.

[0016] In an embodiment, the step of detecting the plurality of data of the driving table for duplicate data comprises:

[0017] calculating the hash value corresponding to each of the plurality of data of the driving table;

[0018] detecting the plurality of data of the driving table for duplicate data based on the hash value of the data.

[0019] In an embodiment, the step of detecting the plurality of data of the driving table for duplicate data comprises:

[0020] sorting the plurality of data of the driving table based on the data value;

[0021] detecting the plurality of data of the driving table for duplicate data based on the obtained sorting result.

[0022] In an embodiment, the step of determining the connection result corresponding to each of the plurality of data of the driving table comprises:

[0023] sequentially determining the sorted plurality of data of the driving table as the first data, and executing the step of determining whether the first data exists duplicate data and whether the first data is the first data in the duplicate data based on the duplicate information of the first data; wherein the same duplicate data is arranged adjacent in the sorted plurality of data.

[0024] In an embodiment, the step of determining the data having a connection relationship with the first data from the plurality of data of the driven table comprises:

[0025] The first data are compared with the plurality of data of the driven table respectively based on the preset connection relationship logic to obtain data having a connection relationship with the first data.

[0026] In an embodiment, the driven table comprises a plurality of sub-tables, and the plurality of sub-tables are respectively stored in a plurality of devices included in the distributed system.

[0027] The step of determining the data having a connection relationship with the first data from the plurality of data of the driven table comprises:

[0028] The plurality of devices are respectively communicated with to determine the data having a connection relationship with the first data from a plurality of data included in the sub-tables of the plurality of devices to obtain corresponding sub-connection results.

[0029] The connection result corresponding to the first data is determined based on the sub-connection results respectively obtained from the plurality of sub-tables.

[0030] In a second aspect, embodiments provide a database query apparatus, wherein a database comprises a driven table and a driven table, and the driven table and the driven table comprise a plurality of data; the apparatus comprises:

[0031] A duplicate detection module is configured to perform duplicate data detection on the plurality of data of the driven table to obtain duplicate information of a single data in the plurality of data when a connection query request is received.

[0032] A determination module is configured to determine, for any first data in the plurality of data of the driven table, whether the first data has duplicate data based on the duplicate information of the first data, and whether the first data is the first data in the duplicate data.

[0033] A first connection module is configured to determine, when it is determined that the first data has duplicate data and the first data is the first data, data having a connection relationship with the first data from the plurality of data of the driven table to obtain a connection result and record a correspondence between the first data and the connection result.

[0034] A second connection module is configured to obtain, when it is determined that the first data has duplicate data and the first data is not the first data, a connection result corresponding to the first data from the recorded correspondence.

[0035] In an embodiment, the apparatus further comprises:

[0036] A third connection module is configured to directly determine, when it is determined that the first data has no duplicate data, data having a connection relationship with the first data from the plurality of data of the driven table to obtain a connection result.

[0037] In an embodiment, the duplicate detection module comprises:

[0038] a reading sub-module, configured to read batch data from the driving table in a batch reading manner;

[0039] a detecting sub-module, configured to detect the batch data for duplicate data, to obtain a plurality of non-identical numerical data;

[0040] a returning sub-module, configured to return to execute the reading sub-module when the number of the plurality of non-identical numerical data is less than a preset threshold;

[0041] a determining sub-module, configured to stop reading data from the driving table when the number of the plurality of non-identical numerical data is not less than the preset threshold, and determine the duplicate information of a single data in the batch data read based on the obtained non-identical numerical data.

[0042] In an embodiment, the apparatus further comprises:

[0043] an executing module, configured to determine the connection results corresponding to the plurality of data of the driving table based on the following manner:

[0044] sequentially determine the plurality of sorted data of the driving table as the first data, and perform the operation of determining whether the first data has duplicate data based on the duplicate information of the first data, and whether the first data is the first data in the duplicate data; wherein, in the plurality of sorted data, the same duplicate data are arranged adjacent to each other.

[0045] In an embodiment, when the first connection module determines the data having a connection relationship with the first data from the plurality of data of the driven table, the first connection module comprises:

[0046] comparing and judging the first data with the plurality of data of the driven table based on a preset connection relationship logic, to obtain the data having a connection relationship with the first data.

[0047] In an embodiment, the driven table comprises a plurality of sub-tables, and the plurality of sub-tables are respectively stored in a plurality of devices included in a distributed system;

[0048] When the first connection module determines the data having a connection relationship with the first data from the plurality of data of the driven table, the first connection module comprises:

[0049] communicate with the plurality of devices respectively, to determine the data having a connection relationship with the first data from the plurality of data included in the sub-tables of the plurality of devices, to obtain corresponding sub-connection results;

[0050] Determine the connection result corresponding to the first data based on the sub-connection results obtained from the plurality of sub-tables respectively.

[0051] In a third aspect, embodiments provide a computer-readable storage medium having stored thereon a computer program, which, when executed in a computer, causes the computer to perform the method of any one of the first aspect.

[0052] In a fourth aspect, embodiments provide a computing device comprising a memory and a processor, the memory having stored therein executable code, which, when executed by the processor, implements the method of any one of the first aspect.

[0053] In the method and device provided by the embodiments of the present specification, when performing table connection, repeated data detection is performed on a plurality of data of a driving table to obtain repeated information of a single data in the plurality of data, and the data in the driving table is judged by using the repeated information. For repeated data in the driving table, only one connection query is performed with a driven table, and the connection result is recorded. The same repeated data in the driving table is no longer connected with the driven table, but is directly obtained from the recorded connection result. In this way, not only the number of connection queries is reduced, but also the number of repeated scans of the driven table is reduced, and the efficiency of database query is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0055] Figure 1 The schematic diagram of the implementation scenario of an embodiment disclosed in the present specification;

[0056] Figure 2 The flowchart of a database query method provided by the embodiment;

[0057] Figure 3 The scenario schematic diagram of a distributed database provided by the embodiment;

[0058] Figure 4 The schematic block diagram of a database query device provided by the embodiment. DETAILED DESCRIPTION

[0059] The schemes provided by the present specification will be described below in combination with the drawings.

[0060] In the query of data in the database, table join is a common operation in querying database data. Table join is based on certain join conditions or join relationships to associate multiple data tables, and then obtain database data based on the associated data tables. In common databases, nested loop join can be used to connect and query data tables. Nested loop join is a table join method that relies on two layers of nested loops to obtain a connection result set when two data tables are connected. The two layers of nested loops include an outer loop and an inner loop. In nested loop join, the data table executed in the outer loop is called the driver table, and the data table executed in the inner loop is called the driven table.

[0061] The common mode of executing the table join process includes the row iteration volcano model and the vector iteration model. The row iteration volcano model processes one row of data in a data table at a time, while the vector iteration model processes a batch of data in a data table at a time. The method provided in this embodiment can be based on the vector iteration model. The vector iteration model can reduce the number of iterations and transfers of data between different operators in the database, thereby improving the computing speed of the database.

[0062] Figure 1 An implementation scenario of an embodiment disclosed in the present specification is shown. It shows that the driver table and the driven table contain multiple data, and the type and value of the data are only an example. In actual applications, the data in the driver table and the driven table can be any other data type. The multiple data in the driver table are shown in the form of row data, of which 256 rows of data are taken as an example. There are many duplicate data between the multiple rows of data in the driver table. After deduplication of the multiple rows of data in the driver table, the following column of data is obtained Figure 1 which contains 11, 12 and 13, for example. After performing the deduplication operation, the deduplicated driver table data is connected with the driven table, for example, the data “11” is connected with the M rows of data in the driven table one by one, which requires M times of connection query. After deduplication of the multiple rows of data in the driver table, only 3*M times of connection query is required, which greatly reduces the number of connection queries compared with 256*M times of connection query.

[0063] The following will be described in detail Figure 2 The embodiments of the present specification will be described in detail.

[0064] Figure 2 A flowchart of a database query method provided by an embodiment is shown. The data table of the database includes a driver table and a driven table, the driver table contains multiple data, and the driven table contains multiple data. The method is executed by a computing device, which can be implemented by any device, equipment, platform, equipment cluster, etc. with computing and processing capabilities. The method includes the following steps.

[0065] Step S210, when receiving the connection query request, the plurality of data of the driving table is repeated data detection, and the repetition information of each data in the plurality of data is obtained.

[0066] The computing device can generate a connection query request when receiving a database query request, or directly receive a connection query request sent by another device. The connection query request is used to indicate the establishment of the connection relationship between the driving table and the driven table.

[0067] When there are a large number of data in the driving table, the operation of determining the repeated data of the driving table increases the CPU load and the memory overhead. In order to improve the processing efficiency, the data of the driving table can be batch processed when the data amount of the driving table is large. When step S210 is executed, the batch data can be read from the driving table in the first place, and the batch data read is taken as the object of repeated data detection, and the repeated data detection is performed on the plurality of data. For example, a batch data can be set to contain a predetermined number of row data, etc. Different data in the driving table can be arranged by row, and a row of data can be understood as a data. All data in the driving table can also be read, and all data is taken as the object of repeated data detection.

[0068] The repeated data can be understood as the same value between different row data except the row number. That is, when the connection query is performed, the connection result between these data and the driven table is the same. The driving table can contain multiple groups of repeated data, and the values of different groups of repeated data are different. Each group of repeated data is a non-distinct value (NDV) data. For example, in Figure 1 After the driving table is deduplicated, three non-distinct value data 11, 12 and 13 can be obtained, which correspond to three groups of repeated data respectively. The number of repeated data can be at least two.

[0069] The repeated data detection of the plurality of data of the driving table can obtain the repetition information of each data in the plurality of data, which includes one or more of the following contents: whether the data has repeated data, the value of the repeated data, the number of repeated data, and the number of rows where the group of repeated data is located, etc. For example, for the value 11 of the first row of the driving table in Figure 1 The data has repeated data, and the rows where the repeated data is located include the third and fourth rows of data, etc.

[0070] When the plurality of data of the driving table is repeated data detection, various embodiments can be included. For example, the hash values corresponding to the plurality of data of the driving table can be calculated, and the plurality of data of the driving table is repeated data detection based on the hash values of the data.Figure 1 Taking the example of driving table containing data of numerical type, in actual application, the data in the driving table can be various data types. After calculating the hash values of the plurality of data in the driving table, the hash table corresponding to the driving table can be obtained, and the hash values in the hash table can uniquely represent the data in the driving table. Since the data amount of the hash table is small, the CPU load or memory overhead can be reduced, and therefore the calculation efficiency can be improved by using the hash table to remove duplicates.

[0071] In another embodiment, the driving table data can be removed based on the ordering of data values. In implementation, the plurality of data in the driving table can be ordered based on data values, and the plurality of data in the driving table can be detected for duplicate data based on the obtained ordering result. The plurality of data is ordered according to the size order of data values, and data with the same value will be arranged together, so that it can be determined from the ordering result which data are duplicate data.

[0072] In performing the join query, in order to reduce the CPU load and memory overhead, the number of NDVs of the plurality of data read from the driving table can be set as an upper limit. For example, the driving table data can be read in batches, and when the number of obtained NDVs is greater than or equal to a certain threshold, the reading of data from the driving table is stopped, and the join query is performed.

[0073] Specifically, the batch data can be read from the driving table in a batch reading manner, the batch data can be detected for duplicate data, and a plurality of non-identical numerical data can be obtained. When the number of the plurality of non-identical numerical data is less than a preset threshold, the step of reading the batch data from the driving table in the batch reading manner and detecting the batch data for duplicate data is returned to be performed. When the number of the plurality of non-identical numerical data is not less than the preset threshold, the reading of data from the driving table is stopped, and the duplicate information of individual data in the read batch data is determined based on the obtained non-identical numerical data. In an embodiment, the number of non-identical numerical data can be recorded, and the non-identical numerical data can also be recorded in a data table.

[0074] For example, the preset threshold can be configured as 256 or 512, and the data of the driving table can be read while being removed for duplicates. When the number of data after being removed for duplicates NDV is greater than or equal to the preset threshold, the reading of data from the driving table is stopped. The number of NDVs is the number of times of join query with the non-driving table.

[0075] In detecting the batch data for duplicate data, the batch data can be detected for duplicate data based on the hash values corresponding to the batch data respectively. The batch data can also be ordered based on data values, and the batch data can be detected for duplicate data based on the obtained ordering result.

[0076] In step S220, for any first data D1 in the plurality of data of the driven table, it is determined, based on the repetition information of the first data D1, whether the first data D1 has repeated data, and whether the first data D1 is the first data in the repeated data. When the first data D1 has repeated data, step S230 or S240 is performed. Specifically, when the first data D1 is the first data in the repeated data, step S230 is performed; when the first data D1 is not the first data, step S240 is performed. When the first data D1 has no repeated data, step S250 is performed.

[0077] In step S230, when it is determined that the first data D1 has repeated data, and the first data D1 is the first data in the repeated data, data having a connection relationship with the first data D1 is determined from the plurality of data of the driven table, a connection result is obtained, and a correspondence relationship between the first data D1 and the connection result is recorded.

[0078] In step S240, when it is determined that the first data D1 has repeated data, and the first data D1 is not the first data, the connection result corresponding to the first data D1 is obtained from the recorded correspondence relationship.

[0079] In step S250, when it is determined that the first data D1 has no repeated data, data having a connection relationship with the first data is directly determined from the plurality of data of the driven table, and a connection result is obtained.

[0080] In steps S230 and S250, the step of determining data having a connection relationship with the first data D1 from the plurality of data of the driven table can be understood as a one-time connection query. In the one-time connection query, the first data D1 needs to be compared and judged with all the data in the driven table one by one to determine whether the first data D1 and the data in the driven table have a preset connection relationship, and if so, the data in the driven table is added to the connection result. The preset connection relationship can include: the same value, greater than or less than the value, etc. When the data is compared and judged based on the preset connection relationship logic, for example, it can include judging whether the value of the first data D1 and the value of the data D2 in the driven table are the same, or whether the first data D1 is greater than the data D2, or whether the first data D1 is less than the data D2, etc. When the judgment result is yes or no, it is considered that the first data D1 and the data D2 have a connection relationship, and the data D2 can be added to the connection result. The specific connection relationship logic can be set as needed, and the above examples are only a list of multiple cases.

[0081] When the connection result is obtained, the correspondence between the first data D1 and the connection result can be recorded in the cache. After outputting the connection result of all data belonging to the repeated data of the first data D1, the correspondence between the first data D1 and the connection result can be deleted from the cache.

[0082] In determining the connection result of the plurality of data of the driving table, each data can be sequentially taken as the first data D1 in the original data order of the plurality of data in the driving table, and it is determined whether the data exists repeated data and whether it is the first data in the repeated data by combining the repeated information of the first data D1, and steps S220-S250 are executed. This way belongs to the ordered connection query mode.

[0083] Alternatively, the plurality of data of the driving table can be sorted based on the data value, so that the same repeated data in the sorted plurality of data is arranged adjacent to each other. In this way, the connection result corresponding to each of the plurality of data sorted by the driving table can be determined in sequence. This way belongs to the unordered connection query mode. When connection query is performed in this way, when step S240 is executed, the same repeated data can be obtained from the recorded correspondence to output the connection result multiple times, and the correspondence can be emptied from the cache recording the correspondence as soon as possible, reducing the occupation of the cache.

[0084] Based on the execution of steps S220-S250, for any group of repeated data existing in the driving table, connection query can be performed only once, without performing connection query for each data, that is, without repeatedly scanning (Rescan) the driven table for each data, thereby greatly reducing the number of connection queries.

[0085] In a distributed database scenario, data tables can be distributed in multiple devices. For example, the distributed system includes multiple devices, the driving table and the driven table are stored in different devices, and the driven table includes multiple sub-tables, and the multiple sub-tables are respectively stored in the multiple devices included in the distributed system. Figure 3 A distributed database scenario provided for an embodiment is shown in the figure. The driving table is stored in the computing device, and the sub-table 1, the sub-table 2, and the sub-table n included in the driven table are respectively stored in the device 1, the device 2, and the device n.

[0086] When performing repeated scanning on the driven table, the cost of resource release, scheduling restart, and network transmission in the distributed scenario is introduced, that is, when the data in the driving table is connected with the data in the multiple sub-tables, higher cost is required. Using the method of the embodiment, for the same repeated data of the driving table, only one repeated scanning can be performed between the multiple sub-tables of the driven table, and the recorded connection result of other repeated data can be read from the cache, thereby greatly reducing the cost of connection query in the distributed scenario and improving the query efficiency.

[0087] In an embodiment of the present scenario, the computing device can respectively communicate with multiple devices of the distributed system, determine data existing a connection relationship with the first data D1 from multiple data contained in multiple sub-tables of the multiple devices, and obtain corresponding sub-connection results; then, based on the sub-connection results obtained from the multiple sub-tables respectively, determine the connection result corresponding to the first data D1.

[0088] In the execution of the connection query, the first data D1 can be compared and judged with multiple data of any one sub-table based on the preset connection relationship logic, and the corresponding sub-connection result is obtained. When multiple sub-connection results corresponding to multiple sub-tables are obtained, the multiple sub-connection results can be spliced to obtain the complete connection result corresponding to the first data D1.

[0089] In the present specification, the "first" of the first data and the like is only for the convenience of distinguishing and describing, and does not have any limiting meaning.

[0090] The above describes a specific embodiment of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments, and the desired results can still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be implemented in the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible, or can be advantageous.

[0091] Figure 4 A schematic block diagram of a database query device for an embodiment is shown. The database includes a driver table and a driven table, and the driver table and the driven table contain multiple data. The device is deployed in a computing device, which can be implemented by any device, equipment, platform, equipment cluster, etc. with computing and processing capabilities. The device embodiment corresponds to the method embodiment shown. The device 400 includes: Figure 2

[0092] The duplicate detection module 410 is configured to, when receiving the connection query request, detect the multiple data of the driver table for duplicate data, and obtain the duplicate information of a single data in the multiple data;

[0093] The determination module 420 determines, for any first data in the multiple data of the driver table, whether the first data has duplicate data based on the duplicate information of the first data, and whether the first data is the first data in the duplicate data:

[0094] ​The first connection module 430 is configured to, when it is determined that the first data has duplicate data and the first data is the first data, determine data having a connection relationship with the first data from the plurality of data in the driven table to obtain a connection result, and record a correspondence between the first data and the connection result.

[0095] The second connection module 440 is configured to, when it is determined that the first data has duplicate data and the first data is not the first data, obtain the connection result corresponding to the first data from the recorded correspondence.

[0096] In an embodiment, the apparatus 400 further comprises:

[0097] The third connection module 450 is configured to, when it is determined that the first data has no duplicate data, directly determine data having a connection relationship with the first data from the plurality of data in the driven table to obtain a connection result.

[0098] In an embodiment, the duplicate detection module 410 comprises:

[0099] The reading submodule (not shown in the figure) is configured to read batch data from the driven table in a batch reading manner;

[0100] The detection submodule (not shown in the figure) is configured to detect duplicate data in the batch data to obtain a plurality of non-identical numerical data;

[0101] The returning submodule (not shown in the figure) is configured to, when the number of the plurality of non-identical numerical data is less than a preset threshold, return to execute the reading submodule;

[0102] The determination submodule (not shown in the figure) is configured to, when the number of the plurality of non-identical numerical data is not less than the preset threshold, stop reading data from the driven table, and determine duplicate information of a single data in the read batch data based on the obtained non-identical numerical data.

[0103] In an embodiment, the duplicate detection module 410 is specifically configured to:

[0104] Calculate a plurality of hash values respectively corresponding to the plurality of data in the driven table;

[0105] Detect duplicate data in the plurality of data in the driven table based on the hash values of the data.

[0106] In an embodiment, the duplicate detection module 410 is specifically configured to:

[0107] Sort the plurality of data in the driven table based on data values;

[0108] Based on the obtained sorting results, duplicate data detection is performed on multiple data in the driving table.

[0109] In one embodiment, the device 400 further includes:

[0110] The execution module (not shown in the figure) is configured to determine the join results corresponding to multiple data points in the driving table based on the following method:

[0111] The sorted data of the driving table is sequentially determined as the first data, and the operation of determining whether the first data has duplicate data and whether the first data is the first data among the duplicate data is performed based on the duplicate information of the first data; wherein, in the sorted data, the same duplicate data are arranged adjacently.

[0112] In one implementation, when the first connection module 430 determines data that has a connection relationship with the first data from multiple data in the driven table, it includes:

[0113] Based on the preset connection relationship logic, the first data is compared and judged with multiple data in the driven table to obtain data that has a connection relationship with the first data.

[0114] In one implementation, the driven table includes multiple sub-tables, which are stored in multiple devices included in the distributed system.

[0115] When the first connection module 430 determines data that has a connection relationship with the first data from multiple data in the driven table, it includes:

[0116] The system communicates with the multiple devices respectively, and determines the data that has a connection relationship with the first data from the multiple data contained in the sub-tables of the multiple devices, thereby obtaining the corresponding sub-connection results;

[0117] Based on the sub-join results obtained from the multiple sub-tables, the join result corresponding to the first data is determined.

[0118] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0119] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform... Figures 1 to 3 Any one of the methods described.

[0120] The embodiments of the present specification also provide a computing device, comprising a memory and a processor, the memory has stored executable codes, when the processor executes the executable codes, the following are implemented Figures 1 to 3 The method of any one of the preceding claims.

[0121] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between each of the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the storage medium and the computing device embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the method embodiments.

[0122] Those skilled in the art should be aware that, in one or more examples described above, the functions described in the embodiments of the present application can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium.

[0123] The above specific embodiments further explain the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above description is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A database query method, wherein, The database includes a driving table and a driven table, each containing multiple data entries; the method includes: Upon receiving a connection query request, duplicate data detection is performed on multiple data in the driving table to obtain duplicate information of a single data in the multiple data. For any first data among multiple data in the driving table, determine whether the first data has duplicate data based on the duplication information of the first data, and whether the first data is the first data among the duplicate data; When it is determined that the first data contains duplicate data and the first data is the first data, the data that has a connection relationship with the first data is determined from the multiple data of the driven table, the connection result is obtained, and the correspondence between the first data and the connection result is recorded in the cache; When it is determined that the first data contains duplicate data and the first data is not the first data, the connection result corresponding to the first data is obtained from the correspondence recorded in the cache; When it is determined that there is no duplicate data in the first data, the data that has a connection relationship with the first data is directly determined from the multiple data in the driven table to obtain the connection result, and the connection result is not recorded in the cache.

2. The method according to claim 1, wherein the step of performing duplicate data detection on multiple data in the driving table includes: Batch data is read from the driver table in a batch read manner; Duplicate data detection is performed on the batch data to obtain several non-identical numerical data. When the number of the several non-identical numerical data is less than a preset threshold, return to the step of reading batch data from the driver table in a batch reading manner; When the number of the aforementioned non-identical numerical data is not less than a preset threshold, data reading from the driving table is stopped, and based on the obtained non-identical numerical data, the duplicate information of a single data in the batch of data that has been read is determined.

3. The method according to claim 1, wherein the step of performing duplicate data detection on multiple data in the driving table includes: Calculate the hash values ​​corresponding to the multiple data items in the driving table; Based on the hash value of the data, duplicate data detection is performed on multiple data in the driving table.

4. The method according to claim 1, wherein the step of performing duplicate data detection on multiple data in the driving table includes: Sort multiple data in the driving table based on the data values; Based on the obtained sorting results, duplicate data detection is performed on multiple data in the driving table.

5. The method according to claim 1, wherein the join results corresponding to the multiple data in the driving table are determined based on the following method: The sorted data from the driving table is sequentially identified as the first data. The step of determining whether the first data contains duplicate data and whether the first data is the first data among the duplicate data is then performed based on the duplicate information of the first data. In the sorted data, identical duplicate data are arranged adjacently.

6. The method according to claim 1, wherein the step of determining data from the plurality of data in the driven table that has a connection relationship with the first data comprises: Based on the preset connection relationship logic, the first data is compared and judged with multiple data in the driven table to obtain data that has a connection relationship with the first data.

7. The method according to claim 1, wherein the driven table comprises multiple sub-tables, and the multiple sub-tables are respectively stored in multiple devices included in the distributed system; The step of determining the data that has a connection relationship with the first data from multiple data in the driven table includes: The system communicates with the multiple devices respectively, and determines the data that has a connection relationship with the first data from the multiple data contained in the sub-tables of the multiple devices, thereby obtaining the corresponding sub-connection results; Based on the sub-join results obtained from the multiple sub-tables, the join result corresponding to the first data is determined.

8. A database query device, wherein, The database includes a driving table and a driven table, which contain multiple data entries; the device includes: The deduplication module is configured to perform duplicate data detection on multiple data in the driving table when a connection query request is received, and obtain the duplicate information of a single data in the multiple data; The determining module, for any one of the multiple data in the driving table, determines whether the first data has duplicate data and whether the first data is the first data among the duplicate data based on the duplication information of the first data; The first connection module is configured to, when it is determined that there is duplicate data in the first data and the first data is the first data, determine the data that has a connection relationship with the first data from multiple data in the driven table, obtain the connection result, and record the correspondence between the first data and the connection result in the cache; The second connection module is configured to, when it is determined that the first data contains duplicate data and the first data is not the first data, obtain the connection result corresponding to the first data from the correspondence recorded in the cache; The third connection module is configured to, when it is determined that there is no duplicate data in the first data, directly determine the data that has a connection relationship with the first data from multiple data in the driven table, obtain the connection result, and not record the connection result in the cache.

9. The apparatus according to claim 8, wherein the checkweighing module comprises: The read submodule is configured to read batch data from the driver table in a batch read manner; The detection submodule is configured to perform duplicate data detection on the batch data to obtain several non-identical numerical data. The return submodule is configured to return to execute the reading submodule when the number of the plurality of non-identical numerical data is less than a preset threshold. The determination submodule is configured to stop reading data from the driving table when the number of the plurality of different numerical data is not less than a preset threshold, and to determine the duplicate information of a single data in the batch data that has been read based on the obtained different numerical data.

10. The apparatus of claim 8, further comprising: The execution module is configured to determine the join results corresponding to multiple data points in the driving table based on the following method: The sorted data of the driving table is sequentially determined as the first data, and the operation of determining whether the first data has duplicate data and whether the first data is the first data among the duplicate data is performed based on the duplicate information of the first data; wherein, in the sorted data, the same duplicate data are arranged adjacently.

11. The apparatus according to claim 8, wherein the driven table comprises a plurality of sub-tables, and the plurality of sub-tables are respectively stored in a plurality of devices included in the distributed system; When the first connection module determines data that has a connection relationship with the first data from multiple data in the driven table, it includes: The system communicates with the multiple devices respectively, and determines the data that has a connection relationship with the first data from the multiple data contained in the sub-tables of the multiple devices, thereby obtaining the corresponding sub-connection results; Based on the sub-join results obtained from the multiple sub-tables, the join result corresponding to the first data is determined.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.

13. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-7.

Images