Data table processing method, device, and storage medium

Abstract

Embodiments of the present disclosure provide a data table processing method and device and a storage medium. The left table and the right table in two data tables to be connected are connected by using a hash connection method to obtain first associated data associated between the left table and the right table. The first associated data is filtered according to a preset non-equality filtering condition to determine second associated data in the first associated data that meets the preset non-equality filtering condition. The second associated data is identified in a preset data structure according to a target connection type of the two data tables to obtain corresponding position information of the second associated data in the left table and / or corresponding position information of the second associated data in a corresponding hash table of the right table. The left table and / or the right table are processed according to the target connection type of the two data tables and the preset data structure to generate a target connection table. Based on the assistance of the preset data structure, non-equality connection can be implemented on the basis of hash connection, different target connection types are supported, and the processing efficiency is relatively high.

Description

Data table processing methods, equipment, and storage media Technical Field

[0001] This disclosure relates to the field of computer and network communication technology, and in particular to a method, apparatus and storage medium for processing data tables. Background Technology

[0002] The join operation of data tables refers to the operation of merging two data tables that have a certain relationship into a single data table. Different processing methods are used for data that are related and data that are not related. Therefore, the join operation of data tables is further subdivided into different join types, including cross join, inner join, outer join, full join, semi join, and anti join.

[0003] However, in the existing technology, it is relatively easy to implement join operations on data tables based on equality conditions, while non-equality conditions are relatively more complex, and the combination of equality and non-equality conditions is even more complex. The existing technology cannot support the combination of equality and non-equality conditions for different join types. Summary of the Invention

[0004] This disclosure provides a data table processing method, device, and storage medium to provide a new data table connection method.

[0005] In a first aspect, embodiments of this disclosure provide a method for processing a data table, including:

[0006] The left and right tables of the two data tables to be joined are equi-joined using a hash join method to obtain the first associated data between the left and right tables;

[0007] The first associated data is filtered according to a preset non-equivalence filtering condition to determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition;

[0008] Based on the target connection type of the two data tables, identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in the preset data structure;

[0009] Based on the target join type of the two data tables and the preset data structure, the left table and / or the right table are processed to generate the target join table.

[0010] Secondly, embodiments of this disclosure provide a data table processing apparatus, comprising:

[0011] An equi-join unit is used to perform an equi-join on the left and right tables of two data tables to be joined using a hash join method, and obtain the first associated data between the left and right tables;

[0012] A non-equivalence filtering unit is used to filter the first associated data according to a preset non-equivalence filtering condition, and determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition;

[0013] An identification unit is used to identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in a preset data structure according to the target connection type of the two data tables.

[0014] The generation unit is used to process the left table and / or the right table according to the target connection type of the two data tables and the preset data structure to generate a target connection table.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory;

[0016] The memory stores computer-executed instructions;

[0017] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the data table processing method described in the first aspect and various possible designs of the first aspect.

[0018] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the data table processing method described in the first aspect and various possible designs of the first aspect.

[0019] Fifthly, embodiments of this disclosure provide a computer program product, including computer execution instructions, which, when executed by a processor, implement the data table processing method described in the first aspect and various possible designs of the first aspect.

[0020] The data table processing method, device, and storage medium provided in this disclosure employ a hash join to perform an equi-join on the left and right tables of two data tables to be joined, obtaining first associated data between the left and right tables. The first associated data is then filtered according to a preset non-equi-join filtering condition to determine second associated data that satisfies the preset non-equi-join filtering condition. Based on the target join type of the two data tables, the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table are identified in a preset data structure. The left table and / or the right table are then processed according to the target join type of the two data tables and the preset data structure to generate a target join table. With the assistance of the preset data structure, non-equi-joins can be implemented based on hash joins, supporting different target join types and exhibiting high processing efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a scenario example diagram of a data table processing method provided in an embodiment of this disclosure;

[0023] Figure 2 is a schematic flowchart of a data table processing method provided in an embodiment of this disclosure;

[0024] Figure 3 is a structural block diagram of a data table processing device provided in an embodiment of this disclosure;

[0025] Figure 4 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] First, we'll define the different join types. A join operation combines two related tables into one. It typically involves one or more join conditions to define the relationship between the two tables, ultimately returning the joined result. The join operation is further subdivided into several different types based on how related and unrelated data are handled:

[0028] 1) Cross Join: Two tables are joined by a Cartesian product, and there is no relationship between them.

[0029] 2) Inner Join: Ultimately, only the data from the two tables that are joined is returned.

[0030] 3) Outer Join: Building upon Inner Join, if data in one table is not linked, it will be added back to the join table, and NULL values ​​will be filled in for the other table. Filling NULL values ​​in the right table is called a Left Outer Join, and filling NULL values ​​in the left table is called a Right Outer Join.

[0031] 4) Full Join: Similar to Outer Join, it adds data that is not related between two tables.

[0032] 5) Semi-join: Returns only the data from the join between two tables, and returns only one result even when one data point is joined with multiple data points. Returning data from the left table is called a left semi-join, and returning data from the right table is called a right semi-join.

[0033] 6) Anti Join: The opposite of Semi Join, it only returns data from the two tables that are not related. A Left Anti Join returns data from the left table that is not related, and a Right Anti Join returns data from the right table that is not related.

[0034] In existing technologies, the common practice for joining two data tables is to create a hash table for the right table and then match each row of data in the left table against the hash table. This method is called Hash Join. However, this method has a drawback: it only supports equality join conditions, such as table1.col = table2.col. It cannot handle non-equality join conditions, such as table1.col > table2.col, because the hash table cannot establish a join based on non-equality conditions. These other join conditions, which differ from equality join conditions, are usually called non-equality joins or "Other Conjuncts," especially when equality and non-equality joins are combined.

[0035] In the standard SQL specification, there are many join types, such as Full Join, Inner Join, Left Outer Join, Right Outer Join, Left Semi Join, etc. Each join type handles the joined and unjoined data differently, which increases the difficulty of implementing non-equi-joins and non-equi-joins in combination.

[0036] Supporting non-equi-value joins is a challenging problem, and many computing engines still haven't implemented this feature. The simplest approach is to abandon Hash Join and instead degenerate into CrossJoin. This involves first performing a Cartesian product on the two tables to obtain all the results, then filtering them using the join condition to return the final result. However, this method typically has high time complexity and poor performance.

[0037] Another common approach is to first establish a relationship between the two tables using equality conditions, and then filter the related data using non-equality conditions to obtain the related data. However, this method is usually only applicable to Inner Join and cannot be used for all Join types, lacking universality.

[0038] To address the aforementioned technical issues, this disclosure provides a data table processing method that, with the aid of a preset data structure, enables non-equi-joins to be implemented based on hash joins. This method supports different target join types. The preset data structure identifies the position information of the data after equi-joins and non-equi-joins in the left table and / or the position information in the right table. Consequently, the required data can be obtained based on the preset data structure for different target join types, and a target join table can be generated.

[0039] The data table processing method provided in this disclosure can be applied to electronic devices such as terminal devices or servers. Taking a server as an example, as shown in Figure 1, the server can use a hash join to perform an equi-join on the left and right tables of the two data tables to be joined, obtaining the first associated data between the left and right tables; filter the first associated data according to a preset non-equi-join filtering condition to determine the second associated data in the first associated data that satisfies the preset non-equi-join filtering condition; identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in a preset data structure according to the target connection type of the two data tables; process the left table and / or the right table according to the target connection type of the two data tables and the preset data structure to generate a target join table.

[0040] The data table processing method of this disclosure will be described in detail below with reference to specific embodiments.

[0041] Referring to Figure 2, which is a schematic flowchart of a data table processing method according to an embodiment of this disclosure, the method of this embodiment can be applied to electronic devices such as terminal devices or servers. The data table processing method includes:

[0042] S201. Using a hash join, perform an equi-join on the left and right tables of the two data tables to be joined, and obtain the first associated data between the left and right tables.

[0043] In this embodiment, the two data tables to be joined include a left table and a right table. A hash table is obtained from the right table, that is, a hash function is used to calculate the hash value of the specified key in the right table, and the key and its corresponding value are mapped to the position corresponding to the hash value to obtain the hash table. Since the hash table is obtained from the entire right table, and the left table needs to be matched with the hash table, the data table with a smaller amount of data (number of data rows) can be used as the right table to avoid the hash table being too large and occupying a lot of memory, and also to avoid the hash join consuming too many computing resources.

[0044] Furthermore, based on the hash table corresponding to the right table, an equi-join is performed on the left and right tables using a hash join. This involves calculating the hash value of each row of data in the left table using a hash function, and then looking up the hash value in the hash table. This achieves an equi-join between the left and right tables based on the key, determining the first associated data (with the same key) between the two tables. The specific hash join process will not be detailed here.

[0045] S202. Filter the first associated data according to the preset non-equivalence filtering conditions, and determine the second associated data in the first associated data that meets the preset non-equivalence filtering conditions.

[0046] In this embodiment, based on the first associated data obtained by equi-joining, the first associated data is filtered according to a preset non-equi-joining filtering condition. The non-equi-joining filtering is applied to a certain value in the first associated data, or it can be applied to the aforementioned key. Taking a certain value in the first associated data as an example, the preset non-equi-joining filtering condition can be that the value is greater than a preset threshold, or the value is less than a preset threshold, or the value is not equal to a preset threshold, etc. Based on the preset non-equi-joining filtering condition, second associated data that meets the preset non-equi-joining filtering condition can be filtered out from the first associated data.

[0047] S203. Based on the target connection type of the two data tables, identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in the preset data structure.

[0048] In this embodiment, during the equi-join and non-equi-join filtering process, a preset data structure can be used to identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table, based on the target join type of the two data tables. Specifically, if the target join type is a join type related to the left table, such as a left outer join, left anti-join, or left half join, then a preset data structure is needed to identify the position information of the second associated data in the left table, but not necessarily in the hash table corresponding to the right table. If the target join type is a join type related to the right table, such as a right outer join, right anti-join, or right half join, then a preset data structure is needed to identify the position information of the second associated data in the hash table corresponding to the right table, but not necessarily in the left table. If the target join type is a join type related to both the left and right tables, such as a full join, then both the position information of the second associated data in the left table and the position information of the second associated data in the hash table corresponding to the right table need to be identified in the preset data structure.

[0049] S204. Based on the target connection type of the two data tables and the preset data structure, process the left table and / or the right table to generate a target connection table.

[0050] In this embodiment, the data required to generate the final target join table differs depending on the target join type of the two data tables. The required data can be determined from the left and / or right tables using a preset data structure, and then the target join table can be generated based on this data. For example, for a full join, the data to be supplemented in the coordinates and the data to be supplemented in the right table can be determined based on the second associated data and the preset data structure. The target join table can then be generated based on the data to be supplemented in the left and right tables of the second associated data. Similarly, for a left inverse join, the data in the left table excluding the second associated data can be determined using the preset data structure, and the target join table can be generated based on this data. Likewise, for a right inverse join, the data in the right table excluding the second associated data can be determined using the preset data structure, and the target join table can be generated based on this data. Other cases will not be listed here.

[0051] It should be noted that the two data tables in this embodiment can be data tables in a columnar storage engine. Correspondingly, the preset data structure is also stored in a columnar manner in the columnar storage engine to facilitate fast querying and improve the generation efficiency of the target join table.

[0052] The data table processing method provided in this embodiment performs an equi-join on the left and right tables of two data tables to be joined using a hash join, obtaining the first associated data between the left and right tables. The first associated data is then filtered according to a preset non-equi-join filtering condition to determine the second associated data that satisfies the preset non-equi-join filtering condition. Based on the target join type of the two data tables, the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table is identified in a preset data structure. Based on the target join type of the two data tables and the preset data structure, the left table and / or the right table are processed to generate a target join table. With the assistance of the preset data structure, non-equi-joins can be implemented on the basis of hash joins, supporting different target join types and exhibiting high processing efficiency.

[0053] Based on any of the above embodiments, for cases where the target connection type is a full join, right outer join, right antijoin, right half join, etc., a preset data structure is required to identify the position information of the second associated data in the hash table corresponding to the right table. In this embodiment, a data structure corresponding to the hash table can be used to maintain the usage status information of each piece of data in the right table, that is, the usage status information of each element included in each offset position of the hash table. Each offset position of the hash table represents the position corresponding to each hash value in the hash table. Since there may be a case where one key corresponds to multiple values ​​in the right table, there may be multiple elements at one offset position of the hash table to correspond to different rows of data with the same key in the right table. One of the elements can be a pointer to a row of data.

[0054] In this embodiment, the data structure corresponding to the hash table maps all matching rows in the right table for each key in the hash table to a one-dimensional array. Each row of data in the right table corresponds to an element in the array, and the array stores whether the row has been used. Initially, all usage status information in the data structure corresponding to the hash table can be initialized to "unused". If a row of data is used later, the corresponding usage status information is modified.

[0055] Furthermore, after identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, the usage status information of the element corresponding to the second associated data can be modified in the data structure corresponding to the hash table according to the position information of the second associated data in the hash table corresponding to the right table.

[0056] In this embodiment, the position information of the second associated data in the hash table corresponding to the right table may include the offset position of the second associated data in the hash table and the element position information at that offset position (i.e., which element in the offset position corresponds to the second associated data). The corresponding position in the data structure corresponding to the hash table can be determined based on the offset position information and the element position information at that offset position, and then the usage status information can be modified at the corresponding position, that is, "unused" can be changed to "used". Specifically, for the case where the target join type is a right half join, the usage status information can be atomically modified at the corresponding position based on the offset position information and the element position information at that offset position. That is, only the first instance of changing "unused" to "used" is considered, and subsequent uses are not considered.

[0057] Furthermore, when querying the usage status information of a certain data in the right table, the corresponding position in the data structure of the hash table can be determined based on the information of the offset position of the data in the hash table and the position information of the element in the offset position, thereby obtaining the usage status information at the corresponding position, which is the usage status information of the required data.

[0058] Based on the above embodiments, when using a preset data structure to identify the position information of the second associated data in the hash table corresponding to the right table, the information of the offset position of the second associated data in the hash table and the element position information in the offset position can be identified respectively. Specifically, the preset data structure can include a first sub-data structure and a second sub-data structure. The first sub-data structure records the information of the offset position of each second associated data in the hash table, and the second sub-data structure records the element position information in the corresponding offset position of each second associated data.

[0059] Optionally, during equi-join, a first sub-data structure can be used to record the offset position information of each first associated data in the hash table, and a second sub-data structure can be used to record the element position information of each first associated data in the corresponding offset position. During non-equi-join filtering, an array (denoted as the fourth sub-data structure) in the preset data structure can be used to record the judgment result of whether the first associated data meets the preset non-equi-join filtering conditions. By combining the two arrays, the offset position information of the first associated data in the hash table that meets the preset non-equi-join filtering conditions and the element position information of the corresponding offset position can be obtained, which is also the offset position information of the second associated data in the hash table and the element position information of the corresponding offset position.

[0060] Based on any of the above embodiments, for cases where the target connection type is a full connection, left outer connection, left anti-connection, left half connection, etc., a preset data structure is required to identify the position information of the second associated data in the left table, such as row number.

[0061] Since the second associated data is the data in the first associated data that meets the preset non-equivalence filtering condition, in one optional method, when performing equi-join, an array (denoted as the third sub-data structure) can be used in the preset data structure to record the position information of the first associated data in the left table. When performing non-equivalence filtering, another array (denoted as the fourth sub-data structure) can be used in the preset data structure to record the judgment result of whether the first associated data meets the preset non-equivalence filtering condition. By combining the two arrays, the position information of the first associated data that meets the preset non-equivalence filtering condition in the left table can be obtained, which is also the position information of the second associated data in the left table.

[0062] Optionally, in a columnar storage engine, where the two data tables are data tables within the columnar storage engine, to efficiently obtain the third sub-data structure, a row position information column can be added to the left table in the columnar storage engine before the equi-join. This column records the position information (such as row number) of each row in the left table. Then, during the equi-join, a hash join is used to perform an equi-join on the left and right tables after adding the row position information column to obtain the first associated data. In this way, the first associated data also contains a row position information column, which can be used to identify the corresponding position information of the first associated data in the left table. The row position information column of the first associated data can then serve as the aforementioned third sub-data structure.

[0063] Based on the above embodiments, the processing method of the above data table will be illustrated below with specific examples.

[0064] Example 1

[0065] In this example, the target join type is a Full Join. Assume the left table key is ID, the ID column data is [1,1,2,3], and the right table key(ID)+value is [(1,10),(1,20),(3,10),(5,10)]. The default non-equality filtering condition is value greater than 10. The specific processing method for the data tables is as follows:

[0066] 1) Create a hash table for the right table, and construct the first and second sub-data structures. Construct the data structure corresponding to the hash table, and initialize the state information of each element in the data structure corresponding to the hash table to "False" to indicate "unused"; the details are as follows:

[0067] The first sub-data structure records the offset position information of each second associated data in the hash table, and the second sub-data structure records the element position information of each second associated data in the corresponding offset position.

[0068]

[0069] 2) Add a row position information column to the left table to record the position information of each row in the left table, as shown below:

[0070] ID row position information column 10112233 surface

[0071] 3) Use a hash join to perform an equi-join on the left and right tables, obtaining the first associated data between them. During the equi-join process, a third sub-data structure can be used to identify the position information of the first associated data in the left table, and a first sub-data structure and a second sub-data structure can be used to record the position information of the first associated data in the hash table corresponding to the right table. The specific correspondence is shown in the intermediate table below:

[0072]

[0073] In the equi-join, the data in the 0th row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; the data in the 1st row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; and the data in the 3rd row of the left table with ID 3 can be associated with (3,10) of the right table.

[0074] 4) Filter the first associated data according to the preset non-equivalence filtering condition (value greater than 10), determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition, and use the fourth sub-data structure to record the judgment result of whether the first associated data meets the preset non-equivalence filtering condition (where 0 indicates not met, 1 indicates met). The specific correspondence is shown in the intermediate table below:

[0075]

[0076] Among them, only the first associated data in the left table with ID 1 in row 0 and the right table (1,20) and the first associated data in the left table with ID 1 in row 1 and the right table (1,20) satisfy the preset non-equal value filtering condition (value greater than 10), that is, these two first associated data are regarded as the second associated data.

[0077] By combining the third and fourth sub-data structures (only considering those with a result of 1), we can determine that the second associated data is located in rows 0 and 1 of the left table. Combining the fourth, first, and second sub-data structures, we can determine the offset position of the second associated data in the hash table and the corresponding element position (101, 1). Based on this offset position and element position, we modify the usage status information of the corresponding element in the hash table from "False" to "True," indicating "used," as shown below:

[0078]

[0079] 5) Based on the position information of the second related data in the left table in the preset data structure (third sub-data structure and fourth sub-data structure), determine the data in the left table other than the second related data as the data to be supplemented in the left table; based on the data structure corresponding to the hash table, determine the data in the right table other than the second related data, and determine the data in the right table other than the second related data as the data to be supplemented in the right table; based on the second related data, the data to be supplemented in the left table and the data to be supplemented in the right table, generate the target join table, where the missing data in the data to be supplemented is filled with "null", and the resulting target join table is shown below:

[0080]

[0081] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0082] Furthermore, the position information of each row in the left table recorded in the position information column of step 2 is the position information of each row in its data block. In subsequent steps, equi-joins and non-equi-joins can be performed at the data block level. Specifically, after completing equi-joins and non-equi-joins with the right table for each data block of the left table, a set of second associated data can be determined. At the same time, the position information of the second associated data in the left table can be determined from the preset data structure through the traversal process. Then, the data in the data block other than the second associated data can be determined as the data to be supplemented in the left table. The set of second associated data and the data to be supplemented in the left table can be written into the target join table without waiting for all data in the left table to be completed before writing it into the target join table.

[0083] The data to be supplemented in the right table can only be determined after all data blocks in the left table have completed equi-joins and non-equi-join filtering. Therefore, only the data structure corresponding to the hash table obtained after completing equi-joins and filtering of all data blocks in the left table can reflect which data in the right table has not been associated with the left table. Thus, by traversing the data structure corresponding to the hash table obtained at the end, the data to be supplemented in the right table is determined and written into the target join table.

[0084] Example 2

[0085] In this example, the target join type is a left outer join. We only need to use a pre-defined data structure to identify the position of the second related data in the left table. Assuming the left table key is ID, and the ID column data is [1,1,2,3], the right table key(ID)+value is [(1,10),(1,20),(3,10),(5,10)], and the pre-defined non-equality filtering condition is value greater than 10. The specific processing method for the data tables is as follows:

[0086] 1) Create a hash table for the right table, as follows:

[0087]

[0088] 2) Add a row position information column to the left table to record the position information of each row in the left table, as shown below:

[0089] ID row position information column 10112233 surface

[0090] 3) Use a hash join to perform an equi-join on the left and right tables, obtaining the first associated data between them. During the equi-join process, a third sub-data structure can be used to identify the position information of the first associated data in the left table; the specific correspondence is shown in the intermediate table below:

[0091]

[0092] In the equi-join, the data in the 0th row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; the data in the 1st row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; and the data in the 3rd row of the left table with ID 3 can be associated with (3,10) of the right table.

[0093] 4) Filter the first associated data according to the preset non-equivalence filtering condition (value greater than 10), determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition, and use the fourth sub-data structure to record the judgment result of whether the first associated data meets the preset non-equivalence filtering condition (where 0 indicates not met, 1 indicates met). The specific correspondence is shown in the intermediate table below:

[0094]

[0095] Among them, only the first associated data in the left table with ID 1 in row 0 and (1,20) in the right table, and the first associated data in the left table with ID 1 in row 1 and (1,20) in the right table satisfy the preset non-equal value filtering condition (value greater than 10), and are used as the second associated data.

[0096] 5) Based on the position information of the second related data in the left table in the preset data structure (third sub-data structure and fourth sub-data structure), the data in the left table other than the second related data is identified as the data to be supplemented in the left table; based on the second related data and the data to be supplemented in the left table, a target join table is generated, where the missing data in the data to be supplemented is filled with "null", and the resulting target join table is shown below:

[0097]

[0098] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0099] Furthermore, the position information of each row in the left table recorded in the position information column of step 2 is the position information of each row in its data block. In subsequent steps, equi-joins and non-equi-joins can be performed at the data block level. Specifically, after completing equi-joins and non-equi-joins with the right table for each data block of the left table, a set of second associated data can be determined. At the same time, the position information of the second associated data in the left table can be determined from the preset data structure through the traversal process. Then, the data in the data block other than the second associated data can be determined as the data to be supplemented in the left table. The set of second associated data and the data to be supplemented in the left table can be written into the target join table without waiting for all data in the left table to be completed before writing it into the target join table.

[0100] Example 3

[0101] In this example, the target join type is a left anti-join. We only need to use a preset data structure to identify the position information of the second related data in the left table. Assuming the key in the left table is ID, and the ID column data is [1,1,2,3], and the key(ID)+value in the right table is [(1,10),(1,20),(3,10),(5,10)], the preset non-equality filtering condition is that the value is greater than 10. The specific processing method for the data table is as follows:

[0102] Steps 1-4 are the same as steps 1-4 in Example 2;

[0103] 5) Based on the position information of the second associated data in the preset data structure (third sub-data structure and fourth sub-data structure) corresponding to the data in the left table, determine the data in the left table other than the second associated data. Generate a target join table based on the data in the left table other than the second associated data (only including the data in the left table, no need to add null values). The resulting target join table is shown below:

[0104] Target join table left table ID23 surface

[0105] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0106] Furthermore, the position information of each row in the left table recorded in the position information column of step 2 is the position information of each row in its data block. In subsequent steps, equi-joins and non-equi-joins can be performed at the data block level. Specifically, after completing equi-joins and non-equi-joins with the right table for each data block of the left table, a set of second associated data can be determined. At the same time, the position information of the second associated data in the left table can be determined from the preset data structure through the traversal process. Thus, the data in the data block other than the second associated data can be determined. The data in the data block other than the second associated data can be written to the target join table without waiting for all data in the left table to complete equi-joins and non-equi-joins before writing it to the target join table.

[0107] Example 4

[0108] In this example, the target join type is a left semi join. We only need to use a preset data structure to identify the position information of the second related data in the left table. Assuming the key in the left table is ID, and the ID column data is [1,1,2,3], and the key(ID)+value in the right table is [(1,10),(1,20),(3,10),(5,10)], the preset non-equality filtering condition is that the value is greater than 10. The specific processing method for the data tables is as follows:

[0109] Steps 1-4 are the same as steps 1-4 in Example 2;

[0110] 5) Obtain the first occurrence position information of each associated data in the left table from the position information of the second associated data in the preset data structure (third sub-data structure and fourth sub-data structure). Generate a target join table based on the data in the left table corresponding to each first occurrence position information (only including the data in the left table, no need to supplement the corresponding data in the right table). The resulting target join table is shown below:

[0111] Target join table left table ID1 surface

[0112] If a row of data in the left table is equivalent to two or more rows of data in the right table and satisfies the preset non-equivalence filtering condition, then there are two corresponding position information of this row of data in the left table in the preset data structure. In step 5, only the first occurrence of the two position information is obtained to determine the corresponding data in the left table and write it into the target join table.

[0113] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0114] Furthermore, the position information of each row in the left table recorded in the position information column of step 2 is the position information of each row in its data block; in subsequent steps, equi-joins and non-equi-joins can be performed at the data block level. Specifically, after completing equi-joins and non-equi-joins with the right table for each data block of the left table, a set of second associated data can be determined. At the same time, the data corresponding to the first occurrence of position information in the data block can be determined from the preset data structure through the traversal process and written into the target join table, without having to wait until all data in the left table has completed equi-joins and non-equi-joins before writing it into the target join table.

[0115] Example 5

[0116] In this example, the target join type is a right outer join. We only need to use a pre-defined data structure to identify the position of the second related data in the hash table corresponding to the right table. Assuming the left table key is ID, and the ID column data is [1,1,2,3], the right table key(ID)+value is [(1,10),(1,20),(3,10),(5,10)], and the pre-defined non-equality filtering condition is that the value is greater than 10. The specific processing method for the data table is as follows:

[0117] 1) Create a hash table for the right table, and construct the first and second sub-data structures. Construct the data structure corresponding to the hash table, and initialize the state information of each element in the data structure corresponding to the hash table to "False" to indicate "unused"; the details are as follows:

[0118] The first sub-data structure records the offset position information of each second associated data in the hash table, and the second sub-data structure records the element position information of each second associated data in the corresponding offset position.

[0119]

[0120] 2) Use a hash join to perform an equi-join on the left and right tables, obtaining the first associated data between them. During the equi-join process, a first sub-data structure can be used to record the position information of the first associated data in the hash table corresponding to the right table; the specific correspondence is shown in the intermediate table below:

[0121]

[0122] In the equi-join, the data in the 0th row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; the data in the 1st row of the left table with ID 1 can be associated with (1,10) and (1,20) of the right table; and the data in the 3rd row of the left table with ID 3 can be associated with (3,10) of the right table.

[0123] 3) Filter the first associated data according to the preset non-equivalence filtering condition (value greater than 10), determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition, and use the fourth sub-data structure to record the judgment result of whether the first associated data meets the preset non-equivalence filtering condition (where 0 indicates not met, 1 indicates met), the specific correspondence is shown in the intermediate table below:

[0124]

[0125] Among them, only the first associated data in the left table with ID 1 in row 0 and (1,20) in the right table, and the first associated data in the left table with ID 1 in row 1 and (1,20) in the right table satisfy the preset non-equal value filtering condition (value greater than 10), and are used as the second associated data.

[0126] By combining the fourth, first, and second sub-data structures, the offset position of the second associated data in the hash table and the corresponding element position (101,1) at that offset position can be determined. Then, based on this offset position and element position information, the usage status information of the corresponding element in the hash table is modified from "False" to "True" in the corresponding data structure, indicating that it is "used," as shown below:

[0127]

[0128] 4) Based on the data structure corresponding to the hash table, determine the data in the right table excluding the second associated data, and identify the data to be supplemented in the right table based on the data excluding the second associated data; based on the second associated data and the data to be supplemented in the right table, generate the target join table, where missing data in the data to be supplemented is filled with "null", and the resulting target join table is shown below:

[0129]

[0130] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0131] Furthermore, the data to be supplemented in the right table can only be determined after all data blocks in the left table have completed equi-joins and non-equi-join filtering. Therefore, only the data structure corresponding to the hash table obtained after completing equi-joins and filtering of all data blocks in the left table can reflect which data in the right table has not been associated with the left table. Thus, by traversing the data structure corresponding to the hash table obtained at the end, the data to be supplemented in the right table is determined and written into the target join table.

[0132] Example 6

[0133] In this example, the target join type is a right anti-join. We only need to use a preset data structure to identify the position of the second related data in the hash table corresponding to the right table. Assuming the left table key is ID, and the ID column data is [1,1,2,3], the right table key(ID)+value is [(1,10),(1,20),(3,10),(5,10)], and the preset non-equality filtering condition is value greater than 10; the specific data table processing method is as follows:

[0134] Steps 1-3 are the same as steps 1-3 in Example 4;

[0135] 4) Based on the data structure corresponding to the hash table, determine the data in the right table excluding the second associated data. Generate the target join table based on the data in the right table excluding the second associated data (only including the data in the right table, no need to add null values). The resulting target join table is shown below:

[0136]

[0137] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0138] Furthermore, since it is necessary to wait until all data blocks in the left table have completed equi-joins and non-equi-join filtering before it can be determined which data in the right table is not associated with the left table, only the data structure corresponding to the hash table obtained after completing equi-joins and filtering of all data blocks in the left table can reflect which data in the right table is not associated with the left table. Therefore, by traversing the data structure corresponding to the hash table obtained at the end, it is determined that the data in the right table other than the second associated data is written to the target join table.

[0139] Example 7

[0140] In this example, the target join type is a right semi join. We only need to use a preset data structure to identify the position information of the second related data in the hash table corresponding to the right table. Assuming the left table key is ID, and the ID column data is [1,1,2,3], the right table key(ID)+value is [(1,10),(1,20),(3,10),(5,10)], and the preset non-equality filtering condition is value greater than 10; the specific data table processing method is as follows:

[0141] 1-2) Steps 1-2 are the same as steps 1-2 in Example 4;

[0142] The first associated data is filtered according to a preset non-equivalence filtering condition (value greater than 10). Second associated data that meets the preset non-equivalence filtering condition is then identified. A fourth sub-data structure records the judgment result of whether the first associated data meets the preset non-equivalence filtering condition (where 0 indicates not met, and 1 indicates met). The specific correspondence is shown in the intermediate table below:

[0143]

[0144] Among them, only the first associated data in the left table with ID 1 in row 0 and (1,20) in the right table, and the first associated data in the left table with ID 1 in row 1 and (1,20) in the right table satisfy the preset non-equal value filtering condition (value greater than 10), and are used as the second associated data.

[0145] By combining the fourth, first, and second sub-data structures, the offset position of the second associated data in the hash table and the corresponding element position (101,1) at that offset position can be determined. Then, based on this offset position and element position information, the usage status information of the corresponding element in the hash table is modified from "False" to "True" in the corresponding data structure, indicating that it is "used," as shown below:

[0146]

[0147] However, in this example, we only focus on the elements in the data structure corresponding to the hash table whose usage status information is modified for the first time. For example, suppose a row of data in the right table is associated with two or more rows of data in the left table and meets the preset non-equality filtering condition. When this row of data in the right table is associated with a row of data in the left table for the first time and meets the preset non-equality filtering condition, we determine the position information of this row of data in the hash table, that is, the offset position of this row of data in the hash table and the position information of the element at that offset position. We then change the usage status information of the corresponding element in the data structure corresponding to the hash table from "False" to "True" for the first time. If this row of data in the right table is associated with another row of data in the left table again and meets the preset non-equality filtering condition, we determine the position information of this row of data in the hash table. When we modify the usage status information of the corresponding element in the data structure corresponding to the hash table, we find that its usage status information is already "True". Therefore, we do not need to modify the usage status information again, and we can also stop retaining this second association data. Optionally, we can also delete the corresponding element from the hash table to avoid subsequent duplicate connection matching.

[0148] 4) Determine the element in the data structure corresponding to the hash table that has had its usage status information modified for the first time. Generate the target join table based on the data in the right table corresponding to the element with the first modified usage status information. (Only include the data in the right table; no need to supplement the corresponding data in the left table). The resulting target join table is shown below:

[0149]

[0150] It should be noted that the left table can be divided into at least two data blocks by row. For example, the default number of rows in each data block is 4096. For columnar storage engines, data is stored column-wise within each data block.

[0151] Furthermore, since any data in the right table can be written into the target join table as long as it can be connected with any data in the left table, without needing to consider whether the data is subsequently connected with other data in the left table, after performing equi-joins and non-equi-joins filtering on each data block in the left table and the right table, the element whose first modification and usage status information is determined based on the data structure corresponding to the hash table at this time, and then the corresponding data in the right table is written into the target join table, without having to wait until all data in the left table has completed equi-joins and non-equi-joins filtering before writing it into the target join table.

[0152] In summary, Examples 1 through 7 show that different target join types have different requirements for the preset data structure and the corresponding data structure of the hash table, as summarized in the table below:

[0153]

[0154] Corresponding to the data table processing method in the above embodiments, Figure 3 is a structural block diagram of the data table processing device provided in this disclosure embodiment. For ease of explanation, only the parts related to this disclosure embodiment are shown. Referring to Figure 3, the data table processing device 300 includes: an equivalence connection unit 301, a non-equivalence filtering unit 302, an identification unit 303, and a generation unit 304.

[0155] The equi-join unit 301 is used to perform an equi-join on the left and right tables of the two data tables to be joined using a hash join method, and obtain the first associated data between the left and right tables.

[0156] The non-equivalence filtering unit 302 is used to filter the first associated data according to a preset non-equivalence filtering condition, and determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition;

[0157] Identification unit 303 is used to identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in a preset data structure according to the target connection type of the two data tables;

[0158] The generation unit 304 is used to process the left table and / or the right table according to the target connection type of the two data tables and the preset data structure to generate a target connection table.

[0159] In one or more embodiments of this disclosure, the identification unit 303 is further configured to:

[0160] Construct a data structure corresponding to the hash table to record the usage status information of each element included in each offset position of the hash table, wherein each offset position of the hash table includes at least one element, and each element corresponds to a row of data in the right table;

[0161] After identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, it also includes:

[0162] Based on the position information of the second associated data in the hash table corresponding to the right table, the usage status information of the element corresponding to the second associated data is modified in the data structure corresponding to the hash table.

[0163] In one or more embodiments of this disclosure, when the generation unit 304 processes the left table and / or the right table according to the target join type of the two data tables and the preset data structure to generate a target join table, it is configured to:

[0164] If the target connection type is a full connection, then according to the position information of the second associated data in the left table in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table.

[0165] Based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table;

[0166] Based on the second associated data, the data to be supplemented in the left table, and the data to be supplemented in the right table, a target join table is generated.

[0167] In one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, when the generation unit 304 determines the data in the left table other than the second associated data as the data to be supplemented in the left table according to the position information of the second associated data in the preset data structure in the left table, it is used to:

[0168] For the preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table, based on the position information of the second associated data in the preset data structure in the left table, the data in the data block other than the second associated data is determined as the data to be supplemented in the left table.

[0169] When the generation unit 304 determines the data to be supplemented in the right table according to the data structure corresponding to the hash table, it is used to:

[0170] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table.

[0171] In one or more embodiments of this disclosure, when the generation unit 304 processes the left table and / or the right table according to the target join type of the two data tables and the preset data structure to generate a target join table, it is configured to:

[0172] If the target join type is a left outer join, then based on the position information of the second associated data in the left table corresponding to the second associated data in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table; based on the second associated data and the data to be supplemented in the left table, a target join table is generated; or

[0173] If the target join type is a left antijoin, then based on the position information of the second associated data in the preset data structure corresponding to the second associated data in the left table, the data in the left table other than the second associated data is determined, and a target join table is generated based on the data in the left table other than the second associated data; or

[0174] If the target connection type is a left half connection, then the first occurrence position information of each second associated data in the preset data structure is obtained from the position information corresponding to the first occurrence position information in the left table, and a target connection table is generated according to the data in the left table corresponding to the first occurrence position information.

[0175] In one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a left outer join, a left antijoin, or a left half join, the preset data structure is a preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table.

[0176] In one or more embodiments of this disclosure, when the generation unit 304 processes the left table and / or the right table according to the target join type of the two data tables and the preset data structure to generate a target join table, it is configured to:

[0177] If the target join type is a right outer join, then based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table; based on the second associated data and the data to be supplemented in the right table, generate the target join table; or

[0178] If the target join type is a right reverse join, then based on the data structure corresponding to the hash table, determine the data in the right table excluding the second associated data, and generate the target join table based on the data in the right table excluding the second associated data; or

[0179] If the target join type is a right join, then determine the element in the data structure corresponding to the hash table that first modifies the usage status information, and generate the target join table based on the data in the right table corresponding to the element that first modifies the usage status information.

[0180] In one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a right outer join, when the generation unit 304 determines the data to be supplemented in the right table according to the data structure corresponding to the hash table, it is used to:

[0181] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table; or

[0182] If the target join type is a right antijoin, when the generation unit 304 determines the data in the right table other than the second associated data according to the data structure corresponding to the hash table, it is used to:

[0183] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data in the right table other than the second associated data is determined according to the data structure corresponding to the hash table; or

[0184] If the target join type is a right half join, when the generation unit 304 determines the element in the data structure corresponding to the hash table that first modifies the usage state information, it is used to:

[0185] For the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on any data block of the left table, determine the element in the data structure corresponding to the hash table that has its usage status information modified for the first time.

[0186] In one or more embodiments of this disclosure, the preset data structure includes a first sub-data structure and a second sub-data structure; correspondingly, when the identification unit 303 identifies the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, it is used for:

[0187] The first sub-data structure is used to record the offset position information of each of the second associated data in the hash table;

[0188] The second sub-data structure is used to record the element position information of each of the second associated data at the corresponding offset position.

[0189] In one or more embodiments of this disclosure, the preset data structure includes a third sub-data structure and a fourth sub-data structure; correspondingly, when the identification unit 303 identifies the position information of the second associated data in the left table within the preset data structure, it is used for:

[0190] The third sub-data structure is used to record the position information of each of the first associated data in the left table;

[0191] The fourth sub-data structure is used to record the judgment results of whether each of the first associated data satisfies the preset non-equal value filtering condition;

[0192] By combining the third sub-data structure and the fourth sub-data structure, the position information of the second associated data in the left table is obtained.

[0193] In one or more embodiments of this disclosure, the two data tables are data tables in a columnar storage engine; correspondingly, before performing an equi-join on the left and right tables of the two data tables to be joined using a hash join, the generation unit 304 is further configured to:

[0194] In the columnar storage engine, a row position information column is added to the left table to record the position information of each row in the left table;

[0195] When the equi-join unit 301 performs an equi-join on the left and right tables of two data tables to be joined using a hash join method to obtain the first associated data between the left and right tables, it is used to:

[0196] The left and right tables, after the row position information column is added, are equi-joined using a hash join method to obtain the first associated data.

[0197] In one or more embodiments of this disclosure, the position information corresponding to any data in the left table is the position information of the data in the data block it is located in;

[0198] When the equi-join unit 301 performs an equi-join on the left and right tables of two data tables to be joined using a hash join method, it is used for:

[0199] A hash join is used to perform equi-joins between each data block of the left table and the right table.

[0200] The data table processing device provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0201] Referring to Figure 4, a schematic diagram of the structure of an electronic device 400 suitable for implementing embodiments of the present disclosure is shown. The electronic device 400 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 4 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0202] As shown in Figure 4, the electronic device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0203] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 shows electronic device 400 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0204] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0205] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0206] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0207] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0208] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0209] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0210] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0211] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0212] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0213] In a first aspect, according to one or more embodiments of this disclosure, a method for processing a data table is provided, comprising:

[0214] The left and right tables of the two data tables to be joined are equi-joined using a hash join method to obtain the first associated data between the left and right tables;

[0215] The first associated data is filtered according to a preset non-equivalence filtering condition to determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition;

[0216] Based on the target connection type of the two data tables, identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in the preset data structure;

[0217] Based on the target join type of the two data tables and the preset data structure, the left table and / or the right table are processed to generate the target join table.

[0218] According to one or more embodiments of this disclosure, the method further includes:

[0219] Construct a data structure corresponding to the hash table to record the usage status information of each element included in each offset position of the hash table, wherein each offset position of the hash table includes at least one element, and each element corresponds to a row of data in the right table;

[0220] After identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, it also includes:

[0221] Based on the position information of the second associated data in the hash table corresponding to the right table, the usage status information of the element corresponding to the second associated data is modified in the data structure corresponding to the hash table.

[0222] According to one or more embodiments of this disclosure, the step of processing the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure includes:

[0223] If the target connection type is a full connection, then according to the position information of the second associated data in the left table in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table.

[0224] Based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table;

[0225] Based on the second associated data, the data to be supplemented in the left table, and the data to be supplemented in the right table, a target join table is generated.

[0226] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, determining the data in the left table other than the second associated data as the data to be supplemented in the left table based on the position information of the second associated data in the preset data structure in the left table includes:

[0227] For the preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table, based on the position information of the second associated data in the preset data structure in the left table, the data in the data block other than the second associated data is determined as the data to be supplemented in the left table.

[0228] The step of determining the data to be supplemented in the right table based on the data structure corresponding to the hash table includes:

[0229] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table.

[0230] According to one or more embodiments of this disclosure, the step of processing the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure includes:

[0231] If the target join type is a left outer join, then based on the position information of the second associated data in the left table corresponding to the second associated data in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table; based on the second associated data and the data to be supplemented in the left table, a target join table is generated; or

[0232] If the target join type is a left antijoin, then based on the position information of the second associated data in the preset data structure corresponding to the second associated data in the left table, the data in the left table other than the second associated data is determined, and a target join table is generated based on the data in the left table other than the second associated data; or

[0233] If the target connection type is a left half connection, then the first occurrence position information of each second associated data in the preset data structure is obtained from the position information corresponding to the first occurrence position information in the left table, and a target connection table is generated according to the data in the left table corresponding to the first occurrence position information.

[0234] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a left outer join, a left antijoin, or a left half join, the preset data structure is a preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table.

[0235] According to one or more embodiments of this disclosure, the step of processing the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure includes:

[0236] If the target join type is a right outer join, then based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table; based on the second associated data and the data to be supplemented in the right table, generate the target join table; or

[0237] If the target join type is a right reverse join, then based on the data structure corresponding to the hash table, determine the data in the right table excluding the second associated data, and generate the target join table based on the data in the right table excluding the second associated data; or

[0238] If the target join type is a right join, then determine the element in the data structure corresponding to the hash table that first modifies the usage status information, and generate the target join table based on the data in the right table corresponding to the element that first modifies the usage status information.

[0239] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a right outer join, determining the data to be supplemented in the right table according to the data structure corresponding to the hash table includes:

[0240] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table; or

[0241] If the target join type is a right antijoin, determining the data in the right table excluding the second associated data according to the data structure corresponding to the hash table includes:

[0242] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data in the right table other than the second associated data is determined according to the data structure corresponding to the hash table; or

[0243] If the target join type is a right half join, determining the element in the data structure corresponding to the hash table that has its usage status information modified for the first time includes:

[0244] For the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on any data block of the left table, determine the element in the data structure corresponding to the hash table that has its usage status information modified for the first time.

[0245] According to one or more embodiments of this disclosure, the preset data structure includes a first sub-data structure and a second sub-data structure; correspondingly, identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure includes:

[0246] The first sub-data structure is used to record the offset position information of each of the second associated data in the hash table;

[0247] The second sub-data structure is used to record the element position information of each of the second associated data at the corresponding offset position.

[0248] According to one or more embodiments of this disclosure, the preset data structure includes a third sub-data structure and a fourth sub-data structure; correspondingly, identifying the position information of the second associated data in the left table within the preset data structure includes:

[0249] The third sub-data structure is used to record the position information of each of the first associated data in the left table;

[0250] The fourth sub-data structure is used to record the judgment results of whether each of the first associated data satisfies the preset non-equal value filtering condition;

[0251] By combining the third sub-data structure and the fourth sub-data structure, the position information of the second associated data in the left table is obtained.

[0252] According to one or more embodiments of this disclosure, the two data tables are data tables in a columnar storage engine; correspondingly, before performing an equi-join on the left and right tables of the two data tables to be joined using a hash join, the method further includes:

[0253] In the columnar storage engine, a row position information column is added to the left table to record the position information of each row in the left table;

[0254] The method of performing an equi-join on the left and right tables of the two data tables to be joined using a hash join, and obtaining the first associated data between the left and right tables, includes:

[0255] The left and right tables, after the row position information column is added, are equi-joined using a hash join method to obtain the first associated data.

[0256] According to one or more embodiments of this disclosure, the location information corresponding to any data in the left table is the location information of the data in the data block in which it is located;

[0257] The method of performing an equi-join on the left and right tables of the two data tables to be joined using hash join includes:

[0258] A hash join is used to perform equi-joins between each data block of the left table and the right table.

[0259] Secondly, according to one or more embodiments of this disclosure, a data table processing apparatus is provided, comprising:

[0260] An equi-join unit is used to perform an equi-join on the left and right tables of two data tables to be joined using a hash join method, and obtain the first associated data between the left and right tables;

[0261] A non-equivalence filtering unit is used to filter the first associated data according to a preset non-equivalence filtering condition, and determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition;

[0262] An identification unit is used to identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in a preset data structure according to the target connection type of the two data tables.

[0263] The generation unit is used to process the left table and / or the right table according to the target connection type of the two data tables and the preset data structure to generate a target connection table.

[0264] According to one or more embodiments of this disclosure, the identification unit is further configured to:

[0265] Construct a data structure corresponding to the hash table to record the usage status information of each element included in each offset position of the hash table, wherein each offset position of the hash table includes at least one element, and each element corresponds to a row of data in the right table;

[0266] After identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, it also includes:

[0267] Based on the position information of the second associated data in the hash table corresponding to the right table, the usage status information of the element corresponding to the second associated data is modified in the data structure corresponding to the hash table.

[0268] According to one or more embodiments of this disclosure, when the generation unit processes the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure, it is configured to:

[0269] If the target connection type is a full connection, then according to the position information of the second associated data in the left table in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table.

[0270] Based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table;

[0271] Based on the second associated data, the data to be supplemented in the left table, and the data to be supplemented in the right table, a target join table is generated.

[0272] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, when the generating unit determines the data in the left table other than the second associated data as the data to be supplemented in the left table based on the position information of the second associated data in the preset data structure corresponding to the data in the left table, it is used to:

[0273] For the preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table, based on the position information of the second associated data in the preset data structure in the left table, the data in the data block other than the second associated data is determined as the data to be supplemented in the left table.

[0274] When the generation unit determines the data to be supplemented in the right table according to the data structure corresponding to the hash table, it is used to:

[0275] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table.

[0276] According to one or more embodiments of this disclosure, when the generation unit processes the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure, it is configured to:

[0277] If the target join type is a left outer join, then based on the position information of the second associated data in the left table corresponding to the second associated data in the preset data structure, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table; based on the second associated data and the data to be supplemented in the left table, a target join table is generated; or

[0278] If the target join type is a left antijoin, then based on the position information of the second associated data in the preset data structure corresponding to the second associated data in the left table, the data in the left table other than the second associated data is determined, and a target join table is generated based on the data in the left table other than the second associated data; or

[0279] If the target connection type is a left half connection, then the first occurrence position information of each second associated data in the preset data structure is obtained from the position information corresponding to the first occurrence position information in the left table, and a target connection table is generated according to the data in the left table corresponding to the first occurrence position information.

[0280] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a left outer join, a left antijoin, or a left half join, the preset data structure is a preset data structure obtained after performing equi-join and filtering with preset non-equi-join conditions on any data block of the left table.

[0281] According to one or more embodiments of this disclosure, when the generation unit processes the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure, it is configured to:

[0282] If the target join type is a right outer join, then based on the data structure corresponding to the hash table, determine the data to be supplemented in the right table; based on the second associated data and the data to be supplemented in the right table, generate the target join table; or

[0283] If the target join type is a right reverse join, then based on the data structure corresponding to the hash table, determine the data in the right table excluding the second associated data, and generate the target join table based on the data in the right table excluding the second associated data; or

[0284] If the target join type is a right join, then determine the element in the data structure corresponding to the hash table that first modifies the usage status information, and generate the target join table based on the data in the right table corresponding to the element that first modifies the usage status information.

[0285] According to one or more embodiments of this disclosure, the left table includes at least two data blocks; correspondingly, if the target join type is a right outer join, when the generating unit determines the data to be supplemented in the right table according to the data structure corresponding to the hash table, it is used to:

[0286] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data to be supplemented in the right table is determined according to the data structure corresponding to the hash table; or

[0287] If the target join type is a right antijoin, when the generation unit determines the data in the right table other than the second associated data according to the data structure corresponding to the hash table, it is used to:

[0288] Based on the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks in the left table, the data in the right table other than the second associated data is determined according to the data structure corresponding to the hash table; or

[0289] If the target join type is a right half join, when the generation unit determines the element in the data structure corresponding to the hash table that first modifies the usage state information, it is used to:

[0290] For the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on any data block of the left table, determine the element in the data structure corresponding to the hash table that has its usage status information modified for the first time.

[0291] According to one or more embodiments of this disclosure, the preset data structure includes a first sub-data structure and a second sub-data structure; correspondingly, when the identification unit identifies the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, it is used for:

[0292] The first sub-data structure is used to record the offset position information of each of the second associated data in the hash table;

[0293] The second sub-data structure is used to record the element position information of each of the second associated data at the corresponding offset position.

[0294] According to one or more embodiments of this disclosure, the preset data structure includes a third sub-data structure and a fourth sub-data structure; correspondingly, when the identification unit identifies the position information of the second associated data in the left table within the preset data structure, it is used for:

[0295] The third sub-data structure is used to record the position information of each of the first associated data in the left table;

[0296] The fourth sub-data structure is used to record the judgment results of whether each of the first associated data satisfies the preset non-equal value filtering condition;

[0297] By combining the third sub-data structure and the fourth sub-data structure, the position information of the second associated data in the left table is obtained.

[0298] According to one or more embodiments of this disclosure, the two data tables are data tables in a columnar storage engine; correspondingly, before performing an equi-join on the left and right tables of the two data tables to be joined using a hash join method, the generation unit is further configured to:

[0299] In the columnar storage engine, a row position information column is added to the left table to record the position information of each row in the left table;

[0300] When the equi-join unit performs an equi-join on the left and right tables of two data tables to be joined using a hash join method, and obtains the first associated data between the left and right tables, it is used to:

[0301] The left and right tables, after the row position information column is added, are equi-joined using a hash join method to obtain the first associated data.

[0302] According to one or more embodiments of this disclosure, the location information corresponding to any data in the left table is the location information of the data in the data block in which it is located;

[0303] The equi-join unit, when performing an equi-join on the left and right tables of two data tables to be joined using a hash join method, is used for:

[0304] A hash join is used to perform equi-joins between each data block of the left table and the right table.

[0305] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0306] The memory stores computer-executed instructions;

[0307] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the data table processing method described in the first aspect and various possible designs of the first aspect.

[0308] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the data table processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0309] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including computer execution instructions that, when executed by a processor, implement the data table processing method described in the first aspect and various possible designs of the first aspect.

[0310] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0311] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0312] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for processing a data table, characterized in that, include: The left and right tables of the two data tables to be joined are equi-joined using a hash join method to obtain the first associated data between the left and right tables; The first associated data is filtered according to a preset non-equivalence filtering condition to determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition; Based on the target connection type of the two data tables, identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in the preset data structure; Based on the target join type of the two data tables and the position information of the second associated data in the preset data structure, the left table and / or the right table are processed based on at least one of the second associated data, a portion of the data in the left table, a portion of the data in the right table, the data to be supplemented in the left table, and the data to be supplemented in the right table to generate a target join table. Specifically, based on the position information of the second associated data in the preset data structure corresponding to the data in the left table, the data in the left table other than the second associated data is determined as the data to be supplemented in the left table; and based on the data structure corresponding to the hash table, the data in the right table other than the second associated data is determined as the data to be supplemented in the right table.

2. The method according to claim 1, characterized in that, The method further includes: constructing a data structure corresponding to the hash table, used to record the usage status information of each element included in each offset position of the hash table, wherein each offset position of the hash table includes at least one element, and one element corresponds to a row of data in the right table; after identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure, the method further includes: modifying the usage status information of the element corresponding to the second associated data in the data structure corresponding to the hash table according to the position information of the second associated data in the hash table.

3. The method according to claim 2, characterized in that, The step of processing the left table and / or the right table according to the target join type of the two data tables and the preset data structure to generate a target join table includes: if the target join type is a full join, determining the data to be supplemented in the left table; determining the data to be supplemented in the right table; and generating a target join table based on the second associated data, the data to be supplemented in the left table, and the data to be supplemented in the right table.

4. The method according to claim 3, characterized in that, The left table includes at least two data blocks; determining the data to be supplemented in the left table includes: for a preset data structure obtained after performing equi-joins and filtering with preset non-equi-join conditions on any data block of the left table, determining the data in the data block other than the second associated data as the data to be supplemented in the left table according to the position information of the second associated data in the preset data structure in the left table; determining the data to be supplemented in the right table includes: for the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks of the left table, determining the data to be supplemented in the right table according to the data structure corresponding to the hash table.

5. The method according to claim 1, characterized in that, The step of processing the left table and / or the right table according to the target join type of the two data tables and the preset data structure to generate a target join table includes: if the target join type is a left outer join, determining the data to be supplemented in the left table; generating a target join table based on the second associated data and the data to be supplemented in the left table; or if the target join type is a left antijoin, determining the data in the left table other than the second associated data according to the position information of the second associated data in the preset data structure corresponding to the data in the left table, and generating a target join table based on the data in the left table other than the second associated data; or if the target join type is a left half join, obtaining the position information of each first occurrence from the position information of the second associated data in the preset data structure corresponding to the position information of each first occurrence in the left table, and generating a target join table based on the data in the left table corresponding to the position information of each first occurrence.

6. The method according to claim 5, characterized in that, The left table includes at least two data blocks; if the target join type is a left outer join, a left antijoin, or a left half join, the preset data structure is a preset data structure obtained after performing an equi-join and filtering with preset non-equi-join conditions on any data block of the left table.

7. The method according to claim 2, characterized in that, The step of processing the left table and / or the right table to generate a target join table based on the target join type of the two data tables and the preset data structure includes: if the target join type is a right outer join, then determining the data to be supplemented in the right table; generating a target join table based on the second associated data and the data to be supplemented in the right table; or if the target join type is a right reverse join, then determining the data in the right table other than the second associated data according to the data structure corresponding to the hash table, and generating a target join table based on the data in the right table other than the second associated data; or if the target join type is a right half join, then determining the element in the data structure corresponding to the hash table that has its usage status information modified for the first time, and generating a target join table based on the data in the right table corresponding to the element that has its usage status information modified for the first time.

8. The method according to claim 7, characterized in that, The left table includes at least two data blocks; if the target join type is a right outer join, determining the data to be supplemented in the right table includes: for the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks of the left table, determining the data to be supplemented in the right table according to the data structure corresponding to the hash table; or if the target join type is a right reverse join, determining the data in the right table other than the second associated data according to the data structure corresponding to the hash table includes: for the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on all data blocks of the left table, determining the data in the right table other than the second associated data according to the data structure corresponding to the hash table; or if the target join type is a right half join, determining the element in the data structure corresponding to the hash table that first modifies the usage status information includes: for the data structure corresponding to the hash table obtained after performing equi-joins and filtering with preset non-equi-join conditions on any data block of the left table, determining the element in the data structure corresponding to the hash table that first modifies the usage status information.

9. The method according to any one of claims 1-5, characterized in that, The preset data structure includes a first sub-data structure and a second sub-data structure; the step of identifying the position information of the second associated data in the hash table corresponding to the right table in the preset data structure includes: using the first sub-data structure to record the offset position information of each second associated data in the hash table; and using the second sub-data structure to record the element position information of each second associated data in the corresponding offset position.

10. The method according to any one of claims 1-5, characterized in that, The preset data structure includes a third sub-data structure and a fourth sub-data structure; identifying the position information of the second associated data in the left table in the preset data structure includes: using the third sub-data structure to record the position information of each first associated data in the left table; using the fourth sub-data structure to record the judgment result of whether each first associated data satisfies the preset non-equal value filtering condition; and combining the third sub-data structure and the fourth sub-data structure to obtain the position information of the second associated data in the left table.

11. The method according to claim 10, characterized in that, The two data tables are data tables in a columnar storage engine; before performing an equi-join on the left and right tables of the two data tables to be joined using a hash join, the method further includes: adding a row position information column to the left table in the columnar storage engine to record the position information of each row in the left table; performing an equi-join on the left and right tables of the two data tables to be joined using a hash join to obtain the first associated data between the left and right tables includes: performing an equi-join on the left table and the right table after adding the row position information column using a hash join to obtain the first associated data.

12. The method according to claim 4, 6 or 8, characterized in that, The location information corresponding to any data in the left table is the location information of that data in the data block it belongs to; The method of using hash join to perform equi-join on the left and right tables of the two data tables to be joined includes: performing equi-join on each data block of the left table and the right table respectively using hash join.

13. A data table processing device, characterized in that, include: An equi-join unit is used to perform an equi-join on the left and right tables of two data tables to be joined using a hash join method, and obtain the first associated data between the left and right tables; A non-equivalence filtering unit is used to filter the first associated data according to a preset non-equivalence filtering condition, and determine the second associated data in the first associated data that meets the preset non-equivalence filtering condition; An identification unit is used to identify the position information of the second associated data in the left table and / or the position information of the second associated data in the hash table corresponding to the right table in a preset data structure according to the target connection type of the two data tables. A generation unit is configured to process the left table and / or the right table based on at least one of the second associated data, a portion of the data in the left table, a portion of the data in the right table, the data to be supplemented in the left table, and the data to be supplemented in the right table, according to the target connection type of the two data tables and the position information of the second associated data in the preset data structure, to generate a target connection table. Specifically, based on the position information of the second associated data in the left table corresponding to the second associated data in the preset data structure, data in the left table other than the second associated data is determined as the data to be supplemented in the left table; and based on the data structure corresponding to the hash table, data in the right table other than the second associated data is determined as the data to be supplemented in the right table.

14. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-12.

16. A computer program product, characterized in that, Includes computer execution instructions, which, when executed by a processor, implement the method as described in any one of claims 1-12.

Images