A data table association relationship classification method and device based on graph search technology

Abstract

The present invention discloses a data table association relationship classification method and device based on graph search technology. The method relies on the statistical classification results of data table fields, constructs an association relationship set of association data tables based on a directed graph, and displays the association relationships between association data tables in a visual graph. The method uses graph search technology to construct an association relationship set of association data tables, and based on the set main table, constructs an association relationship classification for association data table aggregation. The method supports the inspection of the full connectivity of the data table and realizes the automatic generation of association relationships of association data tables. At the same time, the method can automatically locate the main table position and determine the connection fields of the association data table aggregation. The main innovation of the method is to use a directed graph to represent the association relationships between association data tables, and to use a graph search algorithm to find the association relationships between association data tables.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data table association relationship classification method and device based on graph search technology. Background Art

[0002] Associative tables are currently a mainstream data storage format. Using machine learning methods and other methods to study the underlying patterns in data stored in associative tables requires displaying the relationships between them and their associated fields. Based on these relationships and associated fields, associative tables can be automatically aggregated. Currently, the main methods for obtaining relationships in associative tables include scenario-based experience guidance and manual classification based on statistical results from individual table fields. The first method involves manual participation and business logic in determining the relationships between related tables. This business logic is often fixed and based on a specific business scenario. Different business logics require repeated manual effort to identify related table relationships. This scenario-based approach to identifying related table relationships is not scalable, making it difficult to apply data processing experience to other business scenarios and requiring a high level of professional background from data processing personnel. The second method for processing related table relationships starts with statistical results of related table fields and constructs relationships between tables by considering statistical classification fields. This process requires field statistical analysis of each table in the related table and manual attempts to construct possible association paths between individual tables. Finally, through communication with business personnel, the final used related table relationships are determined. This process attempts to minimize reliance on business experience and achieve scalability in determining related table relationships. However, the drawbacks of this method are the high manual workload, the limited number of feasible relationship types, the inability to fully display all feasible relationship categories, and the difficulty in meeting the requirements of localized related table relationship determination. Summary of the Invention

[0003] The purpose of the present invention is to address the deficiencies of the prior art and to provide a method and device for classifying data table association relationships based on graph search technology.

[0004] The object of the present invention is achieved through the following technical solution: a data table association relationship classification method based on graph search technology, comprising the following steps:

[0005] Step 1: Split the field classification summary table of the association data table into a set of single association data tables T; and construct a set of two-item association relationships R between the single association data tables based on the relationship between the primary key and other fields of each single association data table in the set of single association data tables T;

[0006] Step 2: Perform full connectivity check on the 2-item association relationship set R: If the 2-item association relationship set R has full connectivity, then perform aggregation operation on the single association data tables belonging to the same sibling relationship in the single association data table set T to generate a new single association data table set T′, and update the 2-item association relationship set R to obtain the updated 2-item association relationship set R * , generate the corresponding directed graph representation; if the 2-item association relationship set R does not have full connectivity, add more information;

[0007] Step 3: Search and obtain the main table of the single associated data table set T′;

[0008] Step 4: Classify the data table list of the main table of the single associated data table set T′ and convert the data table list of the main table into a hierarchical structure; based on the hierarchical structure, starting from the last layer, check the diversity of the corresponding parent tables of the single associated data tables at the same layer in sequence. For parent tables occupying different layers, delete the paths that pass through a large number of nodes, update the hierarchical structure, and obtain an updated hierarchical structure; based on the updated hierarchical structure, construct a multi-path cycle set; based on the data table list of the main table, the updated hierarchical structure, and the multi-path cycle set, obtain the multiple relationship classification sets corresponding to the single associated data table set T′;

[0009] Step 5: Select any relationship classification set, aggregate the single-association data table set T′, and output the final aggregated data table of the single-association data table set T′.

[0010] Furthermore, the step 1 is specifically as follows:

[0011] (1.1) Split the field classification summary table of the relational data table into N single relational data tables, and obtain a set of single relational data tables T = {T1, T2, ..., T m ,…,T N}, where subscript m is an ordinal number, m = 1, 2, ... m, ... N, T m Represents the mth single associative data table; and vectorizes each single associative data table to obtain a vector representation set Represents the mth single-item associated data table T m The vector representation of Specifically: Among them, T_m represents a single associated data table T m The table name, KEY_m represents a single associated data table T m The primary key of FEATURES_m represents a single associated data table T m Other fields of

[0012] For each vector Vectorize and divide into and

[0013] (1.2) Determine a single associated data table set T = {T1, T2, ..., T m ,…,T N}, construct a 2-item association relationship set Among them, T i and T j Is any single associated data table. If a single associated data table T i and T j The association relationship consists of brother relationships, then it satisfies Indicated as T i -T j ; If a single associated data table T i and T j The association relationship consists of a parent-child relationship, then Indicated as T i →T j .

[0014] Furthermore, the step 2 is specifically as follows:

[0015] (2.1) performing a full connectivity check on the 2-item association relationship set R generated in step (1.2), wherein full connectivity indicates that all single association-type data tables can be connected through a sibling relationship or a parent-child relationship; if the 2-item association relationship set R has full connectivity, proceed to step (2.2); if the 2-item association relationship set R does not have full connectivity, provide more information;

[0016] (2.2) According to the 2-item association relationship set R, the single association data table set T = {T1, T2, ..., T m ,…,T N}, and perform aggregation operation on the single associated data tables belonging to the same sibling relationship to obtain a new single associated data table set T′={T1′,T2′,…,T n ′,…}, where T n ' represents a new single-relational data table obtained by performing an aggregation operation on single-relational data tables of the same sibling relationship;

[0017] And update the 2-item association relationship set R to obtain the updated 2-item association relationship set R * :R * ={T a ′→T b ′}, where T a ′ and Tb ' represents any single associated data table in the set of single associated data tables T'; T a ′→T b ' represents a single associated data table T a ' is a single associated data table T a ''s subtable;

[0018] (2.3) According to the updated 2-item association relationship set R * , mapping the parent-child relationship between each associated data table in the single associated data table set T′ into a graph connection form, and using arrows to point from the child table to the parent table to obtain a directed graph representation; the directed graph representation includes nodes and arrows, the nodes represent the table name of each single associated data table, and the arrow direction is from the child table to the parent table;

[0019] According to the directed graph representation, a corresponding directed graph node information list is obtained, wherein the directed graph node information list includes the parent table and child table information of each single association data table.

[0020] Furthermore, the step three specifically includes the following sub-steps:

[0021] (3.1) Select a single associated data table T from the set of single associated data tables T′ h ' is set as the local master table, and the reverse full-join search algorithm is used to calculate the single associated data table T defined as the local master table h 'Data table list;

[0022] (3.2) For the data table list obtained in step (3.1), check whether there is a single associated data table set T′ that is not included in the single associated data table T h ′; if it exists, select another single associated data table in the associated data table set T′ as the local master table and repeat steps (3.1) and (3.2); if it does not exist, then replace the single associated data table T h ' as the main table.

[0023] Furthermore, the reverse full-join search algorithm is used to calculate a single associated data table T set as the local main table. h The data table list of ′ is as follows:

[0024] (b1) The data table list is initialized to empty, and then a single associated data table T is set as the local main table. h 'Add to the data table list; find the local master table T h ′, update the data table list: in the data table list, add the local main table T h' and each corresponding sub-table form a data table combination; the data table combination is a combination of single associated data tables based on a parent-child relationship, with the single associated data tables connected by directed arrows, with the child table connected to the parent table; the child table of the single data table combination is represented by the last single associated data table in the single data table combination;

[0025] (b2) Based on the data table list obtained in step (b1), check each data table combination in the data table list one by one to see if there is a single associated data table belonging to a sub-table of a data table combination in the data table list. If the check result is "yes", it indicates that the data table is not empty, and the process proceeds to step (b3). If the check result is "no", it indicates that the data table is empty, and the process proceeds to step (b4);

[0026] (b3) Check the sub-tables of the data table combination in the data table list one by one, add the found sub-tables to the end of the current data table combination, connect them with directed arrows in the middle to form a new data table combination, and use the new data table combination to replace the original data table combination to complete the update of the data table list; then input the updated data table list into step (b2);

[0027] (b4) Output a single associated data table T h ’ data table list.

[0028] Furthermore, the step 4 specifically includes the following sub-steps:

[0029] (4.1) Classify the data table list of the main table of a single associated data table set T′ and transform the data table list of the main table into a hierarchical structure RL = [RL1, RL2, RL3, ...RL k ,…], where RL1 represents the structure of the first layer, which contains the main table; RL2 represents the structure of the second layer, which contains the sub-table corresponding to the main table; RL3 represents the structure of the third layer, which contains a single associated data table corresponding to the single associated data table in RL2; k Represents the structure of the kth layer, RL k Including RL k-1 The single associated data table within the corresponding single associated data table; and so on;

[0030] (4.2) Based on the hierarchical structure RL, starting from the last layer, the diversity of the corresponding parent tables of the single association data table at the same layer is checked in sequence. For the parent tables occupying different layers, the paths with a large number of nodes are deleted, and the hierarchical structure RL is updated to obtain the updated hierarchical structure RL′.

[0031] (4.3) Based on the updated hierarchical structure RL′, construct a multi-path cycle set in, Indicates the multipath situation of the kth layer; if Represents a single associated data table T at the kth level ki There are multiple corresponding parent tables, T ki1 Represents a single associated data table T ki The corresponding first parent table, T ki2 Represents a single associated data table T ki The corresponding second parent table; if Indicates that there is no multipath situation at layer k;

[0032] (4.4) According to the data table list of the main table, the updated hierarchical structure RL′ and the multi-path loop set, multiple relationship classification sets corresponding to a single associated data table set T′ are obtained, where any relationship classification set is RM1 represents the parent-child relationship between a single associative data table in the last layer and the previous layer; RM2 represents the parent-child relationship between the second-to-last layer and the previous layer; RM3 represents the parent-child relationship between the third-to-last layer and the previous layer; and so on.

[0033] Furthermore, the step 5 specifically includes the following sub-steps:

[0034] (5.1) Select any relationship classification set, aggregate a single relational data table in the RM1 layer, and then calculate the relationship T based on the RM1 layer. e ′→T f ', a single associated data table T e ' and a single associated data table T f 'Perform aggregation operation to obtain the aggregate data table T' e，f , and aggregate data table T′ e，f Replace a single associated data table T f '; Repeat the above steps, and perform aggregation operations on a single associated data table according to all the association relationships of the RM1 layer to obtain the aggregation results of the RM1 layer;

[0035] (5.2) Based on the aggregation result of the RM1 layer, replace the corresponding single association data table in the RM2 layer with the aggregation result of the RM1 layer; repeat step (5.1) and perform aggregation operations on the single association data table according to all the association relationships in the RM2 layer to obtain the aggregation result of the RM2 layer;

[0036] (5.3) Repeat step (5.2) until the aggregation result of the last layer of the relationship classification set is obtained;

[0037] (5.4) The aggregation result of the last layer is output as the final aggregated data table of a single associated data table set T′.

[0038] The present invention also provides a data table association relationship classification device based on graph search technology, including one or more processors for implementing the above-mentioned data table association relationship classification method based on graph search technology.

[0039] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned data table association relationship classification method based on graph search technology.

[0040] The beneficial effects of the present invention are:

[0041] 1) Solve the problem of full connectivity check of data tables, which is helpful for determining whether data tables can be aggregated into one data table;

[0042] 2) Solve the problem of searching the global master table of the data table, which is beneficial to realize the automatic aggregation of the data table;

[0043] 3) Supporting local master table aggregation operations is beneficial for implementing aggregation of associated data tables based on local master tables. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a data table association relationship classification method based on graph search technology;

[0045] Figure 2 A flowchart for constructing a 2-item association relationship set R;

[0046] Figure 3 It is a father-son relationship diagram;

[0047] Figure 4 It is a flowchart for checking the full connectivity of a 2-item association relationship set R;

[0048] Figure 5 is a directed graph representation of a single associative data table set T′ in Example 2;

[0049] Figure 6 This is a hierarchical structure diagram of a single association data table set T′ in Example 2;

[0050] Figure 7 A schematic diagram of a data table association relationship classification device based on graph search technology. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to illustrate the present invention, rather than to represent all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] Example 1

[0053] like Figure 1 As shown, the present invention provides a data table association relationship classification method based on graph search technology, comprising the following steps:

[0054] Step 1: Split the field classification summary table of the association data table into a set of single association data tables T; and based on the relationship between the primary key and other fields of each single association data table in the set of single association data tables T, construct a 2-item association relationship set R between the single association data tables, such as Figure 2 As shown;

[0055] The step 1 is specifically implemented through the following sub-steps:

[0056] (1.1) Split the field classification summary table of the relational data table into N single relational data tables, and obtain a set of single relational data tables T = {T1, T2, ..., T m ,…,T N}, where subscript m is an ordinal number, m = 1, 2, ... m, ... N, T m Represents the mth single associated data table; each single associated data table is an associated data table.

[0057] The field classification summary table of the association data table is shown in Table 1.

[0058] Table 1: Field classification summary table of relational data table

[0059] Data table name Primary Key Other fields <![CDATA[T_1]]> KEY_1 FEATURES_1 <![CDATA[T_2]]> KEY_2 FEATURES_2 … … … <![CDATA[T_ m ]]> KEY_m FEATURES_m … … … <![CDATA[T_ N ]]> KEY_N FEATURES_N ;

[0060] And vectorize each single associated data table to obtain a vector representation set Represents the mth single-item associated data table T m The vector representation of Specifically: Among them, T_m represents a single associated data table T m The table name, KEY_m represents a single associated data table T m The primary key of FEATURES_m represents a single associated data table Tm other fields; the primary key is a data table field with special characteristics, and the special characteristics are reflected in the uniqueness of the field content of the primary key in the data table; the other fields are fields in the data table that do not contain the primary key.

[0061] The vector representation of all single-relational data tables is as follows:

[0062]

[0063]

[0064] …

[0065]

[0066] …

[0067]

[0068] Represent each single associated data table as a vector Vectorize and divide into and

[0069] (1.2) Determine a single associated data table set {T1, T2, ..., T m ,…,T N}, construct a 2-item association relationship set Among them, T i and T j Represents any single associated data table; if a single associated data table T i and T j The association relationship consists of brother relationships, then it satisfies Represents a single associated data table T i and T j Have the same primary key, T i -T j Represents a single associated data table T i and T j Have a brother relationship; if a single associated data table T i and T j The association relationship consists of a parent-child relationship, then Represents a single associated data table T i The other fields contain a single associated data table T j The primary key of T i →T j Represents a single associated data table T i and T j Have a parent-child relationship, such as Figure 3 shown.

[0070] The relationship between a single associated data table mainly consists of a brother relationship and a parent-child relationship; the brother relationship means that two single associated data tables have the same primary key, and the symbol is T i -T j , satisfying the relationship j By adding a single associated data table T i and a single associated data table T j Direct table merging can realize the aggregation of all single-association data tables with sibling relationships. The direct table merging means aggregating the data tables without any processing of the data fields, and keeping the original content of the data tables; the parent-child relationship means that other fields of a single-association data table contain the primary key of another single-association data table, which is represented by T i →T j , satisfying the relationship That is, a single associated data table T i For a single associated data table T j The subtable is the abbreviation of the subclass data table, and the single associated data table T j The parent table is the abbreviation of the parent class data table. The aggregation process of the parent table and the child table is as follows: first, based on the associated field (parent table primary key) aggregation, and then aggregated with the parent table Ti. The aggregation here can use conventional statistical functions (such as: max, mean, min, count, size, nunique), or custom functions (specific functions are determined based on requirements);

[0071] The 2-item association relationship set R does not depend on the content of the associated data table, but depends entirely on the primary key and other fields of a single associated data table.

[0072] Step 2: Perform full connectivity check on the 2-item association relationship set R: If the 2-item association relationship set R has full connectivity, then perform aggregation operation on the single association data tables belonging to the same sibling relationship in the single association data table set T to generate a new single association data table set T′, and update the 2-item association relationship set R to obtain the updated 2-item association relationship set R * , generate the corresponding directed graph representation; if the 2-item association relationship set R does not have full connectivity, add more information;

[0073] Step 2 is implemented through the following sub-steps: Figure 4 As shown:

[0074] (2.1) performing a full connectivity check on the 2-item association relationship set R generated in step (1.2), wherein full connectivity indicates that all single association-type data tables can be connected through a sibling relationship or a parent-child relationship; if the 2-item association relationship set R has full connectivity, proceed to step (2.2); if the 2-item association relationship set R does not have full connectivity, provide more information;

[0075] For complete business scenarios, a collection of associated data tables naturally maintains full connectivity. However, in actual data operations, data tables are constructed based on data requirements. Without a comprehensive understanding of the original scenario data, the extracted associated data tables may lack full connectivity, resulting in the inability to aggregate the extracted data tables. Therefore, it is necessary to check the full connectivity between a given collection of associated data tables. The main reason for the lack of full connectivity between the obtained associated data tables is the missing data tables. For associated data tables with missing data tables, the corresponding directed graph representations are divided into two types: segmented directed graph representation and multi-sink directed graph representation.

[0076] (2.2) According to the 2-item association relationship set R, the single association data table set T = {T1, T2, ..., T m ,…,T N}, and perform aggregation operation on the single associated data tables belonging to the same sibling relationship to obtain a new single associated data table set T′={T1′,T2′,…,T n ′,…}, where T n ' represents a new single-relational data table obtained by performing an aggregation operation on single-relational data tables of the same sibling relationship;

[0077] Update the 2-item association relationship set R through a single association data table set T′ to obtain the updated 2-item association relationship set R * :R * ={T a ′→T b ′}, where T a ′ and T b ' represents any single associated data table in the set of single associated data tables T'; T a ′→T b ' represents a single associated data table T a ' is a single associated data table T a ''s subtable;

[0078] (2.3) According to the updated 2-item association relationship set R *, mapping the parent-child relationship between each associated data table in the single associated data table set T′ into a graph connection form, and using arrows to point from the child table to the parent table to obtain a directed graph representation; the directed graph representation includes nodes and arrows, the nodes represent the table name of each single associated data table, and the arrow direction is from the child table to the parent table;

[0079] According to the directed graph representation, a corresponding directed graph node information list is obtained, wherein the directed graph node information list includes the parent table and child table information of each single association data table.

[0080] Step 3: Search and obtain the main table of the single associated data table set T′;

[0081] The step three specifically includes the following sub-steps:

[0082] (3.1) Select a single associated data table T from the set of single associated data tables T′ h ' is set as the local master table, T h ′∈T′, use the reverse full-join search algorithm to calculate a single associated data table T defined as the local main table h 'Data table list;

[0083] (3.2) For the data table list obtained in step (3.1), check whether there is a single associated data table set T′ that is not included in the single associated data table T h ′; if it exists, select another single associated data table in the associated data table set T′ as the local master table and repeat steps (3.1) and (3.2); if it does not exist, then replace the single associated data table T h ' is the main table of a single associated data table set T'.

[0084] The reverse full-join search algorithm is used to calculate a single associated data table T defined as the local main table. h The data table list of ′ is as follows:

[0085] (b1) The data table list is initialized to empty, and then a single associated data table T is set as the local main table. h 'Add to the data table list; find the local master table T h ′, update the data table list: in the data table list, add the local main table T h ' and each corresponding sub-table form a data table combination; the data table combination is a combination of single associated data tables based on a parent-child relationship, with the single associated data tables connected by directed arrows, with the child table connected to the parent table; the child table of the single data table combination is represented by the last single associated data table in the single data table combination;

[0086] (b2) Based on the data table list obtained in step (b1), check each data table combination in the data table list one by one to see if there is a single associated data table belonging to a sub-table of a data table combination in the data table list. If the check result is "yes", it indicates that the data table is not empty, and the process proceeds to step (b3). If the check result is "no", it indicates that the data table is empty, and the process proceeds to step (b4);

[0087] (b3) Check the sub-tables of the data table combination in the data table list one by one, add the found sub-tables to the end of the current data table combination, connect them with directed arrows in the middle to form a new data table combination, and use the new data table combination to replace the original data table combination to complete the update of the data table list; then input the updated data table list into step (b2);

[0088] (b4) Output a single associated data table T h ’ data table list.

[0089] Step 4: Classify the data table list of the main table of the single associated data table set T′ and convert the data table list of the main table into a hierarchical structure; based on the hierarchical structure, starting from the last layer, check the diversity of the corresponding parent tables of the single associated data tables at the same layer in sequence. For parent tables occupying different layers, delete the paths that pass through a large number of nodes, update the hierarchical structure, and obtain an updated hierarchical structure; based on the updated hierarchical structure, construct a multi-path cycle set; based on the data table list of the main table, the updated hierarchical structure, and the multi-path cycle set, obtain the multiple relationship classification sets corresponding to the single associated data table set T′;

[0090] The step 4 specifically includes the following sub-steps:

[0091] (4.1) Classify the data table list of the main table of a single associated data table set T′ and transform the data table list of the main table into a hierarchical structure RL = [RL1, RL2, RL3, ...RL k ,…], where RL1 represents the structure of the first layer, which contains the main table; RL2 represents the structure of the second layer, which contains the sub-table corresponding to the main table; RL3 represents the structure of the third layer, which contains a single associated data table corresponding to the single associated data table in RL2; k Represents the structure of the kth layer, RL k Including RL k-1 The single associated data table within the corresponding single associated data table; and so on;

[0092] (4.2) Based on the hierarchical structure RL, starting from the last layer, the diversity of the corresponding parent tables of the single association data table at the same layer is checked in sequence. For the parent tables occupying different layers, the paths with a large number of nodes are deleted, and the hierarchical structure RL is updated to obtain the updated hierarchical structure RL′.

[0093] (4.3) Based on the updated hierarchical structure RL′, construct a multi-path cycle set in, Indicates the multipath situation of the kth layer; if Represents a single associated data table T at the kth level ki There are multiple corresponding parent tables, T ki1 Represents a single associated data table T ki The corresponding first parent table, T ki2 Represents a single associated data table T ki The corresponding second parent table; if Indicates that there is no multipath on layer k.

[0094] (4.4) According to the data table list of the main table, the updated hierarchical structure RL′ and the multi-path loop set, multiple relationship classification sets corresponding to a single associated data table set T′ are obtained, where any relationship classification set is RM1 represents the parent-child relationship between a single associative data table in the last layer and the previous layer; RM2 represents the parent-child relationship between the second-to-last layer and the previous layer; RM3 represents the parent-child relationship between the third-to-last layer and the previous layer; and so on.

[0095] Step 5: Select any relationship classification set, aggregate the single-association data table set T′, and output the final aggregated data table of the single-association data table set T′;

[0096] The step five specifically includes the following sub-steps:

[0097] (5.1) Select any relationship classification set, aggregate a single relational data table in the RM1 layer, and then calculate the relationship T based on the RM1 layer. e ′→T f ', a single associated data table T e ' and a single associated data table T f 'Perform aggregation operation to obtain the aggregate data table T' e，f , and aggregate data table T′ e，f Replace a single associated data table T' f Repeat the above steps to perform aggregation operations on a single associated data table based on all the associations in the RM1 layer to obtain the aggregation results of the RM1 layer;

[0098] (5.2) Based on the aggregation result of the RM1 layer, replace the corresponding single association data table in the RM2 layer with the aggregation result of the RM1 layer; repeat step (5.1) and perform aggregation operations on the single association data table according to all the association relationships in the RM2 layer to obtain the aggregation result of the RM2 layer;

[0099] (5.3) Repeat step (5.2) until the aggregation result of the last layer of the relationship classification set is obtained;

[0100] (5.4) The aggregation result of the last layer is output as the final aggregated data table of a single associated data table set T′.

[0101] Example 2

[0102] In this embodiment, a single association data table set T′={T1′, T2′, . . . , T c ′…,T 11 '} as an example to illustrate the present invention, the single association data table set T' is a single association data set after the sibling relationship has been aggregated, and the single association data table set T' only has a parent-child correspondence relationship;

[0103] (1) The directed graph representation of a single associated data table set T′ is as follows Figure 5 As shown;

[0104] According to Figure 5 The directed graph shown in the figure obtains the directed graph node information list of a single associated data table set T′, as shown in Table 2;

[0105] Table 2: List of directed graph node information for a single associated data table set T′

[0106] Single associated data table Data table out (parent table) Data table entry (subtable) <![CDATA[T′1]]> <![CDATA[[T′4]]]> [ / ] <![CDATA[T′2]]> <![CDATA[[T′4]]]> [ / ] <![CDATA[T′3]]> <![CDATA[[T′7]]]> [ / ] <![CDATA[T′4]]> <![CDATA[[T′6]]]> <![CDATA[[T′1,T′2]]]> <![CDATA[T′5]]> <![CDATA[[T′6,T′9]]]> [ / ] <![CDATA[T′6]]> <![CDATA[[T′ 10 ]]]> <![CDATA[[T′4,T′5]]]> <![CDATA[T′7]]> <![CDATA[[T′ 10 ]]]> <![CDATA[[T′3,T′8]]]> <![CDATA[T′8]]> <![CDATA[[T′7]]]> [ / ] <![CDATA[T′9]]> <![CDATA[[T′ 10 ]]]> <![CDATA[[T′5]]]> <![CDATA[T′ 10 ]]> [ / ] <![CDATA[[T′6,T′7,T′9,T′ 11 ]]]> <![CDATA[T′ 11 ]]> <![CDATA[[T′ 10 ]]]> [ / ]

[0107] As can be seen from Table 2, the directed graph node information list contains the parent table and child table of each single associated data table, where "Data Table_Out" represents the parent table information corresponding to the single associated data table, and "Data Table_In" represents the child table information corresponding to the single associated data table; taking the single associated data table T1′ as an example, the parent table of the single associated data table T1′ is T4′.

[0108] (2) Search and obtain the main table of a single associated data table set T′;

[0109] (2.1) Select a single associated data table T′ from the set of single associated data tables T′ 10 Set as the local master table, use the reverse full join search algorithm to calculate the single associated data table T' set as the local master table 10 List of data tables;

[0110] The reverse full-connection search algorithm is used to calculate the single associated data table T' defined as the local main table 10 The data table list specifically includes the following sub-steps:

[0111] (2.1.1) The data table list is initialized to empty, and then a single associated data table T' is set as the local main table 10 Add to the data table list, the data table list is [T′ 10 ]; then search for the local master table T′ 10 All word tables: T'6, T'7, T'9 and T' 11 , update the data table column: in the data table list, add the local main table T' 10 A data table combination is formed with each corresponding word table. The updated data table list is: In a single data table combination T′6→T′ 10 The subtable in is T′6, which is a single data table combination T′6→T′ 10 The last single associated data table.

[0112] (2.1.2) Based on the updated data table list in step (2.1.1), check each data table combination in the data table list one by one to see if there is a single associated data table that belongs to a subtable of a data table combination in this data table list; If the check result is "yes", it means it is not empty, and go to step (2.1.3);

[0113] (2.1.3) Check the sub-tables of the data table combination in the data table list one by one, add the found sub-tables to the end of the current data table combination, connect them with directed arrows in the middle to form a new data table combination, and use the new data table combination to replace the original data table combination;

[0114] Data table combination T′6→T′ 10 The child tables are T'4 and T'5, and T'4 and T'5 are added to the data table combination T'6→T' 10 The middle part is connected by directed arrows to form a new data table combination: T′4→T′6→T′ 10 and T′5→T′6→T′ 10 ; Repeat the above steps to update the data table list; the updated data table list is

[0115] After the update is completed, enter the updated data table list into step (2.1.2) and repeat the above steps until the check result is "No", which means it is empty, and go to step (2.1.4);

[0116] (2.1.4) Single associated data table T′ 10 The data table list is output, and the single associated data table T′ 10 The data table list is:

[0117] (2.2) For the data table list obtained in step (2.1), check whether there is a single associated data table set T' that is not included in the single associated data table T' 10 A single linked data table in the data table list; if it is found not to exist, the single associated data table T' 10 As the main table of a single associated data table set T'.

[0118] Master table T' 10 The data table list records the aggregation path and aggregation direction of a single associated data table in the associated data table set T'. 10 A data table list is obtained, and an aggregation operation is performed on a single associated data table in the single associated data table set T′ to obtain a data aggregation table; the data aggregation table is a data table that contains data information of the single associated data table set T′.

[0119] (3) Classify the data table list of the main table of the single associative data table set T′ and convert the data table list of the main table into a hierarchical structure; based on the hierarchical structure, starting from the last layer, check the diversity of the corresponding parent tables of the single associative data tables in the same layer in sequence, delete the paths that pass through a large number of nodes for the parent tables occupying different layers, update the hierarchical structure, and obtain an updated hierarchical structure; based on the updated hierarchical structure, construct a multi-path cycle set; based on the data table list of the main table, the updated hierarchical structure, and the multi-path cycle set, obtain a variety of relationship classification sets corresponding to the single associative data table set T′;

[0120] (3.1) For the main table T′ 10 Classify the data table list and classify the main table T′ 10 The data table list is converted into a hierarchical structure RL=[RL1={T 10 ′}, RL2={T6′, T7′, T9′, T 11 '}, RL3 = {T3', T4', T5', T8'}, RL4 = {T1', T2'}]. RL1 represents the structure of the first layer, including the main table T' 10 RL2 represents the structure of the second layer, which contains the sub-tables corresponding to the single relational data tables of the first layer, namely the main table T' 10 Subtables T6′, T7′, T9′ and T′ 11RL3 represents the structure of the third layer, which includes the subtables corresponding to the single associated data tables of the second layer, namely, the subtables T4′ and T5′ of the single associated data table T6′, the subtables T3′ and T8′ of the single associated data table T7′, and the subtable T5′ of the single associated data table T9′; RL4 represents the structure of the fourth layer, which includes the subtables corresponding to the single associated data tables of the third layer, namely, the subtables T1′ and T2′ of the single associated data table T4′; the hierarchical structure of the single associated data table set T′ is as follows: Figure 7 shown.

[0121] (3.2) Based on the hierarchical structure RL, starting from the last layer, the diversity of the corresponding parent tables of the single associative data table at the same layer is checked in turn. For the parent tables occupying different layers, the paths passing through a large number of nodes are deleted, and the hierarchical structure RL is updated to obtain the updated hierarchical structure RL′. In Example 2, there are no parent tables occupying different layers, and the updated hierarchical structure RL′ is consistent with the hierarchical structure RL.

[0122] (3.3) The multi-path cycle set corresponding to the updated hierarchical structure RL′ is obtained: It can be seen from the multi-path cycle set that there is no multi-path situation in the 1st, 2nd and 4th layers; there are two paths in the single associative data table T5′ in the 3rd layer, that is, the single associative data table T5′ has two corresponding parent tables.

[0123] (3.4) According to the main table T′ 10 The data table list, the updated hierarchical structure RL′ and the multi-path loop set are used to obtain a relationship classification set corresponding to a single association data table set T′, including a first relationship classification set and a second relationship classification set;

[0124] The first relationship classification set is: Among them, RM1 represents the parent-child relationship between a single associative data table in the last layer and the previous layer; RM2 represents the parent-child relationship between the second-to-last layer and the previous layer; RM3 represents the parent-child relationship between the third-to-last layer and the previous layer; in this embodiment, there are a total of 4 layers, so RM1, RM2 and RM3 exist in the relationship classification set.

[0125] The second relationship classification set is:

[0126] (4) Select any relationship classification set, aggregate the single-association data table set T′, and output the final aggregated data table of the single-association data table set T′;

[0127] The step (4) is specifically implemented through the following sub-steps:

[0128] (4.1) Select the second relational classification set, aggregate the single relational data table of the RM1 layer, and based on the relational relationship T1′→T4′, aggregate the single relational data table T1′ and the single relational data table T4′ to obtain the aggregated data table T′ 4，1 ; Based on the association relationship T2′→T4′, the single association data table T2′ and the aggregate data table T′ 4，1 Perform aggregation operation to obtain the aggregate data table T′ 4，1，2 ; The aggregate data table T' 4，1，2 This is the aggregation result of the RM1 layer;

[0129] (4.2) Based on the aggregation results of RM1 layer, use the aggregation data table T′ 4，1，2 Replace the corresponding single associated data table T4' in the RM2 layer; based on the relationship T3'→T7', aggregate the single associated data table T3' and the single associated data table T7' to obtain the aggregated data table T' 7，3 ; Based on the relationship T4'→T6', aggregate data table T' 4，1，2 Perform aggregation operation on single associated data table T6′ to obtain aggregate data table T′ 6，4，1，2 Based on the relationship T5′→T9′, perform aggregation operation on the single associated data table T5′ and the single associated data table T9′ to obtain the aggregated data table T′ 9，5 ; Based on the relationship T8′→T7′, the single associated data table T8′ and the aggregate data table T′ 7，3 Perform aggregation operation to obtain the aggregate data table T′ 7，3，8 ; The aggregate data table T' 6，4，1，2 , aggregate data table T′ 9，5 And the aggregate data table T' 7，3，8 It is the aggregation result of RM2 layer;

[0130] (4.3) Based on the aggregation results of the RM2 layer, the single associated data table T7′ of the RM3 layer is replaced with the aggregated data table T′ 7，3，8 , Single associated data table T6' is replaced by aggregate data table T' 6，4，1，2 And the single associated data table T9' is replaced by the aggregate data table T' 9，5 ; Based on the relationship T6′→T′ 10 , a single associated data table T′ 10 and aggregate data table T′ 6，4，1，2 Perform aggregation operation to obtain the aggregate data table T′ 10，6，4，1，2 ; Based on the relationship T7′→T′ 10 , aggregate data table T′ 7，3，8 and aggregate data table T′ 10，6，4，1，2Perform aggregation operation to obtain the aggregate data table T′ 10，6，4，1，2，7，3，8 ; Based on the relationship T9′→T′ 10 , aggregate data table T′ 9，5 and aggregate data table T′ 10，6，4，1，2，7，3，8 Perform aggregation operation to obtain the aggregate data table T′ 10，6，4，1，2，7，3，8，9，5 ; Based on the relationship T′ 11 →T′ 10 , a single associated data table T′ 11 and aggregate data table T′ 10，6，4，1，2，7，3，8，9，5 Perform aggregation operation to obtain the aggregate data table T′ 10，6，4，1，2，7，3，8，9，5，11 ; The aggregate data table T' 10，6，4，1，2，7，3，8，9，5，11 It is the aggregation result of RM3 layer;

[0131] (4.4) aggregate data table T′ 10，6，4，1，2，7，3，8，9，5，11 The final aggregated data table is output as a single associated data table set T′.

[0132] Example 3

[0133] See also Figure 7 An embodiment of the present invention provides a data table association relationship classification device based on graph search technology, which includes one or more processors for implementing the data table association relationship classification method based on graph search technology in the above embodiment.

[0134] The embodiment of the data table association relationship classification device based on graph search technology of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 7 As shown in the figure, it is a hardware structure diagram of any device with data processing capability where the data table association relationship classification device based on graph search technology of the present invention is located. Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0135] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0136] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0137] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for classifying data table association relationships based on graph search technology in the above embodiment is implemented.

[0138] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data table association relationship classification method based on graph search technology, characterized in that: The following steps are involved: Step 1: Split the field classification summary table of the association data table into a set of single association data tables T; and construct a set of two-item association relationships R between the single association data tables based on the relationship between the primary key and other fields of each single association data table in the set of single association data tables T; Step 2: Perform full connectivity check on the 2-item association relationship set R: If the 2-item association relationship set R has full connectivity, then perform aggregation operation on the single association data tables belonging to the same sibling relationship in the single association data table set T to generate a new single association data table set T′, and update the 2-item association relationship set R to obtain the updated 2-item association relationship set R * , generate the corresponding directed graph representation; if the 2-item association relationship set R does not have full connectivity, add more information; Step 3: Search and obtain the main table of the single associated data table set T′; The step three specifically includes the following sub-steps: (3.1) Select a single associated data table T′ from the set of single associated data tables T′ h Set as the local master table, use the reverse full join search algorithm to calculate the single associated data table T' set as the local master table h List of data tables; (3.2) For the data table list obtained in step (3.1), check whether there is a single associated data table set T' that is not included in the single associated data table T' h If it exists, select another single associated data table in the associated data table set T' as the local main table and repeat steps (3.1) and (3.2); if it does not exist, then the single associated data table T' h As the main table; The single associated data table T′ set as the local main table is calculated using the reverse full connection search algorithm h The data table list is as follows: (b1) The data table list is initialized to empty, and then a single associated data table T′ is set as the local main table. h Add to the data table list; Find the local master table T′ h All word tables, update the data table list: in the data table list, add the local main table T' h A data table combination is formed with each corresponding sub-table; the data table combination is a combination of single associated data tables based on a parent-child relationship, with the single associated data tables connected by directed arrows, with the child table connected to the parent table; the child table of the single data table combination is represented by the last single associated data table in the single data table combination; (b2) Based on the data table list obtained in step (b1), check each data table combination in the data table list one by one to see if there is a single associated data table that belongs to a sub-table of a data table combination in the data table list. If the check result is "yes", it indicates that the table is not empty, and the process proceeds to step (b3). If the check result is "no", it indicates that the table is empty, and the process proceeds to step (b4); (b3) Check the sub-tables of the data table combination in the data table list one by one, add the found sub-tables to the end of the current data table combination, connect them with directed arrows in the middle to form a new data table combination, and use the new data table combination to replace the original data table combination to complete the update of the data table list; then input the updated data table list into step (b2); (b4) Output a single relational data table T′ h List of data tables; Step 4: Classify the data table list of the main table of the single associated data table set T′ and convert the data table list of the main table into a hierarchical structure; based on the hierarchical structure, starting from the last layer, check the diversity of the corresponding parent tables of the single associated data tables at the same layer in sequence. For parent tables occupying different layers, delete the paths that pass through a large number of nodes, update the hierarchical structure, and obtain an updated hierarchical structure; based on the updated hierarchical structure, construct a multi-path cycle set; based on the data table list of the main table, the updated hierarchical structure, and the multi-path cycle set, obtain the multiple relationship classification sets corresponding to the single associated data table set T′; Step 5: Select any relationship classification set, aggregate the single-association data table set T′, and output the final aggregated data table of the single-association data table set T′.

2. A data table association relationship classification method based on graph search technology according to claim 1, characterized in that: The step 1 is specifically as follows: (1.1) Split the field classification summary table of the relational data table into N single relational data tables, and obtain a set of single relational data tables T = {T1, T2, ..., T m ,…,T N }, where subscript m is an ordinal number, m = 1, 2, ... m, ... N, T m Represents the mth single associative data table; and vectorizes each single associative data table to obtain a vector representation set Represents the mth single-item associated data table T m The vector representation of Specifically: Among them, T_m represents a single associated data table T m The table name, KEY_m represents a single associated data table T m The primary key of FEATURES_m represents a single associated data table T m Other fields of For each vector Vectorize and divide into and (1.2) Determine a single associated data table set T = {T1, T2, ..., T m ,…,T N }, construct a 2-item association relationship set R: Among them, T i and T j Is any single associated data table. If a single associated data table T i and T j The association relationship consists of brother relationships, then Indicated as T i -T j ; If a single associated data table T i and T j The association relationship consists of a parent-child relationship, then Indicated as T i →T j .

3. The data table association relationship classification method based on graph search technology according to claim 1 is characterized in that: The step 2 is specifically as follows: (2.1) performing a full connectivity check on the 2-item association relationship set R generated in step (1.2), wherein full connectivity indicates that all single association-type data tables can be connected through a sibling relationship or a parent-child relationship; if the 2-item association relationship set R has full connectivity, proceed to step (2.2); if the 2-item association relationship set R does not have full connectivity, provide more information; (2.2) According to the 2-item association relationship set R, the single association data table set T = {T1, T2, ..., T m ,…,T N }, and perform aggregation operations on the single associated data tables belonging to the same sibling relationship to obtain a new single associated data table set T′={T1′,T2′,…,T n ′,…}, where T n ' represents a new single-relational data table obtained by performing an aggregation operation on single-relational data tables of the same sibling relationship; And update the 2-item association relationship set R to obtain the updated 2-item association relationship set R * :R * ={T a ′→T b ′}, where T a ′ and T b ' represents any single associated data table in the set of single associated data tables T'; T a ′→T b ' represents a single associated data table T a ' is a single associated data table T a ''s subtable; (2.3) According to the updated 2-item association relationship set R * , mapping the parent-child relationship between each associated data table in the single associated data table set T′ into a graph connection form, and using arrows to point from the child table to the parent table to obtain a directed graph representation; the directed graph representation includes nodes and arrows, the nodes represent the table name of each single associated data table, and the arrow direction is from the child table to the parent table; According to the directed graph representation, a corresponding directed graph node information list is obtained, wherein the directed graph node information list includes the parent table and child table information of each single association data table.

4. The data table association relationship classification method based on graph search technology according to claim 1 is characterized in that: The step 4 specifically includes the following sub-steps: (4.1) Classify the data table list of the main table of a single associated data table set T′ and transform the data table list of the main table into a hierarchical structure RL = [RL1, RL2, RL3, ...RL k ,…], where RL1 represents the structure of the first layer, which contains the main table; RL2 represents the structure of the second layer, which contains the sub-table corresponding to the main table; RL3 represents the structure of the third layer, which contains a single associated data table corresponding to the single associated data table in RL2; k Represents the structure of the kth layer, RL k Including RL k-1 The single associated data table within the corresponding single associated data table; and so on; (4.2) Based on the hierarchical structure RL, starting from the last layer, the diversity of the corresponding parent tables of the single association data table at the same layer is checked in sequence. For the parent tables occupying different layers, the paths with a large number of nodes are deleted, and the hierarchical structure RL is updated to obtain the updated hierarchical structure RL′. (4.3) Based on the updated hierarchical structure RL′, construct a multi-path cycle set in, Indicates the multipath situation of the kth layer; if Represents a single associated data table T at the kth level ki There are multiple corresponding parent tables, T ki1 Represents a single associated data table T ki The corresponding first parent table, T ki2 Represents a single associated data table T ki The corresponding second parent table; if Indicates that there is no multipath situation at layer k; (4.4) According to the data table list of the main table, the updated hierarchical structure RL′ and the multi-path loop set, multiple relationship classification sets corresponding to a single associated data table set T′ are obtained, where any relationship classification set is RM1 represents the parent-child relationship between a single associative data table in the last layer and the previous layer; RM2 represents the parent-child relationship between the second-to-last layer and the previous layer; RM3 represents the parent-child relationship between the third-to-last layer and the previous layer; and so on.

5. The data table association relationship classification method based on graph search technology according to claim 4 is characterized in that: The step five specifically includes the following sub-steps: (5.1) Select any relationship classification set, aggregate a single relational data table in the RM1 layer, and then calculate the relationship T based on the RM1 layer. e ′→T f ', a single associated data table T e ' and a single associated data table T f 'Perform aggregation operation to obtain the aggregate data table T' e,f , and aggregate data table T′ e,f Replace a single associated data table T' f Repeat the above steps to perform aggregation operations on a single associated data table based on all the associations in the RM1 layer to obtain the aggregation results of the RM1 layer; (5.2) Based on the aggregation result of the RM1 layer, replace the corresponding single association data table in the RM2 layer with the aggregation result of the RM1 layer; repeat step (5.1) and perform aggregation operations on the single association data table according to all the association relationships in the RM2 layer to obtain the aggregation result of the RM2 layer; (5.3) Repeat step (5.2) until the aggregation result of the last layer of the relationship classification set is obtained; (5.4) The aggregation result of the last layer is output as the final aggregated data table of a single associated data table set T′.

6. A data table association relationship classification device based on graph search technology, characterized in that: It includes one or more processors for implementing a data table association relationship classification method based on graph search technology as described in any one of claims 1-5.

7. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement a data table association relationship classification method based on graph search technology as described in any one of claims 1 to 5.

Images