Database table business domain division method and device, medium and product

Abstract

The invention discloses a business domain division method and device for a database table, a medium and a product, and relates to the technical field of data processing. According to the scheme, when the business domain grouping requirement of the database table exists, from the three aspects of the structure, the semantics and the behavior of the database table, the similarity degree of three dimensions including the foreign key relation, the name similarity and the common current mode between the database tables is comprehensively considered; the problem that effective information in metadata is difficult to extract due to the fact that only an NLP model is used for unilaterally extracting database table information in a traditional method is effectively avoided; and finally, determining a corresponding comprehensive weight by utilizing the structure weight, the semantic weight and the behavior weight corresponding to each database table, namely dividing each database table into a plurality of business domain groups according to each comprehensive weight, so that manual intervention can be effectively reduced, and the method is suitable for a large-scale database scene.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, medium and product for dividing a database table into business domains. Background Art

[0002] Business domain segmentation of database tables is a crucial step in data governance and data architecture design. It aims to improve manageability and understandability by logically grouping large numbers of tables based on business relevance. However, with the rapid growth of enterprise data, traditional manual classification methods are no longer sufficient. Modern data environments urgently require systematic business domain segmentation.

[0003] However, current automated business domain segmentation methods face numerous challenges: many database tables lack detailed annotations and descriptions, naming conventions and annotation styles vary, general natural language processing (NLP) models struggle to accurately grasp industry- or enterprise-specific business terminology, and table and column names are often riddled with meaningless abbreviations. These issues make it difficult to extract meaningful information from metadata, leading to erroneous intermediate results and significantly reducing the accuracy of database table classification.

[0004] In view of the above, how to solve the current problems of missing database table annotations, inconsistent naming, and difficult to parse industry terminology, which lead to difficulties in metadata extraction and reduced classification accuracy, is an urgent problem that needs to be solved by technical personnel in this field. Summary of the Invention

[0005] The present invention provides a method, device, medium and product for dividing the business domain of a database table, so as to at least solve the current problems of missing database table annotations, inconsistent naming and difficult parsing of industry terms, which lead to difficulty in metadata extraction and reduced classification accuracy.

[0006] The present invention provides a method for dividing a database table into business domains, comprising: Obtain multiple database tables in the database for the business domain to be divided; Perform foreign key analysis, name similarity calculation, and co-occurrence frequency evaluation on each database table to obtain the structural weight, semantic weight, and behavioral weight corresponding to each database table. The structural weight represents the degree of structural association between database tables with foreign key relationships; the semantic weight represents the degree of semantic association between database tables; and the behavioral weight represents the degree of behavioral association between database tables that co-occur in historical query instructions. Determine the comprehensive weight corresponding to each database table based on the structural weight, semantic weight, and behavioral weight corresponding to each database table; Each database table is divided into multiple business domain groups according to the comprehensive weight.

[0007] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for dividing the service domain of a database table when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for dividing a database table into business domains are implemented.

[0009] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned methods for dividing a database table into business domains when the computer program is executed by a processor.

[0010] The beneficial effect of the present invention is that when there is a demand for business domain grouping of database tables, starting from the three aspects of the structure, semantics and behavior of the database tables, the similarity of the three dimensions of foreign key relationships, name similarity and co-occurrence behavior patterns between the database tables is comprehensively considered, effectively avoiding the problem of traditional methods that only use NLP models to unilaterally extract database table information and it is difficult to extract valid information in metadata; finally, the corresponding structural weight, semantic weight and behavior weight of each database table are used to determine the corresponding comprehensive weight, and each database table can be divided into multiple business domain groups according to the comprehensive weights, which can effectively reduce manual intervention and adapt to large-scale database scenarios.

[0011] In addition, the present invention also provides a business domain division device, medium and product for a database table, which have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A flowchart of a method for dividing a database table into business domains provided by an embodiment of the present invention; Figure 2 A schematic diagram of a device for dividing a database table into service domains provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0015] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0016] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0017] Business domain partitioning of database tables refers to the process of categorizing and organizing database tables according to business functions or modules. This partitioning helps improve the maintainability, scalability, and understandability of the database, allowing developers and business personnel to more clearly understand the database structure and data relationships.

[0018] Currently, automated business domain segmentation methods face numerous challenges: many database tables lack detailed annotations and descriptions, naming conventions and annotation styles vary, general NL (Learning Language) models struggle to accurately capture industry- or enterprise-specific business terminology, and table and column names are often riddled with meaningless abbreviations. These issues make it difficult to extract meaningful information from metadata, leading to erroneous intermediate results and significantly reducing the accuracy of database table classification. To address these issues, the present invention provides a method for segmenting database tables into business domains.

[0019] Figure 1 Flowchart of a method for dividing a database table into business domains according to an embodiment of the present invention. Figure 1 As shown, the method includes: S10: Acquire multiple database tables of the business domains to be divided in the database.

[0020] Specifically, first obtain multiple database tables in the database that represent the business domains to be divided. For example, if there are five tables in the database, including customers, orders, products, employees, and logistics, these tables need to be divided into business domains.

[0021] S11: Perform foreign key analysis, naming similarity calculation, and co-occurrence frequency evaluation on each database table to obtain the structural weight, semantic weight, and behavioral weight corresponding to each database table.

[0022] Among them, the structural weight represents the degree of structural association between database tables with foreign key relationships; the semantic weight represents the degree of semantic association between database tables; and the behavioral weight represents the degree of behavioral association between database tables that co-occur in historical query instructions.

[0023] Furthermore, foreign key analysis, naming similarity calculation and co-occurrence frequency evaluation are performed on each database table to obtain the structural weight, semantic weight and behavioral weight corresponding to each database table.

[0024] It's important to note that structural weight represents the degree of structural association between database tables with foreign key relationships. The closer the foreign key relationship, the higher the structural weight, indicating a closer structural connection between the two tables. Structural weight is one of the key weighting indicators for table grouping, but some database designs don't include explicit foreign keys. This is particularly true in the design of business tables in the big data field, where foreign keys are often not reflected in the Data Definition Language (DDL). Semantic weight represents the degree of semantic association between database tables, including but not limited to the linguistic association between table names and field names. Behavioral weight represents the degree of behavioral association between database tables that co-occur in historical query instructions (Structured Query Language (SQL)).

[0025] It is worth noting that the structural weights, semantic weights, and behavioral weights corresponding to each database table in this embodiment are all structural weights, semantic weights, and behavioral weights between two database tables. For example, if a database contains Table A, Table B, and Table C, and Table A has a corresponding structural weight, these are specifically the structural weight between Table A and Table B, and the structural weight between Table A and Table C. The same applies to the remaining tables and their corresponding weights.

[0026] In this embodiment, there is no limitation on the specific process of determining the structural weight, semantic weight, and behavioral weight corresponding to each database table, which is determined according to the specific implementation situation.

[0027] S12: Determine the comprehensive weight corresponding to each database table according to the structural weight, semantic weight, and behavioral weight corresponding to each database table.

[0028] Subsequently, the comprehensive weight corresponding to each database table is determined based on the structural weight, semantic weight, and behavioral weight corresponding to each database table. In this embodiment, there is no limitation on the method for calculating the comprehensive weight. For example, the structural weight, semantic weight, and behavioral weight can be directly summed, or the structural weight, semantic weight, and behavioral weight can be weighted and summed, depending on the specific implementation.

[0029] S13: Divide each database table into multiple business domain groups according to the comprehensive weights.

[0030] Finally, each database table is divided into multiple business domain groups according to the comprehensive weights, thereby achieving business classification of the database table. In this embodiment, the specific process of dividing the business domain groups is not limited and depends on the specific implementation situation.

[0031] In this embodiment, when there is a need to group database tables by business domain, starting from the three aspects of the structure, semantics and behavior of the database tables, the similarity of the three dimensions of foreign key relationships, name similarity and co-occurrence behavior patterns between the database tables is comprehensively considered, which effectively avoids the problem of traditional methods that only use NLP models to unilaterally extract database table information and are difficult to extract valid information in metadata; finally, the corresponding structural weight, semantic weight and behavioral weight of each database table are used to determine the corresponding comprehensive weight, and each database table can be divided into multiple business domain groups according to the comprehensive weights, which can effectively reduce manual intervention and adapt to large-scale database scenarios.

[0032] Based on the above embodiments, in some embodiments, performing foreign key analysis on each database table includes: S101: Parse the data location language of the database and extract the foreign key relationships between the database tables.

[0033] S102: Determine the parent table corresponding to each database table based on the corresponding foreign key relationship.

[0034] S103: Determine the number of parent tables corresponding to each database table to determine the structural weight between each database table and the corresponding parent table.

[0035] To determine the structural weights corresponding to each database table, we first need to parse the database DLL and extract the foreign key relationships between database tables. A foreign key relationship between database tables is a constraint used to establish a connection between tables. It is implemented by referencing the primary key or unique key of another table (called the parent table) in one table (called the child table or detail table). Foreign keys ensure that data in the child table has a corresponding record in the parent table, thereby maintaining data integrity and consistency. This relationship supports referential integrity, prevents orphaned records, and facilitates data association and query operations between tables.

[0036] Furthermore, based on the corresponding foreign key relationships, the parent table corresponding to each database table is determined. For example, the orders table contains foreign keys: customer_id → customers.id, product_id → products.id; the logistics table contains foreign keys: order_id → orders.id, employee_id → employees.id. Therefore, for orders, its parent tables include customers and products, and for logistics, its parent tables include orders and employees.

[0037] Finally, determine the number of parent tables corresponding to each database table to determine the structural weight between each database table and its corresponding parent table. For example, if the parent table of "orders" includes "customers" and "products," then the resulting weight between "orders" and its corresponding parent table "customers" is 1 (the number of parent tables "customers" in the parent table), and the resulting weight between "orders" and its corresponding parent table "products" is also 1 (the number of parent tables "products" in the parent table). Table 1 shows the structural weights corresponding to several database tables.

[0038] Table 1 Structural weight table It should be noted that although structural weight is one of the decisive weight indicators for grouping database tables in the present invention, some database designs do not have explicit foreign keys. Especially in the business table design in the big data field, foreign keys are often not reflected in the DDL. Therefore, some database tables do not have structural weights.

[0039] In this embodiment, the foreign key relationship between tables is extracted by parsing the DDL of the database to measure the structural correlation between the tables. The tighter the foreign key relationship, the higher the structural weight, indicating that the structural connection between the two tables is tighter. This is used as one of the decisive weight indicators to measure the similarity between the two tables, which can ensure the accuracy of database table grouping.

[0040] Based on the above embodiment, in some embodiments, performing naming similarity calculation on each database table includes: S111: determining the table name of each database table, and determining the similarity between the table names of each database table according to the table name of each database table.

[0041] S112: Determine the field name of each database table, and determine the field similarity between the database tables based on the field name of each database table.

[0042] S113: Determine the semantic weights between the database tables based on the similarity of the table names, the similarity of the fields, the first secondary coefficient, and the second secondary coefficient.

[0043] To determine the semantic weights corresponding to each database table, this embodiment performs semantic analysis on both table and field names to implement naming similarity calculation. Specifically, the table name of each database table is first determined, and then the table name similarity between the database tables is determined based on the table names of each database table. The method for calculating the table name similarity between the database tables in this embodiment is not limited and depends on the specific implementation.

[0044] Next, the field names of each database table are determined. A database table field is a column within a table used to store specific types of data. Each field defines the data's name, data type, and possible constraints to ensure data accuracy and consistency. The field similarity between the database tables is then determined based on the field names of each database table. In this embodiment, the method for calculating the field similarity between the database tables is not limited and depends on the specific implementation.

[0045] Finally, the semantic weights between database tables are determined based on the similarity of table names, field similarity, first-level coefficients, and second-level coefficients. The formula is as follows: Semantic weight = table name similarity × first secondary coefficient + field similarity × second secondary coefficient.

[0046] Among them, the sum of the first secondary coefficient and the second secondary coefficient is 1. In this embodiment, there is no restriction on the specific values ​​of the first secondary coefficient and the second secondary coefficient. It should be noted that in scenarios where the table naming rules are more prominent, the first secondary weight corresponding to the table name similarity can be increased. For example, the first secondary weight corresponding to the table name similarity (0.7) is higher than the second secondary coefficient corresponding to the field similarity (0.3), which is in line with the actual scenario of "table name dominates business classification, and field name supplements functional details" in database design, and improves the business interpretability of semantic weight. As shown in Table 2, the semantic weights corresponding to several database tables are given.

[0047] Table 2 Semantic weight table In this embodiment, the semantic weights between the database tables are determined based on the table name similarity and field similarity of each database table. The similarity of each database table is determined from a semantic perspective and used as one of the decisive weight indicators for measuring the similarity between two tables, which can ensure the accuracy of database table grouping.

[0048] In order to determine the similarity of table names between database tables, based on the above embodiment, in some embodiments, determining the similarity of table names between database tables according to the table names of the database tables includes: S121: Split the table name of each database table into multiple continuous subsequences according to a preset character granularity; wherein the preset character granularity includes at least single-character, double-character, and triple-character.

[0049] S122: Calculate the cosine similarity between the continuous subsequence corresponding to each database table and the continuous subsequences corresponding to the remaining database tables, so as to determine the similarity between the table names of the database tables.

[0050] To address the particularity of database table names, which are complex and contain non-standard vocabulary / abbreviations / mixed characters, and the inability of traditional word-based NLP tools to effectively handle them due to difficulties in word segmentation, this paper adopts a character-level term frequency-inverse document frequency (TF-IDF) vectorization solution.

[0051] Specifically, first split the table name of each database table into multiple consecutive subsequences according to a preset character granularity. The preset character granularity includes at least single-character, double-character, and triple-character granularities. A larger number of characters, such as four or even five, can also be used, but it is not recommended. This is because table names are generally short, and a sequence that is too large will exceed the table name length. Furthermore, a sequence that is too large will lead to semantic generalization and ambiguity, hindering subsequent table grouping. The following takes the specific splitting steps for the database tables orders and customers as an example: Split orders, string: orders → character sequence: o, r, d, e, r, s; 1-gram (single character): all contiguous subsequences of length 1: ['o','r','d','e','r','s']; 2-gram (two-character): all consecutive subsequences of length 2: ['or', 'rd', 'de', 'er', 'rs']; 3-grams (three characters): all consecutive subsequences of length 3: ['ord', 'rde', 'der', 'ers']; Split customers, string: customers → character sequence: c,u,s,t,o,m,e,r,s; 1-gram (single character): all contiguous subsequences of length 1: ['c','u','s','t','o','m','e','r','s']; 2-grams (two-character): all consecutive subsequences of length 2: ['cu','us','st','to','om','me','er','rs']; 3-grams (three characters): all contiguous subsequences of length 3: ['cus','ust','sto','tom','ome','mer','ers'].

[0052] Finally, the continuous subsequences corresponding to each database table are used as TF-IDF vectorized features. The cosine similarity between the continuous subsequences corresponding to each database table and the continuous subsequences corresponding to the remaining database tables is calculated to determine the table name similarity between the database tables. Taking orders and customers as an example, the cosine similarity between the continuous subsequences corresponding to orders and the continuous subsequences corresponding to customers is calculated to determine the table name similarity between orders and customers. It can be understood that the table name similarity between orders and customers is equal to the table name similarity between customers and orders. For example: cosine_similarity(customers,orders) = 0.65; cosine_similarity(orders,logistics) = 0.82.

[0053] In this embodiment, through character-level granularity control, single-character, double-character, triple-character and other features are taken into account, special scenario structures of table names are covered, and the continuous pattern of table name characters is directly captured, thereby improving the accuracy of similarity calculation.

[0054] In order to determine the field similarity between database tables, based on the above embodiment, in some embodiments, the field similarity between database tables is determined according to the field name of each database table, including: S131: Concatenate the field names of each database table into corresponding character strings.

[0055] S132: Based on the vector embedding function, perform vector embedding comparison on the character strings corresponding to each database table and the character strings corresponding to the other database tables to determine the field similarity between the database tables.

[0056] Specifically, concatenate the field names ['id', 'name'] of each database table into the corresponding string 'id, name'. For example, if the fields in the orders table are [id, customer_id, product_id, amount], the corresponding string is 'id, customer_id, product_id, amount'; if the fields in the logistics table are [id, order_id, employee_id, address], the corresponding string is 'id, order_id, employee_id, address'.

[0057] Then, using the vector embedding function, we perform a vector embedding comparison between the strings corresponding to each database table and the strings corresponding to the remaining database tables to determine the field similarity between the database tables. For example, the field similarity value after the vector embedding comparison of the (orders field group, logistics field group) is 0.78.

[0058] In this embodiment, the field name list is concatenated into a string, and the similarity is calculated at one time using a vector embedding function. This avoids the computational complexity of the traditional Cartesian product similarity based on the field matrix, can more accurately reflect the overall similarity of the table field design, and avoid misjudgments caused by differences in field order and partial semantic deviations.

[0059] In order to build on the above embodiments, in some embodiments, performing co-occurrence frequency evaluation on each database table includes: S141: extract historical query instructions based on the database, and determine whether a preset number of historical query instructions are obtained; if so, proceed to step S142.

[0060] S142: Extract the co-occurrence counts between the database tables in each historical query instruction.

[0061] S143: Determine a quotient of each co-occurrence number and a preset number to determine the behavior weights between the database tables.

[0062] To determine the behavior weights corresponding to each database table, this embodiment specifically extracts historical SQL statements from the database and determines whether a preset number of historical query instructions have been obtained. It should be noted that this embodiment does not impose a limit on the preset number; for example, it can be a certain multiple of the total number of database tables, or all historical SQL statements can be directly obtained, depending on the specific implementation.

[0063] Once the preset number of historical query instructions have been obtained, the co-occurrence counts between the database tables in each historical query instruction are extracted. Assuming three SQL statements are extracted, the tool uses these three SQL statements to extract the database table name co-occurrence features: SQL1 co-occurrence table: orders, customers, products, logistics, employees; SQL2 co-occurrence table: customers, orders, products; SQL3 co-occurrence table: employees, logistics. A co-occurrence relationship matrix is ​​constructed based on these three SQL statements, as follows: Table 3 Co-occurrence relationship matrix It can be understood that the values ​​in the table are the number of times the two database tables appear together in historical SQL statements.

[0064] Finally, after extracting the co-occurrence counts between database tables in each historical query instruction, the quotient of each co-occurrence count and a preset number is determined to determine the behavior weight between each database table. For example, among three historical SQL statements, the co-occurrence count of orders and customers is 2, and the number of historical SQL statements is 3. Therefore, the behavior weight between orders and customers is 2 / 3, which is approximately 0.66.

[0065] In this embodiment, the number of co-occurrences between each database table in each historical query instruction is extracted, and the behavioral weights between each database table are determined based on the corresponding co-occurrence numbers. The behavioral correlation between each database table is determined from the perspective of database table co-occurrence, and this is used as one of the decisive weight indicators to measure the similarity between two tables, which can ensure the accuracy of database table grouping.

[0066] In the specific implementation, there are some cases where historical SQL cannot be obtained. In order to ensure the acquisition of behavior weights, based on the above embodiment, in some embodiments, if it is confirmed that the preset number of historical query instructions have not been obtained, it also includes: S144: Obtain the data location language corresponding to each database table respectively.

[0067] S145: performing data standardization processing on each data positioning language and eliminating common words to obtain feature words corresponding to each database table.

[0068] S146: Based on the vector embedding function, the vector similarities between the feature words corresponding to each database table are calculated respectively to determine the behavior weights between the database tables.

[0069] Specifically, first obtain the DLL corresponding to each database table. Taking orders as an example, obtain the original DDL text of orders: "CREATE TABLE orders (id INT PRIMARY KEY COMMENT 'primary key', customer_idINT, ...)".

[0070] Each DLL then undergoes data normalization, for example, converting all statements to lowercase to eliminate case differences and removing common words to obtain the key words corresponding to each database table. For example, after normalization, the key words retained in the original DDL text for orders are "id primary, customer_id, product_id, amount..."

[0071] Finally, based on the vector embedding function, the vector similarities between the feature words corresponding to each database table are calculated to determine the behavioral weights between the database tables. It should be noted that the specific type of vector embedding function used in this embodiment is not limited and depends on the specific implementation. Table 4 shows the behavioral weights corresponding to several database tables.

[0072] Table 4 Behavior Weight Table As shown in the table above, the behavior weights are replaced by the values ​​in the table above.

[0073] In this embodiment, the behavior weight is approximately evaluated by calculating the similarity after preprocessing the DDL of the database table, which can avoid the inability to calculate the behavior weight due to the inability to obtain historical SQL and ensure the normal grouping of the database table.

[0074] Based on the above embodiments, in some embodiments, the comprehensive weight corresponding to each database table is determined according to the structural weight, semantic weight, and behavioral weight corresponding to each database table, including: S151: Determine a first coefficient corresponding to the structural weight, a second coefficient corresponding to the semantic weight, and a third coefficient corresponding to the behavioral weight.

[0075] S152: Determine the comprehensive weight corresponding to each database table according to the structural weight, semantic weight, and behavioral weight corresponding to each database table, as well as the first coefficient, the second coefficient, and the third coefficient.

[0076] In this embodiment, the structural weight, semantic weight and behavioral weight are weighted and summed to obtain the comprehensive weight of the corresponding database table. First, the first coefficient corresponding to the structural weight is determined. , the second coefficient corresponding to the semantic weight The third coefficient corresponding to the behavioral weight .

[0077] Finally, the comprehensive weight corresponding to each database table is determined based on the structural weight, semantic weight, and behavioral weight corresponding to each database table, as well as the first coefficient, second coefficient, and third coefficient. The formula is as follows: Comprehensive weight = structural weight × +Semantic weight× +Behavior Weight× .

[0078] in, is the first coefficient, is the second coefficient, The third coefficient is not limited in this embodiment. + + =1.

[0079] As shown in Table 5, the comprehensive weights corresponding to several database tables are given.

[0080] Table 5 Comprehensive weight table In this way, the comprehensive weight corresponding to each database table is accurately determined, so that the database tables can be grouped according to the comprehensive weights.

[0081] Based on the above embodiments, in some embodiments, each database table is divided into multiple business domain groups according to each comprehensive weight, including: S161: Select a target database table from among the database tables.

[0082] S162: Determine the comprehensive weight between the target database table and the remaining database tables.

[0083] S163: Generate a dynamic threshold corresponding to the target database table according to the comprehensive weights corresponding to the target database table.

[0084] S164: Determine whether there is a target comprehensive weight not less than the dynamic threshold among the comprehensive weights corresponding to the target database table; if so, proceed to step S165; if not, end.

[0085] S165: Divide the database table corresponding to the target comprehensive weight and the target database table into the same business domain group.

[0086] S166: Select a new target database table from the remaining database tables, and return to step S162 until all database tables are grouped.

[0087] To group database tables by business domain, this embodiment specifically selects a target database table from among the database tables. For example, the database table orders is selected as the target database table. The combined weights of the target database table and the remaining database tables are determined. For example, in addition to orders, the remaining database tables include customers, products, logistics, and employees. The combined weights of orders and the remaining database tables are: [0.76 (customers), 0.82 (products), 0.88 (logistics), 0.45 (employees)].

[0088] Subsequently, a dynamic threshold corresponding to the target database table is generated based on the comprehensive weights corresponding to the target database table. It should be noted that the dynamic threshold is a threshold corresponding only to the target database table; that is, each database table has a corresponding threshold. This embodiment does not restrict the process for determining the dynamic threshold.

[0089] Furthermore, it is determined whether any of the comprehensive weights corresponding to the target database table has a target comprehensive weight that is not less than a dynamic threshold. If it is confirmed that no comprehensive weight corresponding to the target database table has a target comprehensive weight that is not less than the dynamic threshold, it is determined that the target database table has a very low correlation with the remaining database tables, and the grouping of the target database table ends.

[0090] If it is confirmed that the target database table has a target comprehensive weight that is not less than the dynamic threshold among the comprehensive weights corresponding to the target database table, the database table corresponding to the target comprehensive weight and the target database table are grouped into the same business domain group. For example, when the dynamic threshold is 0.75, the comprehensive weights corresponding to orders (0.76 for customers), 0.82 for products, and 0.88 for logistics) are all greater than 0.75, and 0.45 for employees is less than 0.75. Therefore, orders is grouped into the same business domain group as logistics, products, and customers. Finally, a new target database table is selected from the remaining database tables, and the process returns to step S162 until all database tables are grouped.

[0091] This embodiment avoids the problem that traditional grouping methods use a global fixed threshold and cannot handle the differences in similarity distribution of different tables. Instead, a threshold is set separately for each table to adapt to its local similarity distribution and improve the rationality of grouping.

[0092] In order to determine the dynamic threshold corresponding to the target database table, based on the above embodiment, in some embodiments, the dynamic threshold corresponding to the target database table is generated according to the comprehensive weights corresponding to the target database table, including: S171: Sort the comprehensive weights corresponding to the target database table in descending order to generate a target sequence.

[0093] S172: Determine the difference value of the target sequence, and determine the absolute difference value based on the difference value.

[0094] S173: Determine the maximum difference value of the target sequence according to the absolute difference value.

[0095] S174: Determine the threshold interval where the maximum drop value is located in the target sequence.

[0096] S175: Determine the dynamic threshold corresponding to the target database table according to the endpoint value of the threshold interval.

[0097] Specifically, first sort the target database table's corresponding comprehensive weights in descending order to generate the target sequence. For example, if the combined weights of orders and the other database tables are [0.76 (customers), 0.82 (products), 0.88 (logistics), 0.45 (employees)], after sorting in descending order, the target sequence becomes [0.88, 0.82, 0.76, 0.45].

[0098] Next, the target sequence's differential values ​​are determined, and the absolute differential values ​​are determined based on these differential values. For example, the target sequence [0.88, 0.82, 0.76, 0.45] has differential values ​​of [-0.06, -0.06, -0.31], and the absolute differential values ​​are [0.06, 0.06, 0.31]. Furthermore, the target sequence's maximum differential value is determined based on the absolute differential values. For example, the maximum differential value in the absolute differential values ​​[0.06, 0.06, 0.31] is 0.31. The threshold interval containing the maximum differential value is determined within the target sequence. For example, when the maximum differential value is 0.31, the threshold interval is [0.46, 0.76]. Finally, the dynamic threshold corresponding to the target database table is determined based on the endpoint values ​​of the threshold interval: 0.76 × 0.99 = 0.75. 0.76 is the right endpoint of the threshold interval, and 0.99 is the correction factor, which adjusts the lower limit to include the edge table.

[0099] In this example, by calculating the difference values ​​after comprehensive weight sorting, we identify "mutation points" as thresholds. Compared to simple mean / median methods, this method is more capable of capturing structural changes in similarity relationships between tables (such as the turning point from strong to weak correlation). A sliding window is introduced to smooth the difference values, reducing the interference of random noise on the threshold and preventing threshold shifts caused by individual abnormal similarity values.

[0100] Based on the above embodiment, in some embodiments, after dividing each database table into a plurality of business domain groups according to each comprehensive weight, the method further includes: S181: traverse the dynamic thresholds corresponding to each database table to determine whether the dynamic threshold is greater than the first threshold; if it is confirmed that the dynamic threshold is greater than the first threshold, proceed to step S182; if it is confirmed that the dynamic threshold is not greater than the first threshold, proceed to step S183.

[0101] S182: Determine the business domain group where the corresponding database table is located as the first business domain group.

[0102] S183: Determine whether the dynamic threshold is greater than the second threshold; wherein the first threshold is greater than the second threshold; if it is confirmed that the dynamic threshold is greater than the second threshold, proceed to step S184; if it is confirmed that the dynamic threshold is not greater than the second threshold, proceed to step S185.

[0103] S184: Determine the business domain group where the corresponding database table is located as the second business domain group; wherein the importance of the first business domain group is greater than that of the second business domain group.

[0104] S185: Delete the business domain group where the corresponding database table is located.

[0105] To facilitate the business side's selection and use of grouped database tables, after completing the business domain grouping, the dynamic threshold corresponding to each database table can be traversed to determine whether the dynamic threshold is greater than the first threshold. If the dynamic threshold is greater than the first threshold, the business domain group containing the corresponding database table is determined as the first business domain group. If the dynamic threshold is not greater than the first threshold, the dynamic threshold is determined to be greater than the second threshold. If the dynamic threshold is greater than the second threshold, the business domain group containing the corresponding database table is determined as the second business domain group. If the dynamic threshold is not greater than the second threshold, the business domain group containing the corresponding database table is deleted.

[0106] It should be noted that in this embodiment, there are no restrictions on the first and second thresholds. For example, the first threshold can be 0.6 and the second threshold can be 0.3. It only needs to ensure that the first threshold is greater than the second threshold. Furthermore, the importance of the first business domain group is greater than that of the second business domain group. For example, the first business domain group is a core group, and the second business domain group is a peripheral group, indicating that the first business domain group is more important than the second business domain group. It should also be noted that when the dynamic threshold of a database table is not greater than the second threshold, it indicates that its correlation with other database tables is extremely low and it can be directly deleted to save storage space.

[0107] In this embodiment, based on the absolute size of the dynamic threshold, the grouped business domain groups are marked as differentiated levels such as core groups and peripheral groups, which is beneficial for the business party to select and use.

[0108] On this basis, to facilitate the rapid division of less important second business domain groups, in some embodiments, before determining the comprehensive weights corresponding to the database tables, after performing foreign key analysis and naming similarity calculations on the database tables to obtain semantic weights and behavioral weights, a rapid grouping threshold, such as 0.5, can be directly determined. When both the semantic weight and behavioral weight corresponding to a database table are greater than the rapid grouping threshold, it is directly assigned to the second business domain group and is not subsequently grouped with other database tables. This enables the rapid division of less important second business domain groups, greatly reducing the computational effort required for the business domain grouping process and significantly improving grouping efficiency.

[0109] Based on the above embodiment, in some embodiments, after dividing each database table into a plurality of business domain groups according to each comprehensive weight, the method further includes: S186: Determine the group size of each service domain group.

[0110] S187: Associate and store the dynamic threshold and group size corresponding to each service domain group.

[0111] To support subsequent tracing of business domain groupings, after grouping is complete, the group size of each business domain group can be further determined. The dynamic threshold and group size corresponding to each business domain group are then associated and stored. This allows for tracking the grouping results, such as "Why does a group include these tables?" This can be verified using the dynamic threshold and corresponding comprehensive weights, improving algorithm transparency.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0113] Figure 2 Schematic diagram of a device for dividing a database table into service domains according to an embodiment of the present invention. Figure 2 As shown, the device includes: An acquisition module 10 is used to acquire multiple database tables of the business domain to be divided in the database; The analysis module 11 is used to perform foreign key analysis, name similarity calculation, and co-occurrence frequency evaluation on each database table to obtain the structural weight, semantic weight, and behavioral weight corresponding to each database table; wherein the structural weight represents the degree of structural association between database tables with foreign key relationships; the semantic weight represents the degree of semantic association between database tables; and the behavioral weight represents the degree of behavioral association between database tables that co-occur in historical query instructions; Determination module 12, for determining the comprehensive weight corresponding to each database table based on the structural weight, semantic weight and behavioral weight corresponding to each database table; The grouping submodule 13 is used to divide each database table into multiple business domain groups according to each comprehensive weight.

[0114] In some embodiments, the analysis module 11 includes: The first parsing submodule is used to parse the data location language of the database and extract the foreign key relationships between the database tables; A first determination submodule is used to determine the parent table corresponding to each database table based on the corresponding foreign key relationship; The second determining submodule is used to determine the number of parent tables corresponding to each database table, so as to determine the structural weight between each database table and the corresponding parent table.

[0115] In some embodiments, the analysis module 11 includes: The third determining submodule is used to determine the table name of each database table, and determine the table name similarity between the database tables based on the table name of each database table; The fourth determination submodule is used to determine the field name of each database table and determine the field similarity between the database tables based on the field name of each database table; The fifth determining submodule is used to determine the semantic weights between the database tables according to the similarity of the table names, the similarity of the fields, the first secondary coefficient and the second secondary coefficient.

[0116] In some embodiments, the third determining submodule includes: A splitting submodule is used to split the table name of each database table into multiple continuous subsequences according to a preset character granularity; wherein the preset character granularity includes at least single character, double character and triple character; The first calculation submodule is used to calculate the cosine similarity between the continuous subsequence corresponding to each database table and the continuous subsequences corresponding to the other database tables, so as to determine the similarity between the table names of the database tables.

[0117] In some embodiments, the fourth determining submodule includes: The splicing submodule is used to splice the field names of each database table into corresponding strings; The sixth determination submodule is configured to perform vector embedding comparison on the character strings corresponding to each database table and the character strings corresponding to the remaining database tables based on the vector embedding function, so as to determine the field similarity between the database tables.

[0118] In some embodiments, the analysis module 11 includes: A first judgment submodule is configured to extract historical query instructions based on the database and determine whether a preset number of historical query instructions are obtained; if so, trigger the first extraction submodule; if not, trigger the first acquisition submodule; The first extraction submodule is used to extract the co-occurrence counts between the database tables in each historical query instruction; The seventh determination submodule is used to determine a quotient of each co-occurrence number and a preset number to determine the behavior weights between the database tables.

[0119] In some embodiments, it further includes: The first acquisition submodule is used to respectively acquire the data positioning language corresponding to each database table; The standardization processing module is used to perform data standardization processing on each data positioning language and eliminate common words to obtain the feature words corresponding to each database table; The second calculation submodule is used to calculate the vector similarity between the feature words corresponding to each database table based on the vector embedding function to determine the behavior weight between each database table.

[0120] In some embodiments, the determination module 12 includes: A coefficient determination submodule, configured to determine a first coefficient corresponding to the structural weight, a second coefficient corresponding to the semantic weight, and a third coefficient corresponding to the behavioral weight; The comprehensive weight determination submodule is used to determine the comprehensive weight corresponding to each database table according to the structural weight, semantic weight and behavioral weight corresponding to each database table, as well as the first coefficient, the second coefficient and the third coefficient.

[0121] In some embodiments, the grouping submodule 13 includes: The first selection submodule is used to select a target database table from various database tables; An eighth determination submodule is used to determine the comprehensive weight between the target database table and the remaining database tables; The threshold generation submodule is used to generate a dynamic threshold corresponding to the target database table according to the comprehensive weights corresponding to the target database table; The second judgment submodule is used to judge whether there is a target comprehensive weight that is not less than the dynamic threshold among the comprehensive weights corresponding to the target database table; if so, the first division submodule is triggered; The first division submodule is used to divide the database table corresponding to the target comprehensive weight and the target database table into the same business domain group; The second selection submodule is used to select a new target database table from the remaining database tables and trigger the eighth determination submodule until all database tables are grouped.

[0122] In some embodiments, the threshold generation submodule includes: The sorting module is used to sort the comprehensive weights corresponding to the target database table in descending order to generate a target sequence; A difference value determination module is used to determine the difference value of the target sequence and determine the absolute difference value based on the difference value; A maximum difference value determination module is used to determine the maximum difference value of the target sequence according to the absolute difference value; An interval determination module is used to determine the threshold interval where the maximum drop value is located in the target sequence; The dynamic threshold determination module is used to determine the dynamic threshold corresponding to the target database table according to the endpoint value of the threshold interval.

[0123] In some embodiments, it further includes: a third judgment submodule, configured to traverse the dynamic threshold values ​​corresponding to each database table and determine whether the dynamic threshold value is greater than the first threshold value; if it is determined that the dynamic threshold value is greater than the first threshold value, triggering the first service domain group determination submodule; if it is determined that the dynamic threshold value is not greater than the first threshold value, triggering the fourth judgment submodule; A first business domain group determination submodule, configured to determine the business domain group where the corresponding database table is located as the first business domain group; a fourth judgment submodule, configured to judge whether the dynamic threshold is greater than a second threshold; wherein the first threshold is greater than the second threshold; if it is confirmed that the dynamic threshold is greater than the second threshold, the second service domain group determination submodule is triggered; if it is confirmed that the dynamic threshold is not greater than the second threshold, the deletion module is triggered; A first business domain group determination submodule is configured to determine the business domain group where the corresponding database table is located as a second business domain group; wherein the importance of the first business domain group is greater than that of the second business domain group; The deletion module is used to delete the business domain group where the corresponding database table is located.

[0124] In some embodiments, it further includes: A group size determination module, used to determine the group size of each service domain group; The storage module is used to associate and store the dynamic threshold and group size corresponding to each business domain group.

[0125] For the description of the features in the embodiment corresponding to the device for dividing a database table into service domains, reference may be made to the relevant description of the embodiment corresponding to the method for dividing a database table into service domains, which will not be described in detail here.

[0126] An embodiment of the present invention further provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the method for dividing a database table into service domains.

[0127] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned database table business domain partitioning method embodiments when running.

[0128] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0129] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned embodiments of the method for dividing a database table into business domains are implemented.

[0130] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned database table business domain division method embodiments.

[0131] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0132] The above is a detailed introduction to the business domain division method, device, medium and product of a database table provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only suitable for helping to understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for dividing a database table into business domains, characterized in that: include: Obtain multiple database tables in the database for the business domain to be divided; Performing foreign key analysis, naming similarity calculation, and co-occurrence frequency evaluation on each of the database tables to obtain a structural weight, a semantic weight, and a behavioral weight corresponding to each of the database tables; wherein the structural weight represents the degree of structural association between the database tables having a foreign key relationship; the semantic weight represents the degree of semantic association between the database tables; and the behavioral weight represents the degree of behavioral association between the database tables that co-occur in historical query instructions; Determining a comprehensive weight corresponding to each of the database tables according to the structural weight, the semantic weight, and the behavioral weight corresponding to each of the database tables; Each of the database tables is divided into a plurality of business domain groups according to each of the comprehensive weights.

2. The method for dividing a database table into business domains according to claim 1, characterized in that: Performing foreign key analysis on each of the database tables, including: Parsing the data location language of the database to extract the foreign key relationships between the database tables; Determine the parent table corresponding to each of the database tables based on the corresponding foreign key relationships; The number of parent tables corresponding to each database table is determined to determine the structural weight between each database table and the corresponding parent table.

3. The method for dividing a database table into business domains according to claim 1, wherein: Performing naming similarity calculation on each of the database tables, including: Determining a table name of each of the database tables, and determining a table name similarity between the database tables based on the table names of the database tables; Determining the field names of the database tables, and determining the field similarities between the database tables based on the field names of the database tables; The semantic weights between the database tables are determined according to the similarities of the table names, the similarities of the fields, the first secondary coefficients, and the second secondary coefficients.

4. The method for dividing a database table into business domains according to claim 3, characterized in that: Determining the similarity between the table names of the database tables according to the table names of the database tables includes: Splitting the table name of each database table into multiple continuous subsequences according to a preset character granularity; wherein the preset character granularity includes at least single character, double character and triple character; The cosine similarities between the continuous subsequence corresponding to each database table and the continuous subsequences corresponding to the remaining database tables are calculated respectively to determine the table name similarities between the database tables.

5. The method for dividing a database table into business domains according to claim 3, characterized in that: Determining the field similarity between the database tables according to the field names of the database tables includes: Concatenate the field names of the database tables into corresponding character strings; Based on a vector embedding function, vector embedding comparison is performed on the character strings corresponding to each database table and the character strings corresponding to the remaining database tables to determine the field similarity between the database tables.

6. The method for dividing a database table into business domains according to claim 1, characterized in that: Performing a co-occurrence frequency evaluation on each of the database tables, including: Extracting historical query instructions based on the database, and determining whether a preset number of historical query instructions are obtained; If yes, extract the co-occurrence counts between the database tables in the historical query instructions; A quotient of each of the co-occurrence times and the preset number is determined to determine the behavior weights between the database tables.

7. The method for dividing a database table into business domains according to claim 6, characterized in that: If it is confirmed that the preset number of historical query instructions have not been obtained, the method further includes: respectively obtaining the data positioning language corresponding to each of the database tables; Performing data standardization processing on each of the data positioning languages ​​and eliminating common words to obtain feature words corresponding to each of the database tables; Based on the vector embedding function, the vector similarities between the feature words corresponding to the database tables are calculated respectively to determine the behavior weights between the database tables.

8. The method for dividing a database table into business domains according to claim 1, characterized in that: Determining the comprehensive weight corresponding to each database table according to the structural weight, the semantic weight, and the behavioral weight corresponding to each database table includes: Determining a first coefficient corresponding to the structural weight, a second coefficient corresponding to the semantic weight, and a third coefficient corresponding to the behavioral weight; The comprehensive weight corresponding to each of the database tables is determined according to the structural weight, the semantic weight, and the behavioral weight corresponding to each of the database tables, as well as the first coefficient, the second coefficient, and the third coefficient.

9. The method for dividing a database table into business domains according to any one of claims 1 to 8, characterized in that: Divide each of the database tables into a plurality of business domain groups according to the comprehensive weights, including: Selecting a target database table from the database tables; Determine the comprehensive weight between the target database table and the remaining database tables; generating a dynamic threshold corresponding to the target database table according to each of the comprehensive weights corresponding to the target database table; Determine whether there is a target comprehensive weight not less than the dynamic threshold among the comprehensive weights corresponding to the target database table; If yes, the database table corresponding to the target comprehensive weight and the target database table are divided into the same business domain group; A new target database table is selected from the remaining database tables, and the process returns to the step of determining the comprehensive weight between the target database table and the remaining database tables until all the database tables are grouped.

10. The method for dividing a database table into business domains according to claim 9, characterized in that: Generating a dynamic threshold corresponding to the target database table according to each comprehensive weight corresponding to the target database table includes: Sort the comprehensive weights corresponding to the target database table in descending order to generate a target sequence; determining a difference value of the target sequence, and determining an absolute difference value based on the difference value; Determine the maximum difference value of the target sequence according to the absolute difference value; Determine a threshold interval in the target sequence where the maximum drop value is located; The dynamic threshold corresponding to the target database table is determined according to the endpoint value of the threshold interval.

11. The method for dividing a database table into business domains according to claim 9, characterized in that: After dividing the database tables into a plurality of business domain groups according to the comprehensive weights, the method further includes: Traversing the dynamic threshold corresponding to each of the database tables, and determining whether the dynamic threshold is greater than a first threshold; If it is confirmed that the dynamic threshold is greater than the first threshold, determining the business domain group where the corresponding database table is located as the first business domain group; If it is confirmed that the dynamic threshold is not greater than the first threshold, determining whether the dynamic threshold is greater than a second threshold; wherein the first threshold is greater than the second threshold; If it is confirmed that the dynamic threshold is greater than the second threshold, the business domain group where the corresponding database table is located is determined as the second business domain group; wherein the importance of the first business domain group is greater than the importance of the second business domain group; If it is confirmed that the dynamic threshold is not greater than the second threshold, the business domain group where the corresponding database table is located is deleted.

12. The method for dividing a database table into business domains according to claim 9, characterized in that: After dividing the database tables into a plurality of business domain groups according to the comprehensive weights, the method further includes: determining a group size of each of the service domain groups; The dynamic threshold and the group size corresponding to each of the service domain groups are associated and stored.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for dividing a database table into business domains according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for dividing the business domain of a database table according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for dividing a database table into business domains are implemented as claimed in any one of claims 1 to 12.

Images