A database metadata relationship knowledge graph generation method

Abstract

The application discloses a database metadata relationship knowledge graph generation method, comprising the following steps: forming a historical query metadata knowledge graph based on historical SQL query statements of a database; forming a data association metadata knowledge graph based on complete value matching between data columns; forming a database schema metadata knowledge graph based on schema information of the database; aligning triple relationship formed by the same entity in each knowledge graph based on the three obtained knowledge graphs, and collecting the triple relationship into a complete database metadata relationship knowledge graph; and reasoning the obtained complete knowledge graph by using a representation-based knowledge reasoning model, complementing the relationship in the knowledge graph, and further improving the database metadata relationship knowledge graph. The application constructs a new metadata relationship knowledge graph, and complements the relationship in the knowledge graph based on a knowledge reasoning method, thereby forming effective support for data exploration of the database and enhancing the effect of data exploration and data analysis.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graphs, and in particular to a method for generating a database metadata relationship knowledge graph. Background Art

[0002] A knowledge graph, also known as a knowledge domain visualization or knowledge domain mapping map in the library and information science community, is a series of diverse graphs that display the development process and structural relationships of knowledge. It uses visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, map, and display knowledge and its interrelationships. Specifically, a knowledge graph is a modern theory that achieves multidisciplinary integration by combining theories and methods from disciplines such as applied mathematics, graphics, information visualization, and information science with methods such as citation analysis and co-occurrence analysis. It uses visual graphs to vividly display the core structure, development history, cutting-edge areas, and overall knowledge architecture of a discipline. It visualizes complex knowledge domains through data mining, information processing, knowledge metrics, and graphing, revealing the dynamic development patterns of knowledge domains and providing practical and valuable references for disciplinary research. While its practical application has gradually expanded in developed countries and achieved promising results, research in my country is still in its infancy. A knowledge graph consists of entities, relations, and triples describing the corresponding relationships between entities and relations.

[0003] Metadata, or data describing data structure and attributes, is a formatted way of describing information to facilitate quick understanding and processing. It describes several data attributes and multi-dimensional characteristics, such as data structure, data storage location, and data flow. Metadata in traditional relational databases typically includes table field information, primary and foreign key information, access permissions, indexes, and data block distribution information.

[0004] However, the data semantics contained in traditional relational database metadata is incomplete. For example, information between fields that have data associations but no primary or foreign key relationships is missing. This lack of data semantics means that traditional metadata lacks the ability to express information, and therefore traditional relational databases can no longer meet analytical needs. In other words, the quality of data exploration results from relational databases based on traditional metadata is limited by the metadata, and poor metadata semantics lowers the performance ceiling of data exploration algorithms. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology and use knowledge reasoning technology to reason and complete the knowledge graph of metadata. It can fully explore the potential data semantics in the existing metadata knowledge and complete the metadata knowledge graph, thereby obtaining richer and more complete data semantics, and further provide effective support for the performance improvement of subsequent relational database data exploration algorithms; thereby providing a method for generating database metadata relationship knowledge graphs.

[0006] The object of the present invention is achieved through the following technical solutions:

[0007] A method for generating a database metadata relationship knowledge graph comprises the following steps:

[0008] Step 1: Generate a historical query metadata knowledge graph based on the historical SQL query statements in the database;

[0009] Step 2: Form a data-related metadata knowledge graph based on the complete matching of values ​​between data columns;

[0010] Step 3: Form a database schema metadata knowledge graph based on the database schema information;

[0011] Step 4: Based on the three knowledge graphs obtained in steps 1, 2, and 3, align the triples of the same entities in each knowledge graph and summarize them into a complete database metadata relationship knowledge graph;

[0012] Step 5: Use the representation-based knowledge reasoning model to reason about the complete knowledge graph obtained in step 4, complete the relationships in the graph, and further improve the database metadata relationship knowledge graph.

[0013] Furthermore, the schema information of the database is definition information of tables and columns.

[0014] Furthermore, the step 1 is specifically as follows: based on the historical SQL query statement of the database, a join T b on T a .C x =T b .C y and select…from T a ,T b where T a .C x =T b .C y Query statements, construct the historical query relationship triples between tables (T a ,history_table_match,T b) and the inter-column historical query relationship triples (T a .C x ,history_column_match,T b .C y ), forming a historical query metadata knowledge graph.

[0015] Furthermore, the step 2 is specifically as follows:

[0016] Based on the complete matching of values ​​between data columns, for each data column T a .C x , compare it with all columns T in other tables in the database b .C y Fill in the following SQL template for query, select count(*)from T a ,T b whereT a .C x =T b .C y

[0017] When the query result exceeds the threshold, the constructed form is (T a .C x , column_value_match, T b .C y ) to form a data association metadata knowledge graph.

[0018] Furthermore, the step 3 is specifically as follows: based on the database schema information, define the relationship between tables and columns, as shown in the following table, to form a database schema metadata knowledge graph.

[0019]

[0020]

[0021] Furthermore, the step 4 includes the following sub-steps:

[0022] Step 401: For a triple of the historical query metadata knowledge graph, if there is a historical matching relationship R between two column entities h , then R h Replace the triple relationship between the two column entities in the data association graph and the database schema graph;

[0023] Step 402: For the triple of the historical query metadata knowledge graph, if there is a historical query relationship R between the two table entities e , then R e Replace the triple relationship between the two table entities in the database schema diagram;

[0024] Step 403: For a triple in the data association graph, if there is a data association relationship R between two column entities v , and there is no relationship between these two column entities in the historical query graph, then R v Replace the relationship between these two column entities in the database schema diagram;

[0025] Step 404: The three knowledge graphs aligned according to steps 401, 402 and 403 are aggregated to form a complete database metadata relationship knowledge graph.

[0026] Furthermore, step 5 includes the following sub-steps:

[0027] Step 501: The triples in the complete database metadata relational knowledge graph are divided into a training set and a test set in a ratio of 9:1. The training set data is used to train a representation-based knowledge reasoning model TransE, and the test set is used to verify the training effect and determine when to stop training.

[0028] Step 502: Use the trained TransE to perform knowledge reasoning on the complete database metadata relationship knowledge graph, complete the relationships in the graph, and form a more complete database metadata relationship knowledge graph.

[0029] Beneficial effects of the present invention: In response to the problem that the traditional database metadata relationship knowledge graph is not perfect, the present invention constructs a new metadata relationship knowledge graph based on historical query relationships, data association relationships, and database schema information, and completes the relationships in the graph based on knowledge reasoning methods, thereby forming a more complete database metadata relationship knowledge graph, which effectively supports database data exploration and enhances the effects of data exploration and data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0031] Figure 1 It is a flow chart of the method of the present invention.

[0032] Figure 2 This is a schematic diagram of the historical query metadata knowledge graph.

[0033] Figure 3 It is a schematic diagram of the data association metadata knowledge graph.

[0034] Figure 4 It is a schematic diagram of the database schema metadata knowledge graph.

[0035] Figure 5 It is a flowchart for using knowledge reasoning model to complete the knowledge graph. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] In this embodiment, if Figure 1 As shown, a method for generating a database metadata relationship knowledge graph includes the following steps:

[0039] Step 1: Generate a historical query metadata knowledge graph based on the database's historical SQL query statements; generate a data association metadata knowledge graph based on the complete match of values ​​between data columns; and generate a database schema metadata knowledge graph based on the database schema (i.e., table and column definitions);

[0040] Step 2: Based on the three knowledge graphs obtained in step 1, align the triples of identical entities and summarize them into a complete database metadata relationship knowledge graph.

[0041] Step 3: Use the representation-based knowledge reasoning model to reason about the complete graph obtained in step 2, complete the relationships in the graph, and form a more complete database metadata relationship knowledge graph.

[0042] Wherein, the step 1 comprises the following steps:

[0043] Step 101: Based on the historical SQL query statements of the database, for the form select...from T a join T b onT a .C x =T b .C y and select…from T a ,T b where T a .C x=T b .C y SQL query statement (where T represents table and C represents column), construct the form (T a ,history_table_match,T b ) and the historical query relationship triples between tables of the form (T a .C x ,history_column_match,T b .C y ) to form a historical query metadata knowledge graph, such as Figure 2 As shown;

[0044] Step 102: Based on the complete matching of values ​​between data columns, that is, for each data column T a .C x , compare it with all columns T in other tables in the database b .C y Fill in the following SQL template for query,

[0045] select count(*)from T a ,T b where T a .C x =T b .C y

[0046] If the query result exceeds a certain threshold, construct a a .C x , column_value_match, T b .C y ) triples to form a data-related metadata knowledge graph, such as Figure 3 As shown;

[0047] Step 103: Based on the database schema (i.e., table and column definitions), define the relationship between tables and columns, as shown in Table 1 below, to form a database schema metadata knowledge graph, as shown in Table 1 below. Figure 4 shown.

[0048] Table 1 Relationship between tables and columns in the database

[0049]

[0050] Wherein, the step 2 comprises the following steps:

[0051] Step 201: For the triples in the historical query graph, that is, if there is a historical matching relationship R between two column entities h , then Rh Replace the triple relationship between the two column entities in the data association graph and the database schema graph;

[0052] Step 202: For the triples in the historical query graph, that is, if there is a historical query relationship R between two table entities e , then R e Replace the triple relationship between the two table entities in the database schema diagram;

[0053] Step 203: For the triples in the data association graph, that is, there is a data association relationship R between two column entities v , and there is no relationship between these two column entities in the historical query graph, then R v Replace the relationship between these two column entities in the database schema diagram;

[0054] Step 204: The three knowledge graphs aligned according to the above three steps are aggregated to form a complete database metadata relationship knowledge graph.

[0055] In one embodiment, step 3 comprises the following steps:

[0056] Step 301: The triples in the complete database metadata relational knowledge graph are divided into a training set and a test set in a ratio of 9:1. The training set data is used to train a representation-based knowledge reasoning model TransE, and the test set is used to verify the training effect to determine when to stop training. Figure 5 As shown; if the model performance is unqualified, then retrain a representation-based knowledge reasoning model TransE with the training set data, and verify the training effect with the test set; until the training effect is verified with the test set and the model performance is qualified, then proceed to step 302;

[0057] Step 302: Use the trained TransE to perform knowledge reasoning on the complete database metadata relationship knowledge graph, complete the relationships in the graph, and form a more complete database metadata relationship knowledge graph.

[0058] The knowledge graph consists of entities, relations, and triples describing the correspondence between entities and relations. It has the following main features:

[0059] 1. The more times users search and the wider the scope, the more information and content the search engine can obtain.

[0060] 2. Give the string new meaning, not just a simple string of words.

[0061] 3. It integrates all disciplines to facilitate coherence when users search.

[0062] 4. Find more accurate information for users, make more comprehensive summaries and provide more in-depth and relevant information.

[0063] 5. Systematically display the knowledge system related to keywords to users.

[0064] 6. Draw useful information from the entire Internet so that users can obtain more relevant public resources.

[0065] Knowledge is systematically extracted and organized within the knowledge graph, laying a solid foundation for efficient knowledge management. Knowledge reasoning based on the knowledge graph can infer new knowledge and correct erroneous knowledge based on existing facts in the knowledge graph. This, in other words, complements and de-noises the knowledge graph, alleviating algorithmic performance limitations caused by incomplete knowledge.

[0066] Using knowledge reasoning technology to reason and complete the knowledge graph of metadata can fully explore the potential data semantics in existing metadata knowledge and complete the metadata knowledge graph, thereby obtaining richer and more complete data semantics, and further providing effective support for the subsequent performance improvement of the data exploration algorithm of the relational database.

[0067] The present invention uses knowledge graph technology to solve the problem that the data semantics contained in the metadata of traditional relational databases are not yet perfect, for example, the information between fields that have data association but no primary and foreign key relationship is missing; the lack of data semantics means that the information expression ability of traditional metadata is not strong enough, that is, the quality of data exploration results of relational databases based on traditional metadata will be limited by metadata, and the poor data semantic quality of metadata will reduce the performance upper limit of data exploration algorithms; and in response to the problem that the traditional database metadata relationship knowledge graph is not perfect, a new metadata relationship knowledge graph based on historical query relationships, data association relationships, and database model information is constructed, and the relationships in the graph are completed based on knowledge reasoning methods, thereby forming a more complete database metadata relationship knowledge graph, which effectively supports database data exploration and enhances the effects of data exploration and data analysis.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0069] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0070] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0071] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention. It should be noted that similar reference numerals and letters denote similar items in the following figures. Therefore, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

[0072] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection set forth in the claims.

Claims

1. A method for generating a database metadata relationship knowledge graph, characterized in that: The following steps are involved: Step 1: Based on the historical SQL query statements of the database, a historical query metadata knowledge graph is formed; Step 1 specifically includes: Based on the historical SQL query statements of the database, the query metadata of the form select ... from T a join T b on T a .C x = T b .C y and select ... from T a , T b where T a . C x =T b . C y Query statements, construct the historical query relationship triples between tables (T a ,history_table_match,T b ) and the inter-column historical query relationship triples (T a . C x ,history_column_match,T b . C y ), forming a historical query metadata knowledge graph; Step 2: Based on the complete matching of values ​​between data columns, a data-related metadata knowledge graph is formed; Step 2 is specifically as follows: Based on the complete matching of values ​​between data columns, for each data column T a . C x , compare it with all columns T in other tables in the database b . C y Fill in the following SQL template for query, select count(*) from T a ,T b where T a . C x =T b . C y When the query result exceeds the threshold, the constructed form is (T a . C x , column_value_match, T b . C y ) triples to form a data-related metadata knowledge graph; Step 3: Form a database schema metadata knowledge graph based on the database schema information; Step 3 specifically includes: defining the relationship between tables and columns based on the database schema information to form a database schema metadata knowledge graph; Step 4: Based on the three knowledge graphs obtained in steps 1, 2, and 3, align the triples of identical entities in each knowledge graph and aggregate them into a complete database metadata relationship knowledge graph. Step 4 includes the following sub-steps: Step 401: For a triple of the historical query metadata knowledge graph, if there is a historical matching relationship R between two column entities h , then R h Replace the triple relationship between the two column entities in the data association metadata knowledge graph and the database schema metadata knowledge graph; Step 402: For the triple of the historical query metadata knowledge graph, if there is a historical query relationship R between the two table entities e , then R e Replace the triple relationship between the two table entities in the database schema metadata knowledge graph; Step 403: For a triple in the data association metadata knowledge graph, if there is a data association relationship R between two column entities v , and there is no relationship between these two column entities in the historical query metadata knowledge graph, then R v Replace the relationship between these two column entities in the database schema metadata knowledge graph; Step 404: The three knowledge graphs aligned in steps 401, 402, and 403 are aggregated to form a complete database metadata relationship knowledge graph; Step 5: Use the representation-based knowledge reasoning model to reason about the complete knowledge graph obtained in step 4, complete the relationships in the graph, and further improve the database metadata relationship knowledge graph.

2. A method for generating a database metadata relationship knowledge graph according to claim 1, characterized in that: The schema information of the database is definition information of tables and columns.

3. A method for generating a database metadata relationship knowledge graph according to claim 1, characterized in that: The step 5 includes the following sub-steps: Step 501: The triples in the complete database metadata relational knowledge graph are divided into a training set and a test set in a ratio of 9:

1. The training set data is used to train a representation-based knowledge reasoning model TransE, and the test set is used to verify the training effect and determine when to stop training. Step 502: Use the trained TransE to perform knowledge reasoning on the complete database metadata relationship knowledge graph, complete the relationships in the graph, and form a more complete database metadata relationship knowledge graph.

Images