Database operation and maintenance knowledge graph construction method and system for optimizing entity alignment

Abstract

The invention provides a database operation and maintenance knowledge graph construction method and system for optimizing entity alignment, and the system comprises 1, a data preprocessing module which carries out the data cleaning of unstructured and semi-structured data; 2, a knowledge extraction module which extracts valuable information and knowledge from the cleaned data, including entities, relationships and attributes; the entity alignment module is used for aligning the same or similar entities in different data sources; and 4, a knowledge representation and storage module which performs standardized representation on the extracted and aligned knowledge and stores the knowledge in a knowledge base. According to the method, the context attention weight is introduced, and the semantic similarity between the words is calculated by calculating the attention weight of the context word vector of each word to the target word vector, so that the accuracy of similarity calculation is improved; and a hybrid editing distance algorithm is utilized to bring the semantic similarity among the words into an editing distance calculation process, so that the calculation method of the overall sentence similarity is optimized. Insertion, deletion and replacement operations of characters are considered, and word similarity calculation of a semantic level is also combined, so that sentence similarity calculation has a semantic understanding capability.

Description

Technical Field

[0001] The present invention relates to a method and system for constructing a database operation and maintenance knowledge graph for optimizing entity alignment, and belongs to the field of intelligent database operation and maintenance. Background Art

[0002] In the field of database operation and maintenance, the construction of knowledge graphs can effectively integrate and organize information from different sources, providing support for intelligent database management and maintenance. In the process of building a knowledge graph for database operation and maintenance, entity description is a very important attribute. Entity description contains rich semantic information, which can more accurately understand and distinguish different entities. However, most existing entity alignment methods focus on the similarity calculation of entity names, ignoring the consideration of entity descriptions, resulting in insufficient accuracy and reliability of entity alignment.

[0003] Although existing entity alignment methods have proposed some integrated similarity calculation methods, including string similarity and semantic similarity, they are independent calculation methods and cannot comprehensively consider string similarity and semantic similarity, and also have other shortcomings.

[0004] (1) Traditional edit distance algorithms, such as Levenshtein distance and Jaccard similarity, mainly consider character-level insertion, deletion, and substitution operations, ignoring the semantic relationship between words. These methods perform well in character matching tasks, but they are less effective in scenarios where semantic similarity needs to be measured and cannot effectively capture the semantic association between words. Existing edit distance algorithms fail to fully utilize the semantic information of words, which shows obvious limitations in entity alignment and similarity calculation.

[0005] (2) In existing research, semantic similarity matrices usually rely on the calculation of static word vectors, without fully considering the impact of contextual information on word meaning. For example, although word vector models such as Word2Vec and GloVe can capture the semantic similarity between words to a certain extent, their static representation cannot cope with the polysemy and semantic changes of words in different contexts. This leads to limited calculation accuracy of semantic similarity in practical applications, especially in the construction of database operation and maintenance knowledge graphs and entity alignment tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 This is a system structure diagram of the present invention. Summary of the invention

[0007] In view of the problems in the prior art, the present invention proposes a method for constructing a database operation and maintenance knowledge graph that optimizes entity alignment, and the specific steps include:

[0008] (1) Data preprocessing: cleaning of unstructured and semi-structured data;

[0009] (2) Knowledge extraction: extracting valuable information and knowledge from the cleaned data, including entities, relationships, and attributes;

[0010] (3) Entity alignment, aligning the same or similar entities in different data sources;

[0011] (4) Knowledge representation and storage: The extracted and aligned knowledge is represented in a standardized manner and stored in the knowledge base.

[0012] Furthermore, in step (3), the entity alignment stage includes entity integration similarity calculation and iterative representation learning, wherein the entity integration similarity is calculated by entity name string similarity, entity name semantic similarity, entity path similarity and entity description mixed similarity. When calculating entity description similarity, it includes:

[0013] 3.1 Calculate the attention weight of word segmentation str[i] Where v(str1[k]) and v(str1[e]) are word vectors; the context word vector set of word vector v(str1[i]) is {v(str1[1]),...,v(str1[i-1]),v(str1[i+1])...,v(str1[m])}; i and j are the word segmentation indexes of sentences str1 and str2, 1≤i≤m, 1≤j≤n, m and n are the number of word segmentations of sentences str1 and str2, respectively; k≠i, 1≤k≤m-1;

[0014] 3.2 Calculate the attention weight of the word str2[j] Among them, v(str2[l]) and v(str2[f]) are word vectors; l≠j, 1≤l≤n-1;

[0015] 3.3 Calculate the semantic similarity between str1[i] and str2[j]

[0016]

[0017] 3.4 Create a (m+1)×(n+1) matrix D, where the element D[i][j] of the matrix D represents the minimum mixed edit distance required to convert word str1[i] to word str2[j]; let D[0][0]=0, calculate D[0][j]=D[0][j-1]+sim(str1[0],str2[j-1]), D[i][0]=D[i-1][0]+sim(str1[i-1],str2[0]), where str1 and str2 are the two sentences to be compared;

[0018] 3.5 For each D[i][j], according to Fill in, and the final mixed edit distance is D[m][n];

[0019] 3.6 Calculating Entity Description Mixed Similarity

[0020] Furthermore, the entity name similarity and entity description similarity are weighted to obtain the integrated similarity, which is then iteratively aligned by the iterative representation learning module.

[0021] The present invention also proposes a database operation and maintenance knowledge graph construction system for optimizing entity alignment, which includes a data preprocessing module, a knowledge extraction module, an entity alignment module, and a knowledge representation and storage module, as follows:

[0022] (1) Data preprocessing module, which performs data cleaning on unstructured and semi-structured data;

[0023] (2) Knowledge extraction module, which extracts valuable information and knowledge from the cleaned data, including entities, relations, and attributes;

[0024] (3) Entity alignment module, which aligns the same or similar entities in different data sources;

[0025] (4) Knowledge representation and storage module: This module normalizes the extracted and aligned knowledge and stores it in the knowledge base.

[0026] Furthermore, in the entity alignment module, the entity alignment stage includes entity integration similarity calculation and iterative representation learning, wherein the entity integration similarity calculation is obtained by calculating the entity name string similarity, entity name semantic similarity, entity path similarity and entity description mixed similarity. When calculating the entity description similarity, it includes:

[0027] 3.1 Calculate the attention weight of word str1[i] Where v(str1[k]) and v(str1[e]) are word vectors; the context word vector set of word vector v(str1[i]) is {v(str1[1]),...,v(str1[i-1]),v(str1[i+1])...,v(str1[m])}; i and j are the word segmentation indexes of sentences str1 and str2, 1≤i≤m, 1≤j≤n, m and n are the number of word segmentations of sentences str1 and str2, respectively; k≠i, 1≤k≤m-1;

[0028] 3.2 Calculate the attention weight of the word str2[j] Among them, v(str2[l]) and v(str2[f]) are word vectors; l≠j, 1≤l≤n-1;

[0029] 3.3 Calculate the semantic similarity between str1[i] and str2[j]

[0030]

[0031] 3.4 Create a (m+1)×(n+1) matrix D, where the element D[i][j] of the matrix D represents the minimum mixed edit distance required to convert word str1[i] to word str2[j]; let D[0][0]=0, calculate D[0][j]=D[0][j-1]+sim(str1[0],str2[j-1]), D[i][0]=D[i-1][0]+sim(str1[i-1],str2[0]), where str1 and str2 are the two sentences to be compared;

[0032] 3.5 For each D[i][j], according to Fill in, and the final mixed edit distance is D[m][n];

[0033] 3.6 Calculating Entity Description Mixed Similarity

[0034] Furthermore, the entity name similarity and entity description similarity are weighted to obtain the integrated similarity, which is then iteratively aligned by the iterative representation learning module.

[0035] The present invention introduces contextual attention weights, calculates the attention weights of the context word vector of each word to the target word vector, calculates the semantic similarity between words, and improves the accuracy of similarity calculation; and uses a hybrid edit distance algorithm to incorporate the semantic similarity between words into the edit distance calculation process, optimizing the calculation method of the overall sentence similarity. Not only does it take into account the insertion, deletion and replacement operations of characters, but it also combines the word similarity calculation at the semantic level, making the sentence similarity calculation more semantically understandable. DETAILED DESCRIPTION

[0036] The present invention includes four parts: data preprocessing, knowledge extraction, entity alignment, and knowledge representation and storage. Figure 1 .

[0037] 1. Data preprocessing

[0038] In the field of database operation and maintenance, data preprocessing is a basic step in building a knowledge graph. First, it is necessary to collect raw data from operation and maintenance logs, monitoring data, configuration files, and documents. The raw data contains a lot of noise and redundant information, such as duplicate log entries and irrelevant system messages, which need to be cleaned. During the standardization process, unified naming rules (such as unified database and table names), time formats, and standardized units of performance indicators (such as MB / s, IOPS, etc.) are formulated to ensure data consistency and availability.

[0039] 2. Knowledge Extraction

[0040] In the knowledge extraction stage, the main task is to extract valuable operation and maintenance knowledge from the preprocessed data. Named entity recognition (NER) is used to identify key entities in the field of database operation and maintenance, including database instances, tables, indexes, configuration parameters, performance indicators, etc. Relationship extraction is used to determine the relationship between entities, such as "which tables a database instance contains", "the relationship between indexes and tables", "the impact of configuration parameters on performance indicators", etc. The goal of attribute extraction is to extract the attribute information of entities from text to provide a more detailed and specific description. The attributes of entities can help better understand and optimize database performance.

[0041] 3. Entity Alignment

[0042] In order to solve the data redundancy and heterogeneity problems in entity extraction, after completing data cleaning and entity, relationship and attribute extraction, knowledge alignment and fusion are required to ensure data accuracy and consistency. Different data sources may use different names and formats to represent the same entity. For example, "DB1" and "Database1" may point to the same database instance. The specific steps are as follows:

[0043] (1) Entity integration similarity calculation

[0044] The main task at this stage is to integrate multiple traditional entity alignment models and construct an integrated similarity by calculating the similarity of various features of entities, including entity name string similarity, entity name semantic similarity, entity path similarity, and entity description similarity. Among them, since entity descriptions are generally longer sentences than entity names, a hybrid similarity calculation method is proposed for entity descriptions to achieve hybrid similarity calculation of strings and semantics. The specific steps are as follows:

[0045] 1) Define the word segmentation semantic similarity matrix: First, define a word segmentation semantic similarity matrix, in which each element sim(str1[i],str2[j]) represents the similarity between word segmentations str1[i] and str2[j], where i and j represent the word segmentation indexes of sentences str1 and str2, respectively, 1≤i≤m, 1≤j≤n, and m and n represent the number of word segmentations in sentences str1 and str2, respectively.

[0046] Before calculating the similarity, first calculate the attention weight of each context word vector to the target word vector to obtain the weighted word vector similarity. The specific steps are as follows:

[0047] a) Assume that the context word vector set of word vector v(str1[i]) is {v(str1[1]),...,v(str1[i-1]),v(str1[i+1])...,v(str1[m])}, calculate the attention weight of str1[i]

[0048] k≠i and 1≤k≤m-1;

[0049] b) Similarly, the attention weight of str2[j] can be calculated as

[0050] l≠j and 1≤l≤n-1;

[0051] c) Finally, calculate the semantic similarity between str1[i] and str2[j]

[0052]

[0053] The above method uses the weight of the context word vector to calculate the similarity of the target word vector. It can dynamically adjust the weight of each word according to the context information, calculate a more accurate weighted word vector similarity, improve the accuracy of the similarity calculation, and better capture the subtle semantic differences between words.

[0054] 2) Initialize the matrix: Create a (m+1)×(n+1) matrix D, where the element D[i][j] of the matrix D represents the minimum mixed edit distance required to transform the word str1[i] into the word str2[j].

[0055] 3) Fill in initial values: Initialize the matrix D as follows:

[0056] a) D[0][0] = 0, indicating that the mixed edit distance of two sentences of length 0 is 0;

[0057] b) For the first row D[0][j] and the first column D[i][0], use the accumulation method to initialize: D[0][j] = D[0][j-1] + sim(str1[0], str2[j-1]), D[i][0] = D[i-1][0] + sim(str1[i-1], str2[0]), where str1 and str2 are the two sentences to be compared.

[0058] 4) Fill the hybrid edit distance matrix: For each D[i][j], fill it according to the following formula:

[0059]

[0060] 5) Final mixed edit distance: The final mixed edit distance is D[m][n], which represents the minimum cost of converting from sentence str1 to sentence str2. This cost takes into account the similarity between words, not just the insertion, deletion, and replacement operations of characters.

[0061] 6) Based on the new hybrid edit distance calculation method, the hybrid similarity of entity description is calculated:

[0062]

[0063] The entity name similarity and entity description similarity are weighted to output the integrated similarity S, which is then iteratively aligned by the iterative representation learning module.

[0064] (2) Iterative Representation Learning

[0065] After building the integrated similarity matrix, the entity alignment module enters the iterative representation learning phase. The main task of this phase is to gradually improve the quality of entity vector representation through iterative optimization, thereby improving the accuracy of entity alignment. The specific process is as follows:

[0066] 1) Generate an initial set of aligned entities based on the integrated similarity S;

[0067] 2) Input the initial aligned entity set into the graph convolutional neural network for training;

[0068] 3) Continuously generate and update aligned entity pairs through a step-by-step learning strategy.

[0069] 4. Knowledge representation and storage

[0070] In the knowledge representation and storage phase, the aligned knowledge is stored in a structured manner in the Neo4j graph database. Each entity (such as database instance, table, index, configuration parameter, etc.) is a node, and the relationship between entities (such as containment relationship, influence relationship, etc.) is an edge. Each entity node contains its detailed description information and some configuration and other related attributes. Cypher is used to implement complex query and reasoning functions, supporting operation and maintenance personnel to efficiently obtain the required information and knowledge, as well as efficient query and fault diagnosis.

[0071] A more specific embodiment:

[0072] Assume that m=6, n=5, and the word segmentation semantic similarity matrix is ​​as follows:

[0073]

[0074] Initialize the matrix D and fill it with initial values:

[0075]

[0076] Follow the above steps to fill the hybrid edit distance matrix:

[0077]

[0078] The final mixed edit distance is D[6][5]=3.0.

[0079] Calculate the entity description mixed similarity as

[0080] The units, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions divided into various modules. Of course, when implementing this application, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0082] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0083] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0084] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0085] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for constructing a database operation and maintenance knowledge graph for optimizing entity alignment, the specific steps comprising: (1) Data preprocessing: cleaning of unstructured and semi-structured data; (2) Knowledge extraction: extracting valuable information and knowledge from the cleaned data, including entities, relationships, and attributes; (3) Entity alignment, aligning the same or similar entities in different data sources; (4) Knowledge representation and storage: the extracted and aligned knowledge is represented in a standardized manner and stored in a knowledge base; its characteristics are: Entity alignment includes entity ensemble similarity calculation and iterative representation learning, where the entity ensemble similarity is calculated by entity name string similarity, entity name semantic similarity, entity path similarity and entity description mixed similarity.

2. A method for constructing a database operation and maintenance knowledge graph for optimizing entity alignment according to claim 1, characterized in that: In step (3), when calculating the entity description similarity, it includes: 3.1 Calculate the attention weight of word str1[i] Where v(str1[k]) and v(str1[e]) are word vectors; the context word vector set of word vector v(str1[i]) is {v(str1[1]),...,v(str1[i-1]),v(str1[i+1])...,v(str1[m])}; i and j are the word segmentation indexes of sentences str1 and str2, 1≤i≤m, 1≤j≤n, m and n are the number of word segmentations of sentences str1 and str2, respectively; k≠i, 1≤k≤m-1; 3.2 Calculate the attention weight of the word str2[j] Among them, v(str2[l]) and v(str2[f]) are word vectors; l≠j, 1≤l≤n-1; 3.3 Calculate the semantic similarity between str1[i] and str2[j] 3.4 Create a (m+1)×(n+1) matrix D, where the element D[i][j] of the matrix D represents the minimum mixed edit distance required to convert word str1[i] to word str2[j]; let D[0][0]=0, calculate D[0][j]=D[0][j-1]+sim(str1[0],str2[j-1]), D[i][0]=D[i-1][0]+sim(str1[i-1],str2[0]), where str1 and str2 are the two sentences to be compared; 3.5 For each D[i][j], according to Fill in, and the final mixed edit distance is D[m][n]; 3.6 Calculating Entity Description Mixed Similarity 3. The method for constructing a database operation and maintenance knowledge graph for optimizing entity alignment according to claim 1, characterized in that: The integrated similarity is obtained by weighting the entity name similarity and the entity description similarity, and then iteratively aligned by the iterative representation learning module.

4. A database operation and maintenance knowledge graph construction system for optimizing entity alignment, the system includes a data preprocessing module, a knowledge extraction module, an entity alignment module and a knowledge representation and storage module, specifically including: (1) Data preprocessing module, which performs data cleaning on unstructured and semi-structured data; (2) Knowledge extraction module, which extracts valuable information and knowledge from the cleaned data, including entities, relations, and attributes; (3) Entity alignment module, which aligns the same or similar entities in different data sources; (4) Knowledge representation and storage module, which normalizes the extracted and aligned knowledge and stores it in the knowledge base; its characteristics are: In the entity alignment module, entity alignment includes entity integration similarity calculation and iterative representation learning, where the entity integration similarity calculation is obtained by calculating the entity name string similarity, entity name semantic similarity, entity path similarity and entity description mixed similarity.

5. A database operation and maintenance knowledge graph construction system for optimizing entity alignment as claimed in claim 4, characterized in that: When calculating entity description similarity, it includes: 3.1 Calculate the attention weight of word str1[i] Where v(str1[k]) and v(str1[e]) are word vectors; the context word vector set of word vector v(str1[i]) is {v(str1[1]),...,v(str1[i-1]),v(str1[i+1])...,v(str1[m])}; i and j are the word segmentation indexes of sentences str1 and str2, 1≤i≤m, 1≤j≤n, m and n are the number of word segmentations of sentences str1 and str2, respectively; k≠i, 1≤k≤m-1; 3.2 Calculate the attention weight of the word str2[j] Among them, v(str2[l]) and v(str2[f]) are word vectors; l≠j, 1≤l≤n-1; 3.3 Calculate the semantic similarity between str1[i] and str2[j] 3.4 Create a (m+1)×(n+1) matrix D, where the element D[i][j] of the matrix D represents the minimum mixed edit distance required to convert word str1[i] to word str2[j]; let D[0][0]=0, calculate D[0][j]=D[0][j-1]+sim(str1[0],str2[j-1]), D[i][0]=D[i-1][0]+sim(str1[i-1],str2[0]), where str1 and str2 are the two sentences to be compared; 3.5 For each D[i][j], according to Fill in, and the final mixed edit distance is D[m][n]; 3.6 Calculating Entity Description Mixed Similarity 6. A database operation and maintenance knowledge graph construction system for optimizing entity alignment as claimed in claim 4, characterized in that: The integrated similarity is obtained by weighting the entity name similarity and the entity description similarity, and then iteratively aligned by the iterative representation learning module.

Images