A method for constructing knowledge graph based on large language model and vector library

Abstract

The present application discloses a method for constructing a knowledge graph based on a large language model and a vector library, which relates to the field of knowledge graph technology, including: using a pre-trained RoBERTa model to perform word segmentation, part-of-speech tagging and semantic analysis on the collected natural language text to obtain a semantic representation vector of the natural language text; obtaining the association between the semantic representation vector and the corresponding named entity through the Seq2Seq framework and the attention mechanism, and obtaining the named entity through the obtained association; constructing a knowledge graph; fusing the constructed knowledge graph with a relational database of a preset domain to generate a fused knowledge graph; using dependency syntactic analysis to obtain the named entity and the corresponding attribute, obtaining the entity relationship of the named entity, and constructing a structured triple of the named entity, attribute and entity relationship of the fused knowledge graph. In view of the difficulty of entity linking in the prior art, the present application converts the knowledge graph into a low-dimensional dense vector representation, and constructs an inverted index library, etc., to reduce the difficulty.

Description

Technical Field

[0001] The present application relates to the field of knowledge graph technology, and in particular to a method for constructing a knowledge graph based on a large language model and a vector library. Background Art

[0002] With the rapid development of technologies such as artificial intelligence and knowledge engineering, knowledge graphs have become a key resource for structured representation of massive knowledge and support for intelligent applications. Knowledge graphs form a comprehensive, accurate, and efficient knowledge base by performing semantic modeling and association mining on knowledge elements such as entities, attributes, and relationships, providing powerful knowledge support and service capabilities for scenarios such as intelligent question answering, semantic search, relational reasoning, and decision support.

[0003] However, building high-quality, large-scale knowledge graphs faces many technical challenges. Among them, entity linking is a key task to map named entities in unstructured text to existing entities in the knowledge base, which directly affects the coverage and accuracy of the knowledge graph. Traditional entity linking methods mainly rely on shallow features such as string matching and statistical co-occurrence, which are difficult to effectively handle complex semantic phenomena such as homonymy and reference disambiguation, resulting in low link accuracy. In addition, the construction of existing knowledge graphs is mostly limited to a single data source, lacking a fusion mechanism for multi-source heterogeneous data, which limits the breadth and depth of knowledge. Summary of the invention

[0004] In response to the problem of difficulty in entity linking in the prior art, the present application provides a method for constructing a knowledge graph based on a large language model and a vector library, constructing a fused knowledge graph through multi-source heterogeneous data, converting the knowledge graph into a low-dimensional dense vector representation, and constructing an inverted index library, thereby reducing the difficulty of entity linking.

[0005] The purpose of this application is achieved through the following technical solutions.

[0006] The present application provides a method for constructing a knowledge graph based on a large language model and a vector library, including: step 1, using a pre-trained RoBERTa model to perform word segmentation, part-of-speech tagging and semantic analysis on the collected natural language text data to obtain semantic representation vector data of the natural language text; inputting the semantic representation vector data into a Seq2Seq framework of a BiLSTM-Attention-CRF network structure, learning the context information features of the input vector through a BiLSTM encoding layer, adjusting the BiLSTM encoding features through an attention mechanism layer, and obtaining named entity recognition result data according to the adjusted feature vector decision by a CRF decoding layer; step 2, according to step Step 1: Identify named entities and construct initial knowledge graph data: use the identified named entities as entity nodes of the knowledge graph; set the type and attributes of the entity nodes according to the predefined Schema; use the pre-trained syntactic dependency analysis model to obtain syntactic dependency tree data; in the syntactic dependency tree, obtain the shortest dependency path between named entity pairs; build a relationship template based on the obtained shortest dependency path; use the constructed relationship template to extract relationships from named entity pairs to obtain entity relationship data; use the extracted entity relationships as the relationship edges of the knowledge graph, connect the corresponding head entity node and tail entity node, and construct knowledge graph data containing entity nodes, attributes and relationship edges.

[0007] Step 3. Fuse the initial knowledge graph with the structured database: select a structured relational database (such as DBpedia) related to the knowledge graph construction task as a supplementary knowledge source; fuse the knowledge graph constructed in step 2 with the relational database in the preset field to generate fused knowledge graph data: use a string matching algorithm to obtain pairs of entities with the same name and the same attributes in the knowledge graph and the relational database; for each pair of entities with the same name, calculate the semantic similarity of the corresponding attributes; when the semantic similarity is higher than the preset threshold, add the attributes of the corresponding entity in the relational database to the corresponding entity node in the knowledge graph; when the semantic similarity is lower than the preset threshold, rename the corresponding entity node in the knowledge graph or add a distinguishing mark.

[0008] Preferably, when using the Levenshtein distance algorithm to calculate the edit distance between entity name strings, a synonym dictionary and semantic similarity calculation are introduced to improve the accuracy of entity alignment and fusion, including: constructing a domain-specific synonym dictionary to characterize the equivalent relationship between synonymous entity name strings: using domain dictionaries, knowledge bases and other resources to extract synonymous entity names in a specific domain; mapping the synonymous entity names to a unified normalized representation, and constructing a domain-specific synonym dictionary; each entry in the synonym dictionary consists of a group of synonymous entity name strings, indicating that they are semantically equivalent and refer to the same real-world entity; when calculating the edit distance between entity name strings, a synonym dictionary matching is introduced: for two entity name strings whose edit distance is to be calculated, first query in the synonym dictionary whether they belong to the same synonym entry; if the two entity name strings belong to the same synonym entry, their edit distance is set to 0, indicating that they are semantically completely equivalent; if the two entity name strings do not belong to the same synonym entry, their edit distance is calculated using the Levenshtein distance algorithm; by introducing a synonym dictionary Matching, aligning semantically equivalent entity name strings to improve the recall rate of entity alignment; when calculating the edit distance between entity name strings, introduce semantic similarity calculation: use pre-trained word embedding models, such as Word2Vec or Glove, to obtain the word vector representation of each word in the entity name string; calculate the semantic similarity of the entity name string through the cosine similarity of the word vector as a supplement to the edit distance; on the basis of the Levenshtein distance, add the semantic similarity score to obtain the entity name string similarity measure that comprehensively considers the literal similarity and semantic similarity; by introducing semantic similarity calculation, capture the entity name strings with different literals but similar semantics, and improve the accuracy of entity alignment; based on the entity name string similarity that integrates the synonym dictionary matching and semantic similarity calculation, align and fuse the entity nodes: according to the preset similarity threshold, select the entity name string pairs with similarity higher than the threshold, align the corresponding entity nodes, and regard them as referring to the same real-world entity; for the aligned entity nodes, fuse their attribute information and relationship edges to retain more complete and accurate attribute values ​​and relationship descriptions.

[0009] Step 4. Perform structured triple representation on the fused knowledge graph: Use dependency syntactic analysis to obtain the named entities, corresponding attributes, and entity relationship data between named entities in the fused knowledge graph; construct structured triple representation data of the fused knowledge graph based on the acquired named entities, attributes, and entity relationship data. Preferably, perform dependency syntactic analysis on the text data in the fused knowledge graph to identify the core words, modifiers, and relational words in the sentence, and construct a dependency syntactic tree for the sentence; identify noun nodes in the dependency syntactic tree as named entities; identify adjectives, numerals, and other modifying nodes connected to named entities in the dependency syntactic tree as attributes; identify verbs, prepositions, and other relational words connecting two named entity nodes in the dependency syntactic tree as entity relationships; use named entity recognition and linking technology to normalize different expressions referring to the same entity into a unique identifier; perform type inference and standardization on attribute values, and convert them into structured data types, such as dates, numbers, enumerations, etc.; and Relationships are identified and mapped, and classified into predefined relationship types, such as "belongs to", "is located in", "subordinate to", etc.; named entities are used as subjects, attributes are used as predicates, and attribute values ​​are used as objects to generate triples describing entity attributes; named entities are used as subjects and objects, and entity relationships are used as predicates to generate triples describing entity relationships; the generated triples are organized in the order of subject, predicate, and object to form a structured representation of the fused knowledge graph; a deduplication algorithm based on the equality of subject, predicate, and object is used to delete duplicate triples; a merging algorithm based on the equality of subject and predicate is used to merge triples with the same subject and predicate to generate multi-valued attributes or multi-valued relationships.

[0010] Furthermore, it also includes: Step 5, converting the structured triples in the knowledge graph into low-dimensional dense vector representation: using the structured triples in the fused knowledge graph as the input data of the TransE model: extracting all the head entities, relations and tail entities in the knowledge graph to form a structured triple data set; preprocessing the structured triple data set, mapping the entities and relations into unique integer numbers; using the TransE algorithm to perform vectorized representation learning on the structured triples: initializing the parameters of the TransE model, including the entity embedding matrix and the relationship embedding matrix; mapping the head entity, relationship and tail entity in the triple to low-dimensional dense vectors respectively: obtaining the corresponding vertex vector of the head entity from the entity embedding matrix according to the number of the head entity; obtaining the corresponding edge vector of the relationship from the relationship embedding matrix according to the number of the relationship; obtaining the corresponding vertex vector of the tail entity from the entity embedding matrix according to the number of the tail entity; modeling the relationship between triples through vector operations Semantic association: add the head entity vector and the relationship vector to obtain the predicted vector of the head entity after relational transformation; calculate the Euclidean distance between the predicted vector and the tail entity vector as the loss function of the model; use the stochastic gradient descent algorithm to optimize the model parameters and minimize the loss function: calculate the gradient of the entity embedding matrix and the relationship embedding matrix according to the loss function; use the gradient to update the entity vector and the relationship vector so that they can better fit the training data; iterate the optimization process until the model converges or reaches the preset number of iterations; use the trained TransE model to perform low-dimensional dense vector representation of the entities and relations in the knowledge graph: map each entity in the knowledge graph to the corresponding vertex vector to obtain a distributed representation of the entity; map each relationship in the knowledge graph to the corresponding edge vector to obtain a distributed representation of the relationship; the entity vector and the relationship vector retain the semantic information in the knowledge graph and can characterize the potential connection between entities and relationships.

[0011] Among them, in this application, low-dimensional vector representation refers to the conversion of structured triples in the knowledge graph into low-dimensional, densely distributed real vector forms. The low-dimensional vectors of vertices, edges, and attributes obtained usually have dimensions between 50 and 500, which is much smaller than the scale of the original knowledge graph. In low-dimensional vectors, semantically similar entities are closer in vector space and have higher cosine similarity. High-dimensional sparse features are converted into low-dimensional dense representations, and low-dimensional real vectors are used to represent objects such as entities, attributes, and relationships. Low-dimensional vectors can effectively capture the semantic similarity between objects, facilitating similarity calculation and rapid retrieval.

[0012] Preferably, in the process of learning the vectorized representation of structured triples using the TransE algorithm, the loss function of the TransE model is further optimized, and entity attribute information and external text corpus are introduced, specifically including: extracting the attribute triples of the entity from the knowledge graph, namely entity-attribute-attribute value: traversing each entity in the knowledge graph to obtain its associated attributes and attribute values; constructing a list of entity-attribute-attribute value triples as input data for attribute information; using an attribute embedding matrix to map attributes to attribute vectors: initializing the attribute embedding matrix to map attributes to low-dimensional dense vectors; according to the number of the attributes in the attribute dictionary, Get the corresponding attribute vector from the attribute embedding matrix; Add the attribute constraint term to the loss function of the TransE model: The attribute constraint term calculates the similarity between the entity vector and its corresponding attribute vector: Use cosine similarity to measure the similarity between the entity vector and the attribute vector; The higher the similarity, the closer the entity vector and its attribute vector are in the semantic space; Encourage the entity vector and its attribute vector to be close in the semantic space, and guide the entity representation learning process: Add the weighted sum of the attribute constraint term and the original triplet loss term to the loss function of the TransE model; Control the influence of attribute information on entity representation learning by adjusting the weight of the attribute constraint term.

[0013] Jointly optimize the triplet loss function and attribute constraint terms so that the model can consider both structural information and attribute information: calculate the gradients of the triplet loss term and the attribute constraint term, and update the parameters of the TransE model; through joint optimization, the model can capture the attribute characteristics of entities while learning the structural relationship between entities; obtain a more accurate and comprehensive entity vector representation that takes into account both structural information and attribute information; extract contextual information related to knowledge graph entities from external text corpus: use named entity recognition technology to identify entities in the knowledge graph from external text corpus; extract context words of entities in the text and construct an entity-context word association list; use word embedding models such as Word2Vec or GloVe to map context words to word vectors: use pre-trained word embedding models to convert context words into low-dimensional dense word vectors; word vectors can characterize the semantic information of words and reflect the semantic association between context words and entities.

[0014] The context word vectors related to the entity are fused with the entity vectors learned by the TransE model: the similarity between the entity vector and its related context word vectors is calculated to obtain the fusion weight: the cosine similarity is used to measure the similarity between the entity vector and the context word vector; the higher the similarity, the greater the contribution of the context word to the semantic representation of the entity; the entity vector is added to the weighted sum of the context word vectors to obtain the fused entity vector: according to the fusion weight, the context word vectors related to the entity are weighted summed; the weighted sum of the context word vectors is added element-wise to the entity vector learned by the TransE model; the entity vector after fusion of external text information is obtained, which enhances the semantic richness of the entity representation; the fusion of external text information enriches the entity representation and improves the semantic expression ability of the entity vector: the fused entity vector not only contains the structural information and attribute information in the knowledge graph, but also incorporates the semantic information in the external text; by introducing text corpus, the dimension and content of the entity representation are expanded, and the semantic expression ability of the entity vector is improved; more comprehensive and accurate entity representation enhances the interpretability and application value of the knowledge graph.

[0015] Furthermore, step six, construct a similarity-based index for the low-dimensional vector representation of the knowledge graph: calculate the cosine similarity between two vectors in the low-dimensional vector representation of the knowledge graph; use the IndexIVFFlat algorithm in the Faiss similarity search library to index the vectors of vertices, attributes and edges: construct indexes for the entity, attribute and relationship vectors of the knowledge graph respectively: traverse all entities in the knowledge graph, extract their corresponding low-dimensional vector representations, use the entity vector as the key and the entity ID as the value to construct a key-value pair, and store it in the entity index; traverse all attributes in the knowledge graph, extract their corresponding low-dimensional vector representations, use the attribute vector as the key and the attribute ID as the value to construct a key-value pair, and store it in the attribute index; traverse all relations in the knowledge graph, extract their corresponding low-dimensional vector representations, use the relationship vector as the key and the relationship ID as the value to construct a key-value pair, and store it in the relationship index; use the IndexIVFFlat algorithm of the Faiss library to construct an inverted index for each index.

[0016] Furthermore, the IndexIVFFlat algorithm of the Faiss library is used to construct an inverted index for each index, including: using the K-means clustering algorithm to cluster all vectors in the index, and dividing the vector space into K Voronoi units, each unit corresponding to a cluster center; for each vector in the index, calculating its distance from all cluster centers, and assigning the vector to the Voronoi unit with the nearest cluster center; for each Voronoi unit, establishing a Flat index, which contains all vectors in the unit; during retrieval, for a given query vector, calculating its similarity with each vector in the unit through the Flat index.

[0017] Preferably, a K-means clustering algorithm is used to cluster all vectors in the index, and the vector space is divided into K Voronoi cells, each of which corresponds to a cluster center: K initial cluster centers are randomly selected as the initial representative vectors of the K Voronoi cells; for each vector in the index, its distance from the current K cluster centers is calculated, and it is assigned to the Voronoi cell where the nearest cluster center is located; for each Voronoi cell, the geometric center of all vectors in the cell is calculated, and the cluster center is updated; the process of vector assignment and cluster center update is repeated until the cluster center no longer changes significantly or the maximum number of iterations is reached; K stable Voronoi cells and corresponding cluster centers are obtained as the division of the vector space; for each vector in the index, its distance from all cluster centers is calculated, and the vector is assigned to the Voronoi cell where the nearest cluster center is located: for each vector to be assigned, its Euclidean distance or cosine similarity with the K cluster centers is calculated; the cluster center with the closest distance or the highest similarity is selected, and the vector is assigned to the corresponding Voronoi cell. oi unit; the distance calculation between the vector and the cluster center can adopt efficient vector similarity calculation methods, such as inner product, Euclidean distance, etc.; by assigning the vector to the nearest cluster center, local partition and aggregation of the vector space are realized; for each Voronoi unit, a Flat index is established, and the similarity between the query vector and all the vectors in the unit is calculated by exhaustive comparison: in each Voronoi unit, a Flat index is constructed, and the index contains all the vectors in the unit; for a given query vector, its similarity with each vector in the unit is calculated to obtain a similarity list; similarity calculation can adopt efficient vector similarity calculation methods, such as inner product, Euclidean distance, etc.; by exhaustive comparison, the exact similarity between the query vector and all the vectors in the unit is obtained; the construction of the Flat index and the similarity calculation are performed independently in each unit, which can realize parallel processing; through the combination of K-means clustering and Flat index, an inverted index based on Voronoi unit partition is constructed, and the hierarchical structure and local search strategy of the inverted index greatly narrow the search range and improve the retrieval efficiency.

[0018] Furthermore, the method further includes: step seven, optimizing and compressing the three types of inverted index vector data of vertices, attributes, and edges obtained in step six, specifically including: setting the number of sub-indexes s, and initializing s clustering centers using the k-means++ algorithm; dividing the three types of inverted index vector data of vertices, attributes, and edges into s sub-indexes evenly through a heuristic sample allocation strategy; for each sub-index, iteratively executing the k-means clustering algorithm, optimizing the clustering centers by minimizing the mean square error, and obtaining s sub-index data containing equal vectors; for the three types of vector sub-indexes of vertices, attributes, and edges, respectively, creating three groups of GPU threads, each group containing s threads; respectively allocating the s sub-indexes to the corresponding s GPU threads; in each In the GPU thread, the mean vector and covariance matrix of all k-dimensional vectors in the sub-index assigned to the thread are calculated; the covariance matrix calculated by each GPU thread is decomposed by eigenvalue, and the first m eigenvectors with the largest contribution are selected based on the preset variance contribution threshold η to obtain the m-dimensional eigenvector after dimension compression; all vectors in the sub-index corresponding to each GPU thread are projected onto the m eigenvectors selected by the thread to obtain the m-dimensional vector representation data after dimension compression of the sub-index; after all GPU threads complete the assigned sub-index compression task, the m-dimensional vector representation data after compression of all sub-indexes are written back, and the corresponding Faiss index is updated to complete the optimization and compression of the three types of inverted indexes of vertices, attributes, and edges. Among them, k-means++ is an improved version of the k-means clustering algorithm, and its main goal is to optimize the initial selection of cluster centers, thereby improving the clustering effect and convergence speed.

[0019] Preferably, the parameter p is set to divide the three types of inverted index vector data of vertices, attributes, and edges into p data partitions respectively, and each data partition contains approximately equal numbers of vectors; for each data partition, steps ae are performed independently: a. The number of sub-indexes s is set, and s clustering centers are initialized using the k-means++ algorithm; b. The vector data in the data partition is evenly divided into s sub-indexes through a heuristic sample allocation strategy; c. For each sub-index, the k-means clustering algorithm is iteratively executed, and the clustering center is optimized by minimizing the mean square error to obtain s sub-index data containing equal numbers of vectors; d. For s sub-indexes, s GPU threads are created, and each sub-index is assigned to a corresponding GPU thread; e. In each GPU thread, execute steps i-iv: i. Calculate the mean vector and covariance matrix of all k-dimensional vectors in the sub-index; ii. Perform eigenvalue decomposition on the covariance matrix, and select the first m eigenvectors with the largest contribution based on the preset variance contribution threshold η to obtain the m-dimensional eigenvector after dimension compression; iii. Project all vectors in the sub-index onto the selected m eigenvectors to obtain the m-dimensional vector representation data after dimension compression; iv. After the GPU thread completes the sub-index compression, write the compressed m-dimensional vector representation data back to update the Faiss index of the corresponding data partition; After completing the optimization and compression of all data partitions, merge the updated p Faiss indexes to obtain the optimized and compressed complete inverted index. The concept of data partitioning is introduced to improve the efficiency of parallel computing.

[0020] Furthermore, step eight performs distributed storage and heat management on the three types of m-dimensional sub-index vector data of vertices, attributes, and edges compressed in step seven, specifically including: obtaining the three types of m-dimensional sub-index vector data of vertices, attributes, and edges compressed by GPU multi-threaded parallel in step seven; importing a distributed database based on HBase to build a multi-column index storage table for distributed storage; wherein the storage table contains vertex column families, edge column families, and attribute column families, which are respectively used to store the corresponding three types of m-dimensional sub-index vector data; importing a Redis distributed memory database for storage and fast retrieval of hot sub-indexes; using a consistent hash partitioning algorithm, hash-map the three types of m-dimensional sub-index vector data of vertices, edges, and attributes to the corresponding column families of the HBase table for storage, to achieve distributed storage of sub-indexes; counting the historical access frequency of each sub-index, and updating the frequency counter data based on the sliding time window, the frequency counter is implemented using the ordered set data structure in Redis; based on a preset frequency threshold Fh , mark the sub-index with access frequency higher than Fh as hot sub-index, and mark the sub-index with access frequency lower than Fh as cold sub-index; store the m-dimensional vector data marked as hot sub-index in the string type key-value pair of Redis, with the key name being the sub-index ID and the key value being the serialized string of the m-dimensional vector, to realize the memory storage of the hot sub-index; store the m-dimensional vector data marked as cold sub-index in the corresponding column family of the HBase table, with the key name being the sub-index ID and the key value being the serialized byte array of the m-dimensional vector, to realize the distributed storage of the cold sub-index; when performing knowledge retrieval and question-answering requests, first obtain the corresponding hot sub-index m-dimensional vector data from Redis according to the requested index ID, perform similarity calculation and sorting, and obtain preliminary retrieval results; then obtain the corresponding cold sub-index m-dimensional vector data from the corresponding column family of the HBase table according to the requested index ID, perform secondary retrieval and similarity calculation, supplement and optimize the preliminary retrieval results, and finally obtain complete knowledge retrieval and question-answering results.

[0021] Preferably, obtain the three types of m-dimensional sub-index vector data of vertices, attributes, and edges after GPU multi-threaded parallel compression in step seven; import the distributed database based on HBase, and build a multi-column index storage table for distributed storage: create three HBase tables, corresponding to the three types of sub-index data of vertices, attributes, and edges respectively; each HBase table contains multiple column families for storing sub-index data of different hash partitions; the row key of the table is designed to be a unique identifier of the sub-index, such as a sub-index ID or a hash value; import the Redis distributed memory database for storage and retrieval of hot sub-indexes: create three Redis databases, corresponding to the three types of sub-index data of vertices, attributes, and edges respectively; use the hash data structure of Redis, with the unique identifier of the sub-index as the key, to store the corresponding m-dimensional vector data.

[0022] Using the consistent hash partitioning algorithm, the three types of m-dimensional sub-index vector data, namely vertices, edges, and attributes, are hashed and mapped to the corresponding column families of the HBase table for storage: a consistent hash function, such as MD5 or SHA-1, is used to calculate the hash value of each sub-index; the sub-index data is mapped to a predefined HBase column family according to the hash value to achieve partitioned storage of the data; the historical access frequency of each sub-index is counted, and the frequency counter data based on the sliding time window is updated: an access frequency counter is maintained for each sub-index in Redis to record the number of accesses within a certain time window; when there is a new access request, the access frequency counter of the corresponding sub-index is updated, and expired access records are deleted according to the time window. Based on the preset frequency threshold Fh, the sub-indexes with access frequencies higher than Fh are marked as hot sub-indexes, and those with access frequencies lower than Fh are marked as cold sub-indexes: a. Regularly scan the access frequency counters of all sub-indexes and compare them with the frequency threshold Fh; b. Mark the sub-indexes with access frequencies higher than Fh as hot sub-indexes, and mark the sub-indexes with access frequencies lower than Fh as cold sub-indexes; store the hot sub-index data in Redis, and the cold sub-index data in HBase: for the data marked as hot sub-indexes, read it from HBase and write it to the hash data structure of Redis; for the data marked as cold sub-indexes, keep it in the column family storage of HBase; regularly synchronize the data between Redis and HBase to ensure data consistency.

[0023] When performing knowledge retrieval and question-answering requests, first obtain the hot sub-index data from Redis for calculation and sorting, and then obtain the cold sub-index data from HBase for secondary retrieval and calculation: perform similarity calculation between the query vector and the hot sub-index data in Redis to obtain candidate results; sort the candidate results according to the similarity score and select the Top-K most similar sub-indexes; if the hot sub-index data in Redis is not enough to meet the query requirements, obtain the cold sub-index data from HBase for secondary retrieval and calculation; merge the candidate results of Redis and HBase, re-sort them, and obtain the final Top-K most similar sub-indexes; based on the retrieved Top-K sub-indexes. Through the consistent hash partitioning algorithm, the sub-index data is evenly mapped to the column family of HBase, avoiding data skew and hot spot problems. At the same time, through access frequency statistics and identification of hot sub-indexes, the frequently accessed sub-index data is dynamically cached in Redis, which improves the query response speed. In the process of knowledge retrieval and question-answering, hot sub-index data is obtained from Redis first for calculation, making full use of the high-speed access characteristics of memory. When the data in Redis is insufficient, the cold sub-index data is obtained from HBase to supplement it to ensure the completeness and accuracy of the query.

[0024] Furthermore, in step 1, the Seq2Seq framework is used to perform named entity recognition on the contextualized word vectors output by RoBERTa. The Seq2Seq framework adopts the BiLSTM-Attention-CRF network structure: the BiLSTM-Attention-CRF network realizes end-to-end named entity recognition through the encoding layer, attention mechanism layer and CRF decoding layer of the bidirectional LSTM; the BiLSTM encoding layer receives the contextualized word vectors generated by RoBERTa, and learns the context information of the input vector and the long-distance dependency relationship between words through the bidirectional LSTM to obtain the feature vector representation containing contextual semantic information; the attention mechanism layer receives the feature vector representation output by the BiLSTM encoding layer and the predefined named entity type embedding vector, and dynamically adjusts the BiLSTM encoding features by calculating the attention weight between the feature vector and the type embedding vector, highlighting the semantic information related to the current named entity recognition task, and obtaining the feature vector representation adjusted by the attention mechanism;

[0025] The CRF decoding layer receives the adjusted feature vector representation output by the attention mechanism layer, and makes the best sequence labeling decision globally by modeling the Markov transition constraints between the named entity labels to obtain the final named entity recognition result. Specifically, the CRF decoding layer uses the adjusted feature vector representation output by the attention mechanism layer as the observation sequence and the named entity label as the hidden state sequence. By defining the emission score matrix and the transfer score matrix, the conditional probability relationship between the observation sequence and the hidden state sequence is established. The emission score matrix describes the compatibility between the feature vector representation of each position in the observation sequence and the named entity label at the corresponding position. The latter feature vector represents the compatibility score with the named entity label; the transfer score matrix characterizes the transfer compatibility between adjacent named entity labels, which is composed of the prior transition probability and context constraint features of the named entity labels, and takes into account the Markov dependency between the named entity labels; the CRF decoding layer uses a dynamic programming algorithm to find the named entity label sequence with the largest global probability as the optimal sequence labeling result on the probabilistic graph model defined by the emission score matrix and the transfer score matrix; the optimal named entity label sequence is aligned with the input natural language text, and by mapping consecutive labels of the same type into named entity mentions, the recognition and extraction of named entities from natural language text is realized.

[0026] By forward propagating the loss function and adjusting the parameters in the BiLSTM-Attention-CRF network, the BiLSTM feature encoding, attention mechanism feature adjustment and CRF sequence labeling are jointly optimized to train the named entity recognition model end-to-end. The contextualized word vector generated by RoBERTa is used as the input of the BiLSTM encoding layer, encoding the context-related semantics of the words, providing rich semantic features for named entity recognition. The attention mechanism layer dynamically adjusts the feature representation by fusing the BiLSTM encoding features and the prior knowledge of named entity types, highlighting the semantic information related to the current named entity recognition task and improving the recognition accuracy. The CRF decoding layer performs global sequence labeling based on the features adjusted by the attention mechanism, and improves the overall coherence and consistency of named entity recognition by modeling Markov transfer constraints between labels. The end-to-end trained BiLSTM-Attention-CRF model can make full use of RoBERTa semantic encoding, attention mechanism feature adjustment and CRF global constraints to accurately extract named entity knowledge elements from the text, laying a solid foundation for the construction of knowledge graphs.

[0027] Furthermore, in step 2, the relationship between entity nodes is obtained through relationship template matching as the relationship edge in the knowledge graph, including: for the named entity pairs obtained in step 1, a syntactic dependency tree is obtained using a pre-trained syntactic dependency analysis model: the sentence containing the named entity pair is input into the pre-trained syntactic dependency analysis model; the syntactic dependency analysis model performs lexical analysis and syntactic analysis on the sentence to obtain the dependency structure of the sentence; the dependency structure connects the word nodes in the sentence through directed edges to form a syntactic dependency tree, which describes the syntactic structure of the sentence and the dependency relationship between words. In the syntactic dependency tree, the shortest dependency path between named entity pairs is obtained: the syntactic dependency tree is converted into a directed graph representation, and the head entity and the tail entity in the entity pair correspond to two nodes in the syntactic dependency tree; the shortest path algorithm is used to search the directed graph converted from the syntactic dependency tree to find the shortest directed path from the head entity node to the tail entity node; the nodes and edges passed on the shortest directed path constitute the shortest dependency path between entity pairs, which characterizes the keywords and key dependency relationships connecting the entity pairs; based on the obtained shortest dependency path, a trigger template for entity relationship extraction is constructed.

[0028] Using the constructed relationship template, the relationship extraction is performed on the named entity pairs obtained in step one: the syntactic dependency tree of the named entity pairs and the sentences they are in is matched with the relationship template; if there is a shortest dependency path matching the relationship template in the syntactic dependency tree, the semantic relationship between the entity pairs is extracted from it; the extracted entity relationship is used as the relationship edge of the knowledge graph to connect the corresponding head entity node and tail entity node; the extracted entity relationship is integrated into the entity node set constructed in step one to obtain a knowledge graph containing entities and relationships: the entity nodes are connected by relationship edges to form the topological structure of the knowledge graph; the semantic relationship labels between the entity pairs are attached to the relationship edges to characterize the association between the entity nodes; the entity relationships obtained by matching the relationship template enrich the semantic information of the knowledge graph and improve the quality and application value of the knowledge graph.

[0029] Furthermore, in the syntactic dependency tree, the shortest dependency path between named entity pairs is obtained, including: traversing each word node and dependency relationship in the syntactic dependency tree, and converting the syntactic dependency tree into a directed graph according to the word nodes and the corresponding dependency relationships: taking each word node in the syntactic dependency tree as a node of the directed graph, and the attributes of the node include the word itself, part of speech, and named entity label; taking each dependency relationship in the syntactic dependency tree as a directed edge of the directed graph, and the starting point of the directed edge is the tail node of the dependency relationship, and the end point is the head node of the dependency relationship; the direction of the dependency relationship represents the modification relationship between words, and the attributes of the directed edge include the type of the dependency relationship; on the converted directed graph, taking the head entity node as the starting point and the tail entity node as the end point, using Dijkstra's shortest path algorithm searches for the shortest dependency path: initialize the distance of the starting node to 0 and the distances of other nodes to positive infinity; initialize the predecessor node of the starting node to be empty, indicating that the starting node has no predecessor; add the starting node to the priority queue, and sort the priority queue in ascending order according to the distance value of the node; take out the node with the smallest distance from the priority queue, mark it as visited, and traverse all its outgoing edges; for each outgoing edge, calculate the distance from the current node to the neighbor node. If it is less than the current distance of the neighbor node, update the distance value and set the current node as the predecessor node of the neighbor node; if the neighbor node has not been visited, add it to the priority queue; repeat the above process until the end node is reached or the priority queue is empty.

[0030] According to the node predecessor relationship obtained by the Dijkstra algorithm, backtrack from the tail entity node to construct the shortest directed path from the head entity node to the tail entity node as the shortest dependency path: starting from the tail entity node, backtrack according to the node's predecessor information; add the current node to the head of the shortest dependency path, and use its predecessor node as the new current node; repeat the above backtracking process until the head entity node is reached to obtain the complete shortest dependency path; the shortest dependency path consists of word nodes and directed edges, which characterizes the keywords connecting the head entity and the tail entity and their dependency structure, and contains the semantic relationship between entity pairs.

[0031] Furthermore, based on the obtained shortest dependency path, a trigger template for entity relationship extraction is constructed, including: obtaining word nodes in the shortest dependency path, extracting part-of-speech tags of the word nodes: traversing each word node in the shortest dependency path, and using a part-of-speech tagging tool to tag the word nodes with part-of-speech; extracting the part-of-speech tagging results of each word node, and obtaining a list of word nodes with part-of-speech tags as node attributes;

[0032] Get the directed edges in the shortest dependency path and extract the dependency type of the directed edges: traverse each directed edge in the shortest dependency path, and use the dependency analysis tool to classify the dependencies of the directed edges; extract the dependency type of each directed edge, and obtain a directed edge list with the dependency type as the edge attribute; according to the part-of-speech tagging and dependency type, convert the shortest dependency path into graph structure data: take each part-of-speech tag in the word node list as a node of the graph, and build a node list; according to the dependency type in the directed edge list, establish directed edges between the corresponding part-of-speech tag nodes and build an edge list; the graph structure representation of the shortest dependency path composed of the node list and the edge list describes the abstract relationship between entity pairs Dependency structure; According to the graph structure, a graph attention network is used to learn the distributed representation of the shortest dependency path: each part-of-speech tagging node in the node list is mapped to a low-dimensional dense vector through a pre-trained word embedding model to obtain the initial feature matrix of the node; the graph attention network is used to aggregate node features and edge information, and the node feature matrix is ​​updated: based on the initial features of the node and the features of neighboring nodes and edges, the importance weight of each node is calculated through the attention mechanism; the initial features of the node are multiplied and added with its importance weight to obtain the updated features of the node, reflecting its semantic contribution in the path; the updated node features are weighted and summed based on the attention weights of the edges to obtain the distributed representation vector of the shortest dependency path.

[0033] According to the distributed representation of the shortest dependency path, a trigger template for entity relationship extraction is constructed: the path representation vector is divided into three sub-vectors, which correspond to the head entity slot, tail entity slot and middle word slot of the trigger template respectively; the head entity slot and tail entity slot are used to match the head entity and tail entity in the input entity pair; the middle word slot is used to match the trigger word connecting the head entity and the tail entity; the dependency type slot is set in the trigger template to match the key dependency between the entity pair and the trigger word; the slot of the trigger template corresponds to the part-of-speech tagging node and the dependency type edge in the shortest dependency path. Realize the mapping from specific paths to abstract templates; use the constructed trigger template to extract relationships between new entity pairs and their dependency paths: convert new entity pairs and their syntactic dependency paths into graph structure data, and extract the shortest dependency path; obtain the part-of-speech tagging nodes and dependency relationship type edges in the shortest dependency path; match the extracted parts of speech and dependency relationships with the trigger template; if the match is successful, the relationship type corresponding to the trigger template is used as the semantic relationship between the entity pairs; use the extracted semantic relationship as a relationship edge to connect the head entity node and the tail entity node in the entity pair, and add them to the knowledge graph.

[0034] Compared with the prior art, the advantages of this application are:

[0035] By using the pre-trained RoBERTa model and BiLSTM-Attention-CRF network for word segmentation, part-of-speech tagging, semantic parsing and named entity recognition, the rich semantic knowledge learned by the pre-trained language model and the sequence labeling model enhanced by the attention mechanism are fully utilized to improve the accuracy of named entity recognition.

[0036] The pre-trained syntactic dependency analysis model is used to obtain the syntactic dependency tree, and the relationship between entities is obtained through the shortest dependency path and relationship template matching, which overcomes the limitation of poor generalization ability of traditional relationship extraction methods based on pattern matching and improves the accuracy and coverage of relationship extraction. At the same time, the graph attention network is introduced to learn the representation of the shortest dependency path, which further improves the semantic expression ability of the relationship extraction template.

[0037] The constructed knowledge graph is integrated with the existing structured relational database to introduce string matching and attribute semantic similarity calculation, which realizes the automatic alignment and completion of the knowledge graph and traditional structured data and expands the knowledge coverage of the knowledge graph.

[0038] The fused knowledge graph is converted into a structured triple representation, and the TransE algorithm is used to learn the low-dimensional vector representation of the triple so that entities and relationships can be mapped to the same vector space. The rationality of the triple can be judged through simple vector operations, reducing the difficulty of subsequent knowledge reasoning and completion tasks based on representation learning.

[0039] The Faiss similarity search library is used to construct a similarity-based inverted index for the low-dimensional vector representations of vertices, attributes, and edges. With the help of vectorized representation and similarity index, fast retrieval and matching of knowledge graph data is achieved, further reducing the computational complexity of candidate entity matching and ranking in entity linking.

[0040] We introduce optimized compression operations such as k-means clustering and PCA dimensionality reduction into the inverted index, and use GPU multi-threaded parallel processing to significantly reduce the storage space of index data and improve the speed of index construction and retrieval while ensuring retrieval accuracy.

[0041] HBase and Redis are used comprehensively to perform distributed storage and heat management on the compressed sub-indexes. By counting the historical access frequency and identifying the heat, local access and real-time response to hot data in knowledge graph queries are achieved, further optimizing the retrieval and question-answering efficiency of knowledge graph data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an exemplary flow chart of a method for constructing a knowledge graph based on a large language model and a vector library according to some embodiments of the present application;

[0043] Figure 2 It is a schematic diagram of constructing a knowledge graph according to some embodiments of the present application;

[0044] Figure 3 It is a schematic diagram of constructing a fusion knowledge graph according to some embodiments of the present application;

[0045] Figure 4 This is a schematic diagram of constructing a vector index according to some embodiments of the present application. DETAILED DESCRIPTION

[0046] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0047] Figure 1This is an exemplary flow chart of a method for constructing a knowledge graph based on a large language model and a vector library according to some embodiments of the present application, including: step 1, using the pre-trained RoBERTa model to perform word segmentation, part-of-speech tagging and semantic analysis on the collected natural language text to obtain a semantic representation vector of the natural language text; inputting the semantic representation vector into the Seq2Seq framework of the BiLSTM-Attention-CRF network structure to obtain named entities; step 2, using the named entities identified in step 1 to construct an initial knowledge graph: using the identified named entities as entity nodes of the knowledge graph; using the pre-trained syntactic dependency analysis model to obtain syntactic dependency tree data; in the syntactic dependency tree, obtaining the shortest dependency path between named entity pairs; based on the obtained shortest dependency path, constructing a relationship template; using the constructed relationship template to extract relationships between named entity pairs to obtain entity relationship data; using the extracted entity relationships as relationships of the knowledge graph. Connect the corresponding head entity node and tail entity node, and construct a knowledge graph data containing entity nodes, attributes and relationship edges; Step three, merge the knowledge graph constructed in Step two with the relational database of the preset field to generate a fused knowledge graph; Use a string matching algorithm to obtain the same-name entity pairs with the same attributes in the knowledge graph and the relational database; For each same-name entity pair, calculate the semantic similarity of the corresponding attributes; When the semantic similarity is higher than the preset threshold, add the attributes of the corresponding entity in the relational database to the corresponding entity node in the knowledge graph; When the semantic similarity is lower than the preset threshold, rename the corresponding entity node in the knowledge graph or add a distinguishing mark; Step four, perform a structured triple representation on the fused knowledge graph: Use dependency syntactic analysis to obtain the named entities, corresponding attributes and entity relationship data between the named entities in the fused knowledge graph; Based on the acquired named entities, attributes and entity relationship data, construct a structured triple representation data of the fused knowledge graph.

[0048] First, the collected natural language text data is cleaned to remove noise data such as HTML tags, special characters, and emoji expressions. The cleaned text is processed into sentences, and the text is divided into sentences according to punctuation marks to obtain sentence-level text data. The preprocessed sentence data is input into the pre-trained RoBERTa model. The RoBERTa model adopts the Transformer encoder structure, and learns the contextual semantic relationship between each word in the sentence through a multi-layer self-attention mechanism and a feedforward neural network. The semantic representation vector of each word is extracted from the output of the last layer of the RoBERTa model, and a sentence is represented as a sequence of word vectors. While extracting the semantic representation vector, the RoBERTa model can also output the segmentation result and part-of-speech tagging result of each word as an auxiliary feature for subsequent named entity recognition.

[0049] Build a BiLSTM-Attention-CRF network: Build a BiLSTM encoding layer: Use a deep learning framework such as Keras to build a bidirectional LSTM encoder. Set the input dimension of the BiLSTM to the dimension of the RoBERTa word vector, such as 768. Define the number of hidden units of the BiLSTM, such as 256. You can adjust it according to the complexity of the task and the scale of the data. Set the number of BiLSTM layers, such as 2 layers. Deeper layers can capture higher-level contextual features, but also increase computational complexity. Add a fully connected layer after the BiLSTM layer and adjust the output dimension to the number of named entity types, such as 7-dimensional output for 7 entity types.

[0050] Add an attention mechanism layer: Add an attention mechanism layer after the BiLSTM layer to introduce named entity type information. Define a set of learnable entity type embedding vectors with the same dimension as the BiLSTM output feature vector, such as 256 dimensions. Dot product the feature vector output by the BiLSTM with the entity type embedding vector to obtain the attention score. Perform softmax normalization on the attention score to obtain the attention weight. Use the attention weight to weight the sum of the BiLSTM output features to obtain a feature representation that incorporates the entity type information.

[0051] Build CRF decoding layer: Use third-party libraries such as Keras-contrib to build CRF decoding layer. The CRF decoding layer receives the feature vector sequence output by the attention layer. Define the state feature function of CRF to map the feature vector output by the attention layer to the emission score of CRF. Define the transfer feature function of CRF to characterize the transition probability between labels. You can use n-gram features, part-of-speech features, etc. The CRF layer outputs the logarithmic probability of the label sequence, which is used to calculate the loss function. The built BiLSTM-Attention-CRF model can be trained and predicted end-to-end, inputting the RoBERTa word vector representation of the sentence and outputting the named entity label corresponding to each word. By introducing entity type embedding in the attention layer, the model can better characterize the semantic features of different types of entities. The CRF decoding layer takes into account the constraints and transfer rules between labels, improving the accuracy and consistency of recognition.

[0052] Prepare training data and annotate named entities: Collect a large amount of domain text corpus from business systems or crawlers, such as news articles, social media comments, etc. Define the type system of named entities, such as names of people, places, institutions, time, numbers, etc. Design types with appropriate granularity according to task requirements. Recruit and train annotators and formulate annotation specifications and guidelines. Standardize the boundaries and ambiguity handling principles of each entity type. Use text annotation tools such as BRAT and doccano to facilitate annotators to efficiently and accurately annotate named entities. Annotate the named entities in each sentence to indicate the type and boundaries of the entity. For example, "[Company A] / Company headquarters is located in [XXX city] / [XXX district] / district and county of the city". Organize data format: Organize the annotated corpus according to a certain format. Common sequence annotation formats include BIO and BIOES. Taking the BIO format as an example, B represents the beginning of the entity, I represents the middle of the entity, and O represents non-entity. Encode the entity type into the label, such as B-company, I-company, etc. For the sentence "[Company A] / Company headquarters is located in [XXX city] / City's [XXX district] / District", convert it to BIO format. Process the labeled data into the form of (sentence, label sequence), where the sentence is the original text and the label sequence is the label in BIO format. Divide the training set, validation set, and test set, usually in a ratio of 8:1:1 or 7:2:1. Ensure that the data distribution of different sets is consistent.

[0053] Model training: Input the sentences of the training set into the pre-trained RoBERTa model to extract the semantic feature vector of each word. RoBERTa can learn the contextual representation of words from large-scale corpus. The word vector sequence output by RoBERTa is used as the input of the BiLSTM encoding layer. The context information of the sentence is learned through bidirectional LSTM to obtain a feature vector sequence containing forward and backward semantic information. The attention mechanism layer introduces a learnable named entity type embedding vector and calculates the attention weight of each word feature vector and the entity type vector. Through attention weighting, the word features related to the current entity type are highlighted. The CRF decoding layer receives the feature vector sequence output by the attention layer, and characterizes the global dependency of the label sequence by defining the state feature function and the transition feature function. The Viterbi algorithm is used for decoding to obtain the globally optimal named entity label sequence. The cross entropy loss function of the CRF decoding layer is calculated to measure the difference between the predicted label sequence and the true label sequence. The parameters of the BiLSTM, attention layer and CRF layer are updated through the back propagation algorithm to make the model fit the training data. During the training process, the learning rate decay strategy is adopted to gradually reduce the learning rate as the number of training rounds increases so that the model converges to the optimal point. Gradient clipping is used to avoid gradient explosion. Insert the Dropout layer between the BiLSTM layers to randomly block some neurons and enhance the generalization performance of the model.

[0054] Figure 2 It is a schematic diagram of constructing a knowledge graph according to some embodiments of the present application, and the named entities identified in step one are used to construct an initial knowledge graph: Syntactic dependency analysis: For a given sentence containing a named entity pair, a pre-trained syntactic dependency analysis model is used to perform dependency analysis. Commonly used syntactic dependency analysis models include StanfordParser, spaCy, etc. These models are trained on large-scale corpora and can accurately analyze the syntactic structure of sentences. The result of syntactic dependency analysis is a syntactic dependency tree, which is represented in the form of a directed acyclic graph. The nodes in the syntactic dependency tree are the words in the sentence, and the directed edges represent the dependency relationships between words, such as subject, object, attributive, etc. The dependency tree depicts the grammatical structure of the sentence and the modification and dominance relationships between words, providing a basis for subsequent shortest dependency path calculations.

[0055] Find the shortest dependency path and convert the syntactic dependency tree into a directed graph: Given a sentence containing a named entity pair, its dependency tree representation is obtained through syntactic dependency analysis. Convert the dependency tree into a directed graph G, with the word nodes of the dependency tree as nodes of graph G and the dependency edges as directed edges of graph G. Each word node contains attribute information such as the word itself, part-of-speech tag, and position. Each directed edge represents the dependency relationship between words, such as "subject-predicate relationship" and "verb-object relationship", and the weight of the edge can be set as needed. Define the starting point and end point: In the directed graph G, the head entity node is the starting point s and the tail entity node is the end point t. The head entity node and the tail entity node can be obtained through named entity recognition or manual annotation. If there are multiple head entities or tail entities, the representative node can be selected as the starting point or end point. Initialize the distance and predecessor node: Define two arrays dist and prev to store the distance and predecessor node of the node respectively. Initialize the distance dist[s] of the starting point s to 0, indicating that the distance from the starting point to the starting point is 0. Initialize the distance dist of other nodes to infinity, indicating that they have not been visited yet. Initialize the predecessor node prev of all nodes to NULL, indicating that the predecessor has not been determined. Initialize the priority queue: define a priority queue q to store the nodes to be visited. The nodes in the priority queue are sorted according to the size of the distance dist, and the nodes with the smallest distance are removed from the queue first. Add the starting point s to the priority queue q as the first node to be visited.

[0056] Dijkstra shortest path search: When the priority queue q is not empty, repeat the following steps: Take out the node u with the smallest distance in q and mark u as visited. Traverse all unvisited neighbor nodes v of u, that is, nodes connected to u by outgoing edges. For each neighbor node v, calculate the distance dist[u]+w(u,v) from the starting point s through u to v, where w(u,v) is the weight of the edge (u,v). If dist[u]+w(u,v) is less than v's current distance dist[v], update dist[v]=dist[u]+w(u,v), and set v's predecessor node prev[v] to u. If v is not in the priority queue q, add v to q, indicating that v becomes a new node to be visited. Repeat the above steps until the end point t is reached or the queue q is empty. Shortest path construction: If the distance dist[t] of the end point t is infinite, it means that the starting point s cannot reach the end point t, there is no shortest path, and the algorithm exits. Otherwise, starting from the end point t, backtrack along the predecessor node pointer prev to construct the shortest path path from the starting point s to the end point t. The specific process is: starting from prev[t], continuously find the predecessor node of the current node until backtracking to the starting point s. The nodes and edges passed in the backtracking process are stored in the order from s to t to form the shortest path path. Shortest dependency path representation: The shortest path path is the shortest dependency path connecting the head entity and the tail entity in the sentence. The nodes on the path contain key words, and the edges represent the dependency relationship between words. The path can be represented as a dependency path string, such as "head entity-subject-predicate relationship-verb-verb-object relationship-tail entity". The path can also be represented as a dependency path graph, with nodes as words and edges as dependency relationships, for further relationship representation learning. Using the Dijkstra shortest path algorithm to search on the directed graph obtained by converting the syntactic dependency tree, the shortest dependency path connecting the head entity and the tail entity can be efficiently found. The shortest dependency path describes the grammatical dependency relationship between entities, contains key trigger words and context information, and provides an important semantic basis for subsequent relationship extraction and knowledge graph construction. Preferably, the algorithm can be optimized and improved according to task requirements, such as introducing edge weights based on dependency types, using the A* search algorithm, and utilizing the special structure of the syntactic dependency tree, to further improve the efficiency and accuracy of shortest path search. At the same time, it is also possible to combine syntactic analysis and semantic role labeling techniques to obtain richer syntactic semantic information and enhance the representation capability of the shortest dependency path.

[0057] Relational representation learning: Graph structure representation of the shortest dependency path: The shortest dependency path path is represented as a directed acyclic graph G, where nodes are words on the path and edges are dependencies between words. Each node contains attribute information such as the word itself, part-of-speech tag, and position. Each directed edge represents the type of dependency relationship between words, such as "subject-predicate relationship" and "verb-object relationship". The graph G depicts the structure and semantic information of the shortest dependency path and is the input for relational representation learning. Node and edge feature extraction: For each word node on the path, its part-of-speech tag is extracted as the attribute feature of the node. Part-of-speech tagging reflects the grammatical function of the word in the sentence, such as noun, verb, adjective, etc., which is of great significance for relational representation. For each dependency edge on the path, its dependency type is extracted as the attribute feature of the edge. The dependency type represents the grammatical dependency relationship between words, such as subject-predicate, verb-object, attributive, etc., and depicts the semantic role between words. Application of graph attention network GAT: Use graph attention network GAT to learn the distributed representation of the shortest dependency path. GAT assigns different attention weights to each node through the attention mechanism, aggregates the node's neighbor information, and updates the node representation.

[0058] The specific process is as follows: the attribute features of the nodes and the attribute features of the edges are input into GAT. The attention weight between each node and its neighboring nodes is calculated through the attention mechanism, and the weight reflects the importance of the neighboring nodes to the current node representation. The representation of the neighboring nodes is weighted and summed using the attention weights to obtain the new representation of the current node. The above process is iterated multiple times to continuously update and optimize the node representation and capture the structure and semantic information of the path. After learning multiple layers of GAT, the final distributed representation vector of each node in the path is obtained. GAT effectively models the structure and semantic information of the dependency path through the attention mechanism and learns high-quality relationship representations. Relationship template learning: Learning of relationship extraction trigger templates: Using the distributed representation of the shortest dependency path to learn the trigger template for relationship extraction. Match the path representation vector with the representation of the relationship type, build a classification model, and predict whether the entity pair represented by the path satisfies a certain relationship. Common classification models include binary classification (whether the relationship is satisfied) and multi-classification (which type of relationship it belongs to).

[0059] Preparation of training data: Collect entity pairs with known relationships and their shortest dependency paths as training data. For each relationship type, find entity pairs that satisfy the relationship and extract the shortest dependency path between them as positive examples. At the same time, randomly select entity pairs that do not satisfy the relationship and extract the shortest dependency path between them as negative examples. Positive examples and negative examples constitute the training data of the binary classification model. Positive examples indicate that the relationship is satisfied, and negative examples indicate that the relationship is not satisfied. Training of classification model: Take the distributed representation of the shortest dependency path as input features and the relationship type as output label to train the classification model. Commonly used classification models include logistic regression, support vector machine, neural network, etc. Select an appropriate model according to task requirements and data scale. By optimizing model parameters, the classification model can accurately determine whether a new entity pair path represents a certain relationship. During the model training process, cross-validation, regularization and other technologies are used to avoid overfitting and improve the generalization ability of the model. Application of relationship templates: The classification model obtained by training is the trigger template for relationship extraction, which can be used for new entity pair relationship prediction. For new entity pairs, extract the shortest dependency path between them and use the relationship template for prediction. If the template predicts that an entity pair satisfies a certain relationship, a corresponding relationship edge is added to the knowledge graph to connect the entity pair. By applying the relationship template, the implicit entity relationship in the text can be automatically discovered and the knowledge graph can be expanded.

[0060] Knowledge graph construction, relationship extraction: Use the learned relationship template to extract relationships for new named entity pairs. Identify named entities from text data, such as names of people, places, and organizations, to form entity pairs. For each entity pair, obtain its shortest dependency path representation through syntactic dependency analysis. Input the shortest dependency path representation into the relationship template for relationship prediction. The relationship template can be a binary classification model (predicting whether a relationship exists) or a multi-classification model (predicting the type of relationship). If the relationship template predicts that there is a relationship between entity pairs, the relationship is extracted and the relationship type is marked. Knowledge graph construction: Use the extracted entity relationship as the relationship edge of the knowledge graph to connect the corresponding head entity and tail entity nodes. The head entity and tail entity correspond to the subject and object of the relationship, respectively, and represent the semantic association between entities. Each entity is a node in the knowledge graph, containing information such as entity name, type, and attributes. Entity nodes can be extracted and identified from text through named entity recognition, entity linking, and other technologies. Relationship edges represent semantic relationships between entities, including attributes such as relationship type and relationship strength. The extracted entity relationships are converted into triples of the knowledge graph, such as (head entity, relationship, tail entity).

[0061] Knowledge fusion: Integrate entity node and relationship edge data to build a knowledge graph containing entities and relationships. The knowledge graph is represented by a graph data structure, where nodes represent entities and edges represent relationships between entities. De-duplicate, disambiguate and merge the extracted entities and relationships to ensure the consistency and accuracy of the knowledge graph. Use external knowledge bases, dictionaries and other resources to link and align entities and relationships to expand the coverage of the knowledge graph. Through knowledge fusion technologies such as entity resolution and relationship reasoning, discover implicit entity relationships and enrich the knowledge graph. Knowledge storage: The knowledge graph is stored in the form of a graph database, which facilitates efficient query, reasoning and visualization. Commonly used graph databases include Neo4j, JanusGraph, ArangoDB, etc., which support the storage and query of graph structures. Import the constructed knowledge graph into the graph database, and each entity and relationship corresponds to a node or edge in the database. Graph databases support complex queries based on graphs, such as shortest path queries, subgraph matching, relationship reasoning, etc. The structure and content of the knowledge graph can be intuitively displayed through the visualization tools of the graph database.

[0062] Figure 3 It is a schematic diagram of building a fusion knowledge graph according to some embodiments of the present application, entity alignment: obtain a list of entity names in the knowledge graph and relational database. Preprocess the entity names, such as removing spaces, punctuation marks, etc. Define edit distance: edit distance refers to the minimum number of edit operations required to convert one string to another. Edit operations include: inserting a character, deleting a character, and replacing a character. For example, converting the string "kitten" to "sitting" requires the following operations: replace "k" with "s"; replace "e" with "i"; insert "g"; the edit distance is 3. Create a two-dimensional matrix: create a two-dimensional matrix dp to store the intermediate calculation results of the edit distance. The number of rows of the matrix is ​​the length of the first string plus 1, and the number of columns is the length of the second string plus 1. dp[i][j] represents the minimum edit distance required to convert the first i characters of the first string to the first j characters of the second string. Initialize the matrix: initialize the first row and the first column of the matrix to increasing values. dp[0][j] represents the edit distance required to convert the empty string to the first j characters of the second string, that is, the number of insertion operations. dp[i][0] represents the edit distance required to convert the first i characters of the first string into an empty string, that is, the number of deletion operations.

[0063] Fill the matrix: traverse each position (i, j) of the matrix, where i and j represent the first i characters of the first string and the first j characters of the second string respectively. For each position, calculate the edit distance of the following three cases: insertion operation: dp[i][j-1]+1; deletion operation: dp[i-1][j]+1; replacement operation: dp[i-1][j-1]+(0 if the characters are the same, 1 if the characters are different); select the minimum value of the three cases as the edit distance of the current position and store it in dp[i][j]. Return result: After the matrix is ​​filled, dp[m]n is the edit distance between the two strings. By calculating the edit distance, the similarity between two entity names can be measured. The smaller the edit distance, the more similar the two entity names are, and the more likely they are different representations of the same entity. In the entity alignment task, a threshold can be set based on the edit distance to determine whether two entities are entities with the same name. For each pair of entities with the same name, extract their corresponding attribute information from the knowledge graph and the relational database. The attribute information can be attribute name, attribute value, attribute description, etc.

[0064] Use the pre-trained semantic matching model RoBERTa for attribute fusion. Load the pre-trained RoBERTa model and its corresponding vocabulary and tokenizer from the model library. RoBERTa is a pre-trained language model based on the Transformer architecture that can capture semantic information in text. For each pair of entities with the same name, extract their attribute descriptions. The attribute description can be text information such as the name, value, and description of the attribute. Organize the attribute descriptions into pairs, with each sample containing two attribute descriptions. Use RoBERTa's tokenizer to tokenize the attribute descriptions and convert the text into an input format acceptable to the model. Truncate or pad the input to ensure that the length of each sample is consistent. Convert the tokenized attribute descriptions to the input representation required by the RoBERTa model, such as token_ids, attention_mask, etc. Input the pre-processed attribute description pairs into the RoBERTa model. The RoBERTa model encodes the input and generates a semantic representation vector for each attribute description. Use a metric such as cosine similarity to calculate the similarity between the semantic representation vectors of two attribute descriptions. The similarity score usually ranges from [0, 1]. The higher the score, the more semantically similar the attribute descriptions are.

[0065] For each attribute description pair, compare its semantic similarity score with the set threshold. If the similarity score is higher than the threshold, it is considered that the two entities with the same name are consistent in this attribute and belong to matching attributes. If the similarity score is lower than the threshold, it is considered that the two entities with the same name have differences in this attribute and belong to unmatched attributes. For matching attributes, add the attribute value of the corresponding entity in the relational database to the entity node with the same name in the knowledge graph. If the attribute already exists in the knowledge graph, merge or update the attribute value to ensure the consistency of the attribute value. For unmatched attributes, differentiation processing is required. One of the following two methods can be adopted: rename the entity node in the knowledge graph, such as adding a data source identifier to distinguish it. Add a new distinguishing attribute to the entity node in the knowledge graph to indicate the difference in attribute values. Using the pre-trained semantic matching model RoBERTa, the semantic similarity calculation of the attribute descriptions of the entities with the same name can be realized, and the attribute fusion and differentiation processing can be performed according to the similarity threshold. In this way, the attribute information in the relational database can be effectively integrated into the knowledge graph, enriching the attribute information of the knowledge graph, and dealing with the problem of attribute conflicts.

[0066] Integrate the results of attribute fusion into the knowledge graph. For each pair of entities with the same name, add or update the attribute value in the relational database to the corresponding entity node in the knowledge graph. If the entity node in the knowledge graph already contains the attribute, update the attribute value. If the entity node in the knowledge graph does not have the attribute, add the attribute and its corresponding attribute value. For each entity node, analyze its attribute values ​​and explore the possible relationships between entities. Attribute values ​​can provide clues about the relationship between entities, such as "department" and "located in the city". According to domain knowledge and data characteristics, predefine some common relationship types, such as "belongs to", "located in", "belongs to", etc. Match the attribute values ​​with the predefined relationship types to determine the relationship type between entities. In addition to using predefined relationship types, new relationship types can also be automatically discovered through the analysis of attribute values. Use natural language processing techniques, such as named entity recognition and relationship extraction, to extract relationship information between entities from attribute values.

[0067] Generate new relationship types based on the extracted relationship information. Create relationship edges between corresponding entity nodes in the knowledge graph based on the determined relationship types. Relationship edges represent semantic connections between entities and describe the relationship types between entities. Check whether there are attribute value conflicts in the fused knowledge graph. Attribute value conflicts refer to the existence of multiple inconsistent attribute values ​​for the same attribute of the same entity. Check whether there are missing or incomplete data in the fused knowledge graph. For missing key attributes or relationships, they can be supplemented by data completion or manual annotation. Choose a suitable storage method based on the scale, query requirements, and application scenarios of the knowledge graph. Common storage methods include graph databases (such as Neo4j) and RDF storage. If you choose to use a graph database, convert the knowledge graph into a graph database data model. Entities are nodes in the graph and relationships are edges in the graph. Store the attributes of the entity as the attributes of the node and the type of the relationship as the type of the edge. If you choose to use RDF storage, convert the knowledge graph into the form of RDF triples. Each triple consists of a subject, a predicate, and an object, corresponding to an entity, a relationship, and another entity or attribute value, respectively. Use an RDF storage engine, such as Apache Jena or OpenLink Virtuoso, to store RDF triples.

[0068] Figure 4 It is a schematic diagram of constructing a vector index according to some embodiments of the present application, and the vector similarity is calculated: the low-dimensional vector representation of entities, attributes and relationships is learned using the TransE model. For two vectors x and y, their cosine similarity is calculated: sim(x, y) = x·y / (||x||*||y||). x·y represents the dot product of the vectors. ||x|| and ||y|| represent the L2 norm of the vectors. The cosine similarity value range is [-1, 1]. The larger the value, the higher the semantic similarity between the vectors. Construct the IVFFlat index of Faiss: collect all vertex vectors, attribute vectors and edge vectors to form a vector set respectively. Determine the target number K of clustering, and reasonably select the K value according to the data scale and performance requirements. Use the k-means clustering algorithm to cluster the vertex vector set to obtain K cluster centers. Each cluster center represents a Voronoi unit for subsequent vector allocation.

[0069] Vectors are assigned to Voronoi cells. For each vertex vector in the index, its distance to all vertex cluster centers is calculated. Each vertex vector is assigned to the nearest Voronoi cell. For each attribute vector in the index, its distance to all attribute cluster centers is calculated. Each attribute vector is assigned to the nearest Voronoi cell. For each edge vector in the index, its distance to all edge cluster centers is calculated. Each edge vector is assigned to the nearest Voronoi cell. A Flat index is constructed within each Voronoi cell. For each vertex Voronoi cell, all vertex vectors within the cell are stored using the Flat index. For each attribute Voronoi cell, all attribute vectors within the cell are stored using the Flat index. For each edge Voronoi cell, all edge vectors within the cell are stored using the Flat index. A Flat index is an exact match index structure used for similarity calculation within a cell.

[0070] Similarity retrieval: Determine the Voronoi cell to which the query vector belongs. Given a query vector q, select the corresponding set of cluster centers according to its type (vertex, attribute, or edge). Calculate the distance between the query vector q and all cluster centers. Select the cluster center with the closest distance to determine the Voronoi cell to which the query vector q belongs. Perform similarity calculation within the Voronoi cell, and select the corresponding Flat index based on the Voronoi cell to which the query vector q belongs. In the selected Flat index, calculate the similarity between the query vector q and each vector in the cell. Similarity calculation can use metrics such as cosine similarity and Euclidean distance. Select the Top-K most similar vectors, and sort all the calculated similarities from large to small according to the similarity value. Select the top K vectors with the highest similarity as the retrieval result. Return the entities, attributes, or relationships corresponding to these K vectors as the similarity results of the query.

[0071] Index compression optimization: Determine the number of sub-indexes, and reasonably select the number of sub-indexes s according to the data scale and available GPU resources. The selection of the number of sub-indexes s requires a trade-off between compression efficiency and parallelism. Usually, a value that matches or slightly exceeds the number of GPUs is selected. Initialize the sub-index partitioning, and use the k-means++ algorithm to initialize s cluster centers as the initial partitioning of the sub-index. The k-means++ algorithm can better select the initial cluster centers, improve the quality of clustering and convergence speed. The specific steps are as follows: Randomly select a vector as the first cluster center. For the remaining vectors, calculate their shortest distance to the selected center. Select the next cluster center according to the square probability of the distance, and the vector with a longer distance has a higher probability of being selected as the new center. Repeat the above steps until s cluster centers are selected. Samples are assigned to sub-indexes. Using a heuristic sample allocation strategy, the three types of inverted index vector data, vertex, attribute, and edge, are evenly divided into s sub-indexes. For each vector, calculate its distance to the s cluster centers and assign it to the sub-index where the nearest cluster center is located. During the allocation process, try to keep the number of vectors contained in each sub-index equal to avoid uneven data distribution. Optimize the cluster centers of the sub-indexes. For each sub-index, iteratively execute the k-means clustering algorithm to optimize the cluster centers. In each iteration, the vectors in the sub-index are redistributed to the nearest cluster center, and the cluster center is updated to the mean of the vectors. The iterative process continues until the cluster center no longer changes significantly or the maximum number of iterations is reached. The cluster center is optimized by minimizing the mean square error (MSE) so that the vectors in each sub-index are as compact as possible.

[0072] GPU parallel processing: Create a GPU thread group, and create three groups of GPU threads for the three types of vector sub-indexes: vertex, attribute, and edge. Each group of GPU threads contains s threads, corresponding to s sub-indexes. GPU parallel compression of sub-indexes, assign s sub-indexes to corresponding s GPU threads respectively, to achieve parallel processing. In each GPU thread, PCA dimension compression is performed on the assigned sub-index: the mean vector of all k-dimensional vectors in the sub-index is calculated as the data center. The difference between all vectors in the sub-index and the mean vector is calculated to obtain the centered vector matrix X. The covariance matrix C = X^T*X of the centered vector matrix X is calculated. The covariance matrix C is decomposed by eigenvalue to obtain eigenvalues ​​and eigenvectors. Based on the preset variance contribution threshold η (such as 0.95), the first m eigenvectors with the largest contribution are selected to obtain the m-dimensional eigenvector matrix P after dimension compression. All vectors in the sub-index are projected onto the selected m eigenvectors to obtain the m-dimensional vector representation data Y = X*P after dimension compression. Update the compressed index. After the GPU thread completes the sub-index compression, it writes back the compressed m-dimensional vector representation data and updates the corresponding Faiss sub-index. Each GPU thread independently completes the compression and update of the sub-index, making full use of the parallel computing capabilities of the GPU.

[0073] Index update and query: Merge the compressed sub-indexes, merge the m-dimensional vector representation data compressed by all GPU threads, and update the original Faiss index. The merged index contains the compressed vector data of all sub-indexes, maintaining the original index structure and query method. Compressed representation of query vectors: For a new query vector q, first project it onto the selected m eigenvectors to obtain the m-dimensional compressed representation q'. The projection process is the same as the projection method when the sub-index is compressed, and the same eigenvector matrix P is used. Similarity retrieval of compressed index: In the compressed Faiss index, similarity retrieval is performed using the compressed representation q'. The retrieval process is the same as the retrieval method of the original index. By calculating the similarity between q' and the vector in the index, the Top-K most similar m-dimensional vectors are obtained. Map back to the original vector space: map the m-dimensional vectors in the retrieval results back to the original k-dimensional vector space. The mapping process is implemented by inverse transformation or searching the original vector. The Top-K original k-dimensional vectors most similar to the query vector q are obtained as the final retrieval results.

[0074] Finally, to achieve distributed storage and fast access to knowledge graph data, the compressed sub-index data is stored in a hybrid storage architecture of HBase and Redis. The m-dimensional sub-index vector data of vertices, attributes, and edges are obtained after GPU multi-threaded parallel compression in step 7. These data are low-dimensional vector representations after PCA dimension compression.

[0075] Importing HBase distributed database: Building multi-column index storage tables: Design and create table structures in HBase for storing compressed sub-index data. Each table represents a type of sub-index data, such as vertex tables, attribute tables, and edge tables. The column families of the table are used to divide the sub-index data of different partitions to improve the efficiency of data access and management. For example, you can create multiple column families for each table, such as "partition1" and "partition2", to store sub-index data of different partitions. Create three sub-index tables: Execute the command to create tables in HBase to create vertex tables, attribute tables, and edge tables respectively. The vertex table is used to store compressed vertex sub-index data, the attribute table is used to store compressed attribute sub-index data, and the edge table is used to store compressed edge sub-index data. Each table contains predefined column families for organizing and storing sub-index data of different partitions. Design the row key of the table: Design a unique row key for each sub-index data to quickly locate and access the corresponding data in the HBase table. The row key can use the unique identifier of the sub-index, such as the sub-index ID or hash value. The design of row keys should take into account the distribution characteristics and query patterns of data to achieve efficient data retrieval and access. Import sub-index data: Import the compressed vertex, attribute, and edge sub-index data into the corresponding HBase table respectively. Use HBase's data import tool or API to write the sub-index data into the table according to the organization of row keys and column families. During the import process, the sub-index data is mapped to the corresponding column family according to the consistent hash partitioning algorithm to achieve distributed storage of data.

[0076] Importing Redis distributed memory database: Create three Redis databases: Create three independent databases in Redis to store sub-index data of vertices, attributes, and edges. Each database has a unique database number, such as 0, 1, 2, etc. Different types of sub-index data are stored in different databases, which facilitates data isolation and management. Use hash data structure to store sub-indexes: In Redis, use hash data structure to store compressed sub-index data. Hash data structure stores data in the form of key-value pairs, where the key is the unique identifier of the sub-index and the value is the corresponding m-dimensional vector data. Hash data structure provides fast data access and query capabilities, suitable for storing and retrieving sub-index data. Import sub-index data: Import the compressed vertex, attribute, and edge sub-index data into the corresponding Redis databases respectively. Use Redis commands or APIs to use the unique identifier of each sub-index as the hash key and the m-dimensional vector data as the hash value to store. During the import process, batch import can be used to improve the efficiency of data writing. The compressed sub-index data is imported into the hybrid storage architecture of HBase and Redis. The row key design and column family organization of the HBase table realize the distributed storage and efficient access of sub-index data. The hash data structure of Redis provides fast data reading, writing and querying capabilities, which is suitable for storing frequently accessed hot sub-index data. This hybrid storage architecture combines the distributed storage capabilities of HBase and the caching capabilities of Redis, and can effectively support the storage, retrieval and question-answering of knowledge graph data. By storing different types of sub-index data in different HBase tables and Redis databases, data isolation and management are achieved, which improves the maintainability and scalability of the system.

[0077] Use the consistent hash partitioning algorithm to store sub-index data in the HBase table. Select an appropriate consistent hash function, such as MD5 or SHA-1, to calculate the hash value of the sub-index data. The consistent hash function should have good hashing properties and be able to evenly map the sub-index data to the hash space. Determine the hash space range of the consistent hash algorithm, usually using an integer space of 0 to 2^32-1 or 0 to 2^64-1. The size of the hash space should be large enough to accommodate all sub-index data and provide enough space for future data growth. Create a virtual hash ring in the hash space to map the column families of the HBase table. Treat each column family as a node on the hash ring and calculate its position on the hash ring based on the name or identifier of the column family. The consistent hash function can be used to hash the name of the column family to obtain its position on the hash ring. For each sub-index data to be stored, use the selected consistent hash function to calculate its hash value. Map the calculated hash value to the hash ring and find the first column family node that is greater than or equal to the hash value clockwise. Store the sub-index data in the column family of the HBase table corresponding to the column family node. Through the consistent hashing algorithm, sub-index data is evenly distributed to different column families to achieve partitioned storage of data. Each column family is responsible for storing a part of the sub-index data, and the distribution of data is determined by the mapping relationship of the hash ring. The consistent hashing algorithm can ensure the load balancing of data between column families and avoid excessive concentration or uneven distribution of data. When a new column family node needs to be added, it is only necessary to insert a new node on the hash ring and recalculate the hash value of a part of the sub-index data. The affected sub-index data will be migrated to the corresponding column family node according to the new hash mapping relationship, and other unaffected data will remain unchanged. The consistent hashing algorithm minimizes the overhead of data migration. Only a small amount of data needs to be migrated between column families, which improves the scalability of the system. When a sub-index data needs to be queried, its hash value is calculated using the same consistent hashing function. According to the hash value, the corresponding column family node is found on the hash ring, and the required sub-index data is retrieved from the column family. The consistent hashing algorithm provides an efficient data routing and query mechanism, avoiding the centralized query bottleneck.

[0078] Identify and store hot sub-indexes, and count the access frequency of sub-indexes: Maintain access frequency counters: Create an access frequency counter for each sub-index to record the number of times the sub-index is accessed within a certain time window. You can use Redis's string data structure to store the access frequency counter, with the sub-index's unique identifier as the key and the number of accesses as the value. Update access frequency: When a new access request arrives, find the corresponding access frequency counter based on the requested sub-index identifier. Atomically increase the value of the access frequency counter by 1, indicating that the number of accesses to the sub-index is increased by 1. You can use Redis's INCR command to atomically increase the value of the counter. Deletion of expired access records: In order to maintain the number of accesses within a certain time window, it is necessary to delete expired access records regularly. You can set an expiration time for each access frequency counter to indicate the validity period of the access record. Use Redis's expiration mechanism to automatically delete the counter after it reaches the expiration time, or clean up expired access records regularly through scheduled tasks.

[0079] Mark hot sub-indexes: Set frequency threshold: Preset a frequency threshold Fh to determine whether a sub-index is a hot sub-index. The frequency threshold Fh can be adjusted according to the system's access pattern and performance requirements to balance the number of hot sub-indexes and cache efficiency. Scan access frequency regularly: Through regular scanning tasks, traverse the access frequency counters of all sub-indexes. For each sub-index, compare its access frequency with the frequency threshold Fh. If the access frequency is higher than Fh, the sub-index is marked as a hot sub-index; otherwise, it is marked as a cold sub-index. Update hot sub-index mark: According to the scan results, update the hot sub-index mark status of each sub-index. A hot sub-index mark field can be maintained for each sub-index in Redis, using a Boolean value to indicate whether it is a hot sub-index. Set the hot sub-index mark of the sub-index with an access frequency higher than Fh to true, and set the hot sub-index mark of the sub-index with an access frequency lower than Fh to false. Store hot sub-indexes to Redis: Read hot sub-index data: For data marked as hot sub-indexes, read the corresponding complete sub-index data from HBase. Use HBase's read interface to obtain the sub-index data stored in HBase according to the row key and column family of the sub-index. Write hot sub-indexes to Redis: Write the read hot sub-index data to Redis's hash data structure. Use the sub-index's unique identifier as the hash key and store the complete sub-index data as the hash value. Use Redis's HSET command to write the hot sub-index data to the corresponding Redis database. Keep cold sub-index data: For data marked as cold sub-indexes, keep it in HBase's column family storage and do not need to write it to Redis. Cold sub-index data continues to remain in HBase and is accessed and retrieved through HBase's read interface.

[0080] Data synchronization: Synchronize data regularly: In order to ensure the consistency between the hotspot sub-index data in Redis and the data in HBase, data synchronization needs to be performed regularly. You can use a scheduled task or trigger mechanism to synchronize the hotspot sub-index data in Redis back to HBase regularly. Synchronize updated data: For the hotspot sub-index data that has been updated in Redis, the updated data needs to be synchronized back to HBase. By comparing the data versions or timestamps in Redis and HBase, determine the data that needs to be synchronized. Use the write interface of HBase to write the updated hotspot sub-index data in Redis back to the corresponding rows and column families of HBase. Synchronize deleted data: For the hotspot sub-index data that has been deleted in Redis, it needs to be deleted from HBase accordingly. By regularly scanning the hotspot sub-index data in Redis, identify the data that has been deleted. Use the delete interface of HBase to delete these deleted hotspot sub-index data from the corresponding rows and column families of HBase.

[0081] Knowledge retrieval and question-answering, prioritize hotspot sub-index data from Redis: Receive knowledge retrieval and question-answering requests: When a user initiates a knowledge retrieval or question-answering request, the system receives and parses the request and extracts the key information of the query. Convert the query information into a query vector for subsequent similarity calculation and matching. Obtain hotspot sub-index data from Redis: Based on the query vector, prioritize hotspot sub-index data related to the query from Redis. Use Redis's hash data structure to quickly locate and read the corresponding hotspot sub-index data through the unique identifier of the sub-index. Calculate similarity: Calculate the similarity between the query vector and the hotspot sub-index data in Redis to obtain candidate results. Common similarity metrics such as cosine similarity and Euclidean distance can be used for similarity calculation. For each hotspot sub-index, calculate the similarity score between it and the query vector. Sort candidate results: Sort the candidate results obtained in Redis according to the similarity score. Select the top-K sub-indexes with the highest similarity scores as candidate results, where K can be set according to system requirements and performance considerations.

[0082] Supplementary acquisition of cold sub-index data from HBase: Determine whether supplementary search is needed: If the hot sub-index data in Redis is not sufficient to meet the query requirements, or the similarity score is low, it is necessary to obtain cold sub-index data from HBase for supplementary search. A similarity threshold or result number threshold can be set to determine whether supplementary search is needed. Obtain cold sub-index data from HBase: Obtain cold sub-index data related to the query from HBase based on the query vector. Use the HBase read interface to read the cold sub-index data stored in HBase based on the row key and column family of the sub-index. Calculate similarity: Calculate the similarity between the query vector and the cold sub-index data in HBase to obtain supplementary candidate results. The similarity calculation method is consistent with the calculation method in Redis, and the same similarity measurement method is used. Merge candidate results: Merge the supplementary candidate results obtained from HBase with the candidate results in Redis. Re-sort the merged candidate results to obtain the final Top-K most similar sub-indexes.

[0083] Merge and sort the results: Merge the candidate results from Redis and HBase: Merge the candidate results from Redis and HBase to form a complete set of candidate results. Deduplication is required during merging to ensure that the same sub-index does not appear multiple times. Re-sort: Re-sort the merged candidate result set and arrange them from high to low based on the similarity score. Select the sorted Top-K sub-indexes as the final search results.

[0084] Get the original knowledge graph data: Get the original data based on the search results: Get the corresponding original knowledge graph data from HBase or Redis based on the retrieved Top-K sub-indexes. If the original data corresponding to the sub-index is stored in HBase, use the HBase read interface to obtain it. If the original data corresponding to the sub-index is stored in Redis, read it directly from the hash data structure of Redis. Return the original data: Return the obtained original knowledge graph data to the requester for subsequent knowledge reasoning and question-answering processing. The original data can include basic elements of the knowledge graph such as entities, relationships, attributes, and related contextual information.

Claims

1. A method for constructing a knowledge graph based on a large language model and a vector library, comprising: Step 1: Use the pre-trained RoBERTa model to perform word segmentation, part-of-speech tagging and semantic analysis on the collected natural language text to obtain the semantic representation vector of the natural language text; Input the semantic representation vector into the Seq2Seq framework of the BiLSTM-Attention-CRF network structure to obtain the named entity; Step 2: Use the named entities identified in step 1 to build an initial knowledge graph: Use the identified named entities as entity nodes in the knowledge graph; Use the pre-trained syntactic dependency analysis model to obtain syntactic dependency tree data; In the syntactic dependency tree, obtain the shortest dependency path between named entity pairs; Based on the obtained shortest dependency path, a relationship template is constructed; using the constructed relationship template, the relationship between the named entity pairs is extracted to obtain entity relationship data; The extracted entity relationships are used as the relationship edges of the knowledge graph, and the corresponding head entity nodes and tail entity nodes are connected to construct the knowledge graph data containing entity nodes, attributes and relationship edges; Step 3: Fuse the knowledge graph constructed in step 2 with the relational database of the preset field to generate a fused knowledge graph; Use string matching algorithms to obtain pairs of entities with the same name and attributes in the knowledge graph and relational database; For each pair of entities with the same name, calculate the semantic similarity of the corresponding attributes; When the semantic similarity is higher than a preset threshold, the attributes of the corresponding entity in the relational database are added to the corresponding entity node in the knowledge graph; When the semantic similarity is lower than the preset threshold, the corresponding entity node in the knowledge graph is renamed or a distinguishing mark is added; Step 4: Structure the triple representation of the fused knowledge graph: Using dependency syntactic analysis, we can obtain the named entities, corresponding attributes, and entity relationship data between named entities in the fused knowledge graph; Based on the acquired named entities, attributes and entity relationship data, a structured triple representation data of the fused knowledge graph is constructed; Also includes: Step 5: Convert the structured triples in the knowledge graph into low-dimensional dense vector representation; Step 6: Construct a similarity-based index for the low-dimensional vector representation of the knowledge graph: Calculate the cosine similarity between two vectors in the low-dimensional vector representation of the knowledge graph; Use the IndexIVFFlat algorithm in the Faiss similarity search library to index the vectors of vertices, attributes, and edges, and construct an inverted index based on cosine similarity; The IndexIVFFlat algorithm in the Faiss similarity search library is used to index the vectors of vertices, attributes, and edges, and to construct an inverted index based on cosine similarity, including: The K-means clustering algorithm is used to cluster all vectors in the index, and the vector space is divided into K Voronoi cells, each of which corresponds to a cluster center. For each vector in the index, calculate its distance to all cluster centers and assign the vector to the Voronoi cell with the nearest cluster center. For each Voronoi cell, create a Flat index, which contains all the vectors in the cell; During retrieval, for a given query vector, the similarity between it and each vector in the cell is calculated through the Flat index; Step 7: Optimize and compress the inverted index data obtained in step 6; Step 8: Distribute and manage the heat of the sub-index compressed in step 7.

2. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 1, characterized in that: Also includes: Step 5: Convert the structured triples in the knowledge graph into low-dimensional dense vector representations: The structured triples in the fused knowledge graph are used as input data for the TransE model; The TransE algorithm is used to learn vectorized representations of structured triples, mapping named entities in triples to vertex vectors; mapping attributes in triples to attribute vectors; and mapping entity relationships in triples to edge vectors. Through the optimization solution of the TransE model, each triple (h, r, t) satisfies the vector space constraint of h+r≈t, and a low-dimensional vector representation of all named entities, attributes and relationships in the fusion knowledge graph is obtained.

3. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 1, characterized in that: Also includes: Step 7: Optimize and compress the inverted index data obtained in step 6: Set the number of sub-indexes s, and use the k-means++ algorithm to initialize s cluster centers; Through the heuristic sample allocation strategy, the three types of inverted index vector data, namely vertex, attribute and edge, are evenly divided into s sub-indexes; For each sub-index, the k-means clustering algorithm is iteratively executed to optimize the cluster center by minimizing the mean square error, and s sub-index data containing equal vectors are obtained; For the three types of vector sub-indexes, vertex, attribute, and edge, three groups of GPU threads are created respectively, each group contains s threads; the s sub-indexes are respectively assigned to the corresponding s GPU threads; In each GPU thread, the mean vector and covariance matrix of all k-dimensional vectors in the sub-index are calculated; Perform eigenvalue decomposition on the covariance matrix, select the first m eigenvectors with the largest contribution based on the preset variance contribution threshold η, and obtain the m-dimensional eigenvector after dimension compression; Project all vectors in the sub-index onto the selected m eigenvectors to obtain the m-dimensional vector representation data after dimension compression; After the GPU thread completes the sub-index compression, it writes the compressed m-dimensional vector representation data back to update the Faiss index.

4. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 3, characterized in that: Also includes: Step 8: Distribute and manage the heat of the sub-index compressed in step 7: Obtain the three types of m-dimensional sub-index vector data of vertices, attributes, and edges after GPU multi-threaded parallel compression in step 7; Import the HBase-based distributed database to build a multi-column index storage table for distributed storage; import the Redis distributed memory database for storage and retrieval of hot sub-indexes; Using the consistent hash partitioning algorithm, the three types of m-dimensional sub-index vector data of vertices, edges, and attributes are hash-mapped to the corresponding column families of the HBase table for storage; Count the historical access frequency of each sub-index and update the frequency counter data based on the sliding time window; Based on the preset frequency threshold Fh, the sub-index with access frequency higher than Fh is marked as a hot sub-index, and the sub-index with access frequency lower than Fh is marked as a cold sub-index; Store the hot sub-index data in Redis and the cold sub-index data in HBase; When performing knowledge retrieval and question-answering requests, the hot sub-index data is first obtained from Redis for calculation and sorting, and then the cold sub-index data is obtained from HBase for secondary retrieval and calculation.

5. The method for constructing a knowledge graph based on a large language model and a vector library according to any one of claims 2 to 4, characterized in that: In step 1, the BiLSTM-Attention-CRF network structure includes a BiLSTM encoding layer, an attention mechanism layer, and a CRF decoding layer; The BiLSTM encoding layer receives the semantic representation vector output by the RoBERTa model, learns the context information of the input vector through the bidirectional LSTM, and obtains the feature vector representation containing the context information; The attention mechanism layer adjusts the encoding features of the BiLSTM encoding layer by calculating the attention weights between the feature vector representation and the predefined named entity type embedding vector; The CRF decoding layer makes the best sequence labeling decision globally based on the adjusted feature vector representation and obtains the final named entity recognition result by modeling the Markov transition constraints between named entity labels.

6. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 5, characterized in that: In step 2, the relationship between entity nodes is obtained through relationship template matching as the relationship edge in the knowledge graph, including: For the named entity pairs obtained in step 1, a syntactic dependency tree is obtained using a pre-trained syntactic dependency analysis model; In the syntactic dependency tree, obtain the shortest dependency path between named entity pairs; Input the shortest dependency path data into the graph attention network to learn the distributed representation features of the path; Based on the distributed representation features of the shortest dependency path, a trigger template for relationship extraction is constructed; Using the constructed relationship template, relationships are extracted for named entity pairs, and the extraction results are used as the relationship edge data of the knowledge graph. The corresponding head entity node and tail entity node are connected to construct knowledge graph data containing entities and relationships.

7. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 6, characterized in that: In the syntactic dependency tree, obtain the shortest dependency path between named entity pairs, including: Traversing the word nodes and dependency data of the syntactic dependency tree, and converting the syntactic dependency tree into directed graph data according to the word nodes and dependency relationships; According to the directed graph data, take the head entity node as the starting point and the tail entity node as the end point, initialize the distance of each node and the predecessor node data, and add the starting node to the priority queue; Use Dijkstra algorithm to iteratively calculate the shortest distance from the starting point to each node in the directed graph, and record the node predecessor until the end point is reached; According to the node predecessor data obtained by the Dijkstra algorithm, backtracking starts from the tail entity node and constructs the shortest directed path data from the head entity node to the tail entity node as the shortest dependency path.

8. The method for constructing a knowledge graph based on a large language model and a vector library according to claim 7, characterized in that: Based on the obtained shortest dependency path, a trigger template for entity relationship extraction is constructed, including: Obtain word node data of the shortest dependency path and extract part-of-speech tagging features of the word nodes; obtain directed edge data of the shortest dependency path and extract dependency relationship type features of the directed edges; According to the part-of-speech tagging and dependency type features, the shortest dependency path data is converted into graph structure data; Input graph structure data into the graph attention network to learn the distributed representation features of the shortest dependency path; Based on the distributed representation features of the shortest dependency path, a trigger template for entity relationship extraction is constructed.

Images