A knowledge graph construction and query method, an electronic device, and a storage medium

Abstract

The application discloses a knowledge graph construction and query method, an electronic device and a storage medium. A to-be-processed text is acquired and a source level identifier is associated. A text segment is obtained based on the to-be-processed text, and an entity detail object and a relationship detail object are generated. An entity aggregation object is generated based on the entity detail object, and a relationship aggregation object is generated based on the relationship detail object. The entity aggregation object and the relationship aggregation object are written into a graph database, and the text segment, the entity aggregation object and the relationship aggregation object are subjected to vectorization processing to be written into a vector index library. In response to a query request, source filtering is performed in at least two retrieval processing stages. In response to an update request or a deletion request, a target object is subjected to deletion, reservation or reconstruction based on a remaining source corresponding detail object based on a relationship between a source set associated with the target object and a to-be-processed source set. The method can improve source traceability, query isolation and maintenance consistency of a multi-source knowledge graph.

Description

Technical Field

[0001] This application relates to the field of knowledge graph technology, and in particular to a knowledge graph construction and query method, electronic device and storage medium. Background Technology

[0002] With the continuous accumulation of information such as internal enterprise documents, business rules, product data, operation manuals, and question-and-answer data, how to effectively organize knowledge from multi-source heterogeneous texts and use it for retrieval and question answering has become an important issue in the field of intelligent information processing. Knowledge graph technology can represent knowledge in the form of entities, attributes, and relationships between entities in a structured way, and is therefore widely used in scenarios such as enterprise knowledge management, intelligent retrieval, and intelligent question answering.

[0003] In existing technologies, the original text is typically preprocessed and segmented to obtain multiple text fragments. Then, entities, relationships, and related descriptive information are extracted from these fragments using information extraction models or large language models to form nodes and edges in a knowledge graph. To improve semantic retrieval capabilities during queries, vector representations are often generated for text fragments, entity descriptions, and relationship descriptions, and corresponding indexes are established. During the query phase, relevant entities, relationships, or text fragments are typically retrieved based on the user's question, and the query results or answers are output by combining the relationships within the knowledge graph.

[0004] However, existing technologies still have some shortcomings in enterprise-level multi-source knowledge management scenarios. First, the same entities or relationships may be extracted from texts from different sources. After merging these objects, existing solutions often struggle to manage the multiple sources in a fine-grained manner, thus affecting subsequent source tracing and object maintenance. Second, in scenarios with multiple knowledge bases or access isolation requirements, existing solutions typically focus more on filtering the final query results. For intermediate processing steps such as candidate entity determination, candidate relationship determination, graph expansion, and source text backtracking, there is a lack of more consistent source constraint mechanisms, which can easily lead to cross-source expansion or unclear source issues during the query process.

[0005] Furthermore, when a source text is updated or deleted, related entities and relationships should not necessarily be deleted entirely, but may still remain associated with other source texts. In such multi-source sharing scenarios, existing technologies often struggle to accurately determine whether related objects should be deleted, retained, or updated, easily leading to problems such as graph data residue, distorted source information, or inaccurate object state maintenance. Meanwhile, knowledge graph data and vector index data are typically maintained separately. If there is a lack of effective linkage between the two after objects are updated, deleted, or adjusted, inconsistencies between graph data and index data can easily arise, affecting the accuracy and interpretability of query results.

[0006] Therefore, in multi-source knowledge management scenarios, improving the traceability of source information, the isolation and control capabilities of the query process, and the maintenance consistency after object updates and deletions have become urgent technical problems to be solved in this field. Summary of the Invention

[0007] This application provides a knowledge graph construction and query method, an electronic device, and a computer-readable storage medium to at least solve the problems of insufficient and imperfect source management in existing knowledge graph construction and query technologies under multi-source knowledge management scenarios.

[0008] This application provides a knowledge graph construction and query method, comprising: acquiring text to be processed and associating the text with a source hierarchy identifier, wherein the source hierarchy identifier includes at least two of a dataset identifier, a source group identifier, and a text identifier; fragmenting the text to be processed to obtain text fragments, and storing the text fragments in association with the source hierarchy identifiers; generating entity detail objects and relation detail objects based on the text fragments, wherein the entity detail objects and the relation detail objects respectively include a direct source set and a complete source path set; generating entity aggregation objects based on multiple entity detail objects, and generating relation aggregation objects based on multiple relation detail objects; and configuring the entity aggregation objects and the relation aggregation objects... The text fragments, entity aggregation objects, and relation aggregation objects are written into a graph database and vectorized to be written into a vector index library. In response to a query request, source filtering is performed at least twice in the retrieval processing stages, based on at least one of the source hierarchy identifier, the direct source set, and the complete source path set. In response to an update or deletion request for a target source, a set of sources to be processed is determined, and based on the relationship between the source set associated with the target object and the set of sources to be processed, the target object is deleted, retained, or reconstructed based on the detailed objects corresponding to the remaining sources. The target object is either an entity aggregation object or a relation aggregation object. The graph data and vector index data corresponding to the target object are updated in conjunction with this process.

[0009] Furthermore, the complete source path in the complete source path set is formed by combining the dataset identifier, the source group identifier, and the text identifier in a preset order; when the dataset identifier, the source group identifier, or the text identifier contains a preset separator, the corresponding identifier is escaped or encoded before forming the complete source path.

[0010] Furthermore, associating and storing the text fragment with the source hierarchy identifier includes: writing the text fragment into a text fragment table, wherein the text fragment table records at least the fragment identifier, text identifier, dataset identifier, source group identifier, and fragment content; wherein the records in the text fragment table are used for entity relation generation, source text lookup, and text fragment vectorization processing.

[0011] Further, generating entity detail objects and relationship detail objects based on the text fragment includes: performing multiple rounds of entity relationship generation on the text fragment; performing structured parsing on the results of the multiple rounds of entity relationship generation to obtain the entity detail objects and the relationship detail objects; wherein, the entity detail objects include entity name, text identifier, direct source set, complete source path set, and descriptive contribution information; the relationship detail objects include relationship start point identifier, relationship end point identifier, text identifier, direct source set, complete source path set, weight contribution information, keyword contribution information, and descriptive contribution information.

[0012] Furthermore, multiple rounds of entity relation generation are performed on the text fragment, including: performing a first round of entity relation generation to identify entities and relations in the text fragment; performing a second round of entity relation generation to supplement missing entities, missing relations, or descriptive information; before the start of subsequent rounds of the second round, determining whether to continue generation based on the generated entity relation results; if the determination result indicates that there are still entities or relations to be generated, then entity relation generation continues; if the determination result indicates that there are no entities or relations to be generated, then entity relation generation ends.

[0013] Furthermore, generating an entity aggregation object based on multiple entity detail objects includes: merging multiple entity detail objects according to entity names; deduplicating and concatenating descriptive contribution information in multiple entity detail objects; deduplicating and aggregating the direct source set in multiple entity detail objects; and deduplicating and aggregating the complete source path set in multiple entity detail objects to generate the entity aggregation object.

[0014] Furthermore, generating a relationship aggregation object based on multiple relationship detail objects includes: merging multiple relationship detail objects according to the relationship start point identifier, relationship end point identifier, and relationship type; deduplicating and concatenating the relationship description contribution information and keyword contribution information in multiple relationship detail objects; deduplicating and aggregating the direct source set and complete source path set in multiple relationship detail objects; and accumulating or recalculating the weight contribution information in multiple relationship detail objects to generate the relationship aggregation object.

[0015] Furthermore, the at least two retrieval processing stages include at least two of the following: entity recall stage, relation recall stage, graph expansion stage, neighbor expansion stage, and source text lookup stage; wherein, in the entity recall stage, source filtering is performed based on the direct source set and complete source path set of the entity aggregation object; in the relation recall stage, source filtering is performed based on the direct source set and complete source path set of the relation aggregation object; in the graph expansion stage or the neighbor expansion stage, source filtering is performed based on the source information corresponding to the expanded object; and in the source text lookup stage, source filtering is performed based on the source hierarchy identifier corresponding to the text fragment.

[0016] Furthermore, when performing source filtering, if only some sources of an entity aggregation object or relationship aggregation object satisfy the visibility condition, then based on the detail objects corresponding to the sources that satisfy the visibility condition, at least one visibility attribute of the entity aggregation object or relationship aggregation object is re-determined; the visibility attribute includes at least one of description, keyword, weight, direct source set, and complete source path set.

[0017] Further, responding to a query request includes: performing semantic analysis on the query request to obtain high-level keywords and low-level keywords; recalling candidate entities in the entity vector index based on the low-level keywords, and determining the connectivity of the candidate entities based on the graph database; recalling candidate relations in the relation vector index based on the high-level keywords; performing graph expansion and source text lookup based on the candidate entities and / or the candidate relations; merging, deduplicating, and reordering the recall results and lookup results to obtain entity results, relation results, and source results.

[0018] Furthermore, based on the relationship between the source set associated with the target object and the source set to be processed, the target object is deleted, retained, or reconstructed based on the detailed objects corresponding to the remaining sources. This includes: if the source set associated with the target object consists only of sources in the source set to be processed, then the target object is deleted, and the graph data and vector index data corresponding to the target object are also deleted; if the source set associated with the target object includes both sources in the source set to be processed and other sources, then the target object is retained, the source information corresponding to the source set to be processed is stripped, and the aggregation attributes of the target object are reconstructed based on the detailed objects corresponding to the remaining sources; if the source set associated with the target object and the source set to be processed have no intersection, then the target object remains unchanged.

[0019] Furthermore, reconstructing the aggregate attributes of the target object based on the detailed objects corresponding to the remaining sources includes: when the target object is an entity aggregate object, redetermining at least one of the entity description, direct source set, and complete source path set based on the entity detailed objects corresponding to the remaining sources; when the target object is a relationship aggregate object, redetermining at least one of the relationship description, keywords, weight, direct source set, and complete source path set based on the relationship detailed objects corresponding to the remaining sources.

[0020] Furthermore, the linked update of graph data and vector index data corresponding to the target object includes: generating the same operation identifier corresponding to the graph database write operation and the vector index database write operation; recording the graph write status, vector write status, overall status, and number of retries in the processing log table; and performing a retry or compensation recovery based on the processing log table when either the graph write status or the vector write status indicates a write failure.

[0021] Furthermore, the processing log table also records the object primary key and the source field; when performing the retry or compensation recovery, the affected object is relocated based on the object primary key and the source field, and the deletion, source stripping, or reconstruction operations are repeated.

[0022] This application also provides an electronic device, including: at least one processor; a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device performs the knowledge graph construction and query method described in any of the above technical solutions.

[0023] This application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the knowledge graph construction and query method described in any of the above technical solutions.

[0024] Compared with existing technologies, the above technical solution has at least the following beneficial effects: By associating source hierarchy identifiers with the text to be processed when it enters the processing chain, and ensuring that text fragments, entity detail objects, relationship detail objects, entity aggregation objects, and relationship aggregation objects all maintain a correspondence with source information, a source traceability chain from the original text to detail objects and then to aggregation objects can be formed during the knowledge graph construction stage. Furthermore, entity aggregation objects and relationship aggregation objects are generated based on detail objects, and a direct source set and a complete source path set are stored, enabling accurate identification of the source composition even when the same entity or relationship is formed by contributions from multiple sources, thereby improving source traceability and object maintenance accuracy in multi-source knowledge management scenarios.

[0025] The above-mentioned technical solutions also have the following advantages: During the query process, source filtering can be performed in multiple retrieval processing stages such as entity retrieval, relationship retrieval, graph expansion, neighbor expansion, and source text lookup, avoiding the problem of intermediate expanded objects propagating across sources due to filtering only in the final output stage; In update or deletion scenarios, the relationship between the source set associated with the target object and the source set to be processed can be used to determine whether to delete, retain, or reconstruct the detailed object corresponding to the remaining source, thereby avoiding the accidental deletion of shared objects or attribute distortion after source stripping; At the same time, by processing the log table to record the write status of the graph database and vector index library, and performing retries or compensation recovery in case of anomalies, the consistency maintenance capability between graph data and vector index data can be improved. Attached Figure Description

[0026] The present application will be further described below with reference to the accompanying drawings and embodiments: Figure 1 A schematic diagram illustrating the overall process of a knowledge graph construction and query method provided in this application; Figure 2 A schematic diagram of the knowledge graph construction process provided for this application; Figure 3 This is a schematic diagram of the graph retrieval and hybrid search process provided in this application; Figure 4 A schematic diagram of the multi-data plane permission filtering process provided in this application; Figure 5 A schematic diagram of the source-based deletion, retention, and reconstruction process provided for this application; Figure 6 A schematic diagram illustrating the data relationship between the aggregation table and the detail table provided in this application; Figure 7 A schematic diagram of the electronic device structure provided in this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and beneficial effects of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0028] In this application, terms such as "first" and "second" are used only to distinguish different objects and do not indicate limitations on order, quantity, or importance. Expressions such as "comprising," "including," and "having" are open-ended, indicating that they include not only the explicitly listed technical features but also other technical features not explicitly listed but capable of achieving the corresponding function. "Based on" can mean entirely, partially, or in combination with other factors. "At least one" can mean one or more.

[0029] In this application, the dataset identifier is used to represent a unique identifier for a knowledge base or dataset; the source group identifier is used to represent a unique identifier for a collection, directory, or file group under the dataset; and the text identifier is used to represent a unique identifier for the original text record or text slice object. The direct source set is used to represent the set of text identifiers for the direct source text to which an entity or relation belongs; and the complete source path set is used to represent the complete source path set formed by the combination of the dataset identifier, source group identifier, and text identifier. For example, the complete source path can be encoded using a three-segment structure of "dataset identifier - source group identifier - text identifier". When the original identifier contains a preset separator or connector, the original identifier can be escaped or encoded before writing to avoid ambiguity during subsequent parsing.

[0030] In this application, the text fragment table is used to store fragment-level raw text data; the entity detail table is used to store entity-level detail objects extracted from text fragments; the relation detail table is used to store relation-level detail objects extracted from text fragments; the entity aggregation table is used to store aggregate objects merged by entity dimension; the relation aggregation table is used to store aggregate objects merged by relation dimension; and the processing log table is used to record the processing status, number of retries, and overall status during the dual-write process of the graph database and vector index database, so as to perform retries and compensation recovery in case of anomalies.

[0031] In this application, "system" can refer to a computer system, server, electronic device, software platform, or processing environment formed by the cooperation of processors, memory, databases, and program modules used to execute the knowledge graph construction and query methods of this application. "System" is used to describe the executing entity of each processing step and is not limited to an entity system with a fixed hardware structure or fixed module division. Accordingly, expressions such as "system acquisition," "system generation," "system writing," and "system update" in this document can be understood as one or more processors executing corresponding program instructions to complete the corresponding processing.

[0032] The following combination Figures 1-7 The embodiments of this application will be described in detail below. In the following embodiments, "system" refers to a computer system, server, electronic device, or software processing platform that executes the knowledge graph construction and query method of this application, which can execute program instructions in memory through one or more processors to implement the following steps.

[0033] like Figure 1As shown, the knowledge graph construction and query method provided in this embodiment may include: S1, obtaining the text to be processed and associating it with the source level identifier when the text enters the processing chain; S2, fragmenting the text to be processed and generating entity detail objects and relation detail objects based on the text fragments; S3, generating entity aggregation objects and relation aggregation objects based on the entity detail objects and relation detail objects, and writing them into the graph database and vector index database respectively; S4, after receiving a query request, performing entity retrieval, relation retrieval, graph expansion, source text lookup and result sorting based on different levels of search keywords; S5, when text is added, updated or deleted, performing deletion, retention or reconstruction of the aggregation objects according to the source set relationship, and linking to maintain the consistency between the graph data and the vector index data.

[0034] In one implementation, such as Figure 2 As shown, at the beginning of the knowledge graph construction process, the system first acquires the text to be processed. This text can originate from business knowledge text, question-and-answer text, explanatory documents, rule documents, the segmentation results of uploaded documents, or other unstructured text content. The system can read the text from data tables in the business database, or trigger the construction process when a user uploads a document, imports knowledge entries, or adds a new text record.

[0035] When text enters the processing chain, the system associates it with a source hierarchy identifier. This source hierarchy identifier can include at least a dataset identifier, a source group identifier, and a text identifier. The dataset identifier indicates the dataset to which the knowledge belongs, the source group identifier indicates the grouping range within the dataset, and the text identifier represents the specific text object. By associating the source hierarchy identifier at the very beginning of the text processing chain, subsequent text fragments, entity objects, relational objects, graph nodes, graph edges, and index objects generated from that text can all maintain a correspondence with the same source system.

[0036] In one implementation, the dataset identifier can be generated using a universally unique identifier or a unique number already existing on the business side. To facilitate unified parsing and comparison, the aforementioned dataset identifier, source group identifier, and text identifier can be uniformly converted into string format and pre- and post-whitespace cleaned. When the original identifier in the external system contains preset delimiters or hierarchical connectors, the system can perform escaping or encoding processing on the original identifier before writing to avoid ambiguity when splitting the complete source path later.

[0037] After binding the source hierarchy identifier, the system performs fragmentation processing on the text to be processed. Before fragmentation, preprocessing such as text cleaning, special character normalization, paragraph segmentation, and line break adjustment can be performed. The preprocessed text can be segmented into multiple text fragments according to preset length, sentence boundaries, paragraph boundaries, or semantic integrity. Each text fragment can inherit the source hierarchy identifier of its parent text.

[0038] In one implementation, the segmented text fragments are written to a text fragment table. The text fragment table can store at least fragment identifiers, text identifiers, dataset identifiers, source group identifiers, and fragment content. Thus, each record in the text fragment table stores not only the text content but also its dataset, source group, and original text. The fragment content in the text fragment table can serve as input for entity relation generation, and also as the basis for subsequent source text lookup, evidence fragment output, and text fragment vectorization processing.

[0039] In one implementation, the system invokes a large language model to perform multiple rounds of entity-relation generation on the text fragment. The first round of generation can be used to identify more explicit entities and relations in the text fragment. The second round of generation can be used to supplement entities, relations, or descriptive information not identified in the first round. Subsequent rounds can continue execution if it is determined that new entities or relations still exist, until a stopping condition is met. The output of the large language model may include entity name, entity type, entity description, relation start point, relation end point, relation description, keywords, and weights. To facilitate subsequent program parsing, an output format can be preset, outputting entity records and relation records separately through a fixed field order and a fixed delimiter structure.

[0040] It should be noted that this application does not limit the specific large language model used, nor does it limit the specific prompt word template. The system improves the completeness of entity relationship discovery through multi-round progressive generation, and retains the corresponding source before the generation results of each round enter the structured parsing, so that subsequent aggregation, filtering, and deletion rollback all have traceable evidence.

[0041] After obtaining the output of the large language model, the system performs structured parsing on the output to generate entity detail objects and relation detail objects. Figure 6 In the text, the solid arrow pointing from the text fragment table to the entity detail table and the relation detail table indicates that both the entity detail object and the relation detail object are generated by extracting text fragments. Figure 6 The solid arrows in the diagram are used to represent the main business data flow, namely the flow of object generation, transformation, merging, writing, and vectorization.

[0042] In one implementation, each record in the entity detail table may include at least a detail identifier, entity name, text identifier, direct source set, complete source path set, and descriptive contribution information. Each record in the relationship detail table may include at least a detail identifier, relationship start identifier, relationship end identifier, text identifier, direct source set, complete source path set, weight contribution information, keyword contribution information, and descriptive contribution information. The entity detail table and relationship detail table are used to store detailed contribution relationships, that is, to store "which text fragment contributed to which entity, which relationship, which descriptions, which keywords, and how much relationship weight."

[0043] In one implementation, the direct source set can store a set of text identifiers that directly represent the sources of entities or relationships, while the complete source path set can store a set consisting of dataset identifiers, source group identifiers, and text identifiers. For each entity detail object or relationship detail object generated from a text fragment, the system can construct a complete source path based on the source hierarchy identifier of the current text fragment. For example, a complete source path can be represented as "dataset identifier - source group identifier - text identifier". Multiple source paths can be concatenated using a preset separator, and deduplication can be performed.

[0044] In one implementation, the system generates an entity aggregation object based on multiple entity detail objects and stores the entity aggregation object in an entity aggregation table. Figure 6 In the diagram, the solid arrow pointing from the entity detail table to the entity aggregation table indicates that aggregation is performed by entity name. This means the system can use entity name as the basis for determining entity aggregation objects. When the newly generated entity detail object has the same entity name as an existing entity aggregation object, they are considered to represent the same entity and are merged; when the entity names are different, a new entity aggregation object is created.

[0045] During entity merging, the system can deduplicate and concatenate descriptive contribution information from multiple entity detail objects, deduplicate and aggregate direct source sets from multiple entity detail objects, and deduplicate and aggregate complete source path sets from multiple entity detail objects. If entities with the same name have different entity types, non-unknown types can be retained first, or the final entity type can be determined according to a preset type priority. In this way, the entity aggregation object can retain both the aggregated entity description and the multiple direct and complete source paths that formed the entity.

[0046] In one implementation, the system generates a relation aggregation object based on multiple relation detail objects and stores the relation aggregation object in a relation aggregation table. Figure 6In the diagram, the solid arrow pointing from the relationship detail table to the relationship aggregation table indicates that merging and aggregation are performed based on the relationship start identifier, relationship end identifier, and relationship type. The system can use the relationship start identifier, relationship end identifier, and relationship type as the criteria for determining the relationship aggregation object. When multiple relationship detail objects have the same relationship start identifier, relationship end identifier, and relationship type, they can be considered to correspond to the same relationship and merged.

[0047] During the relation merging process, the system can deduplicate and concatenate relation description contribution information and keyword contribution information from multiple relation detail objects, deduplicate and aggregate direct source sets and complete source path sets from multiple relation detail objects, and accumulate or recalculate the weight contribution information from multiple relation detail objects. If a relation detail object does not contain valid weight contribution information, a preset default weight can be used. Therefore, the relation aggregation object can reflect the common contribution of multiple source texts to the same relation, and can represent the stability or intensity of the relation across multiple source texts through weights.

[0048] After generating entity aggregation objects and relation aggregation objects, the system writes the entity aggregation objects into nodes in the graph database and the relation aggregation objects into relation edges in the graph database. Simultaneously, the system can also vectorize text fragments, entity aggregation objects, and relation aggregation objects respectively and write them into a vector index library. Figure 6 In the diagram, the solid arrow pointing from the entity aggregation table to the node region in the graph database represents the process of writing entity nodes; the solid arrow pointing from the relation aggregation table to the relation edge region in the graph database represents the process of writing relation edges; and the solid arrows pointing from the text fragment table, entity aggregation table, and relation aggregation table to the vector index library respectively represent the text fragment vectorization, entity vectorization, and relation vectorization processes.

[0049] In one implementation, a graph database is used to store structured relationships between entity objects, and a vector index is used to store semantic representations of text fragments, entity objects, and relational objects. By combining the graph database and the vector index, the graph structure association capability and the vector semantic recall capability can be utilized simultaneously in subsequent query stages, thereby supporting hybrid retrieval.

[0050] In one implementation, the system can perform idempotent writes based on a unique business key. For example, when writing entity vectors, the entity name can be used as the idempotency criterion; when writing relation vectors, the relationship start identifier, relationship end identifier, and relation type combination can be used as the idempotency criterion. If the same entity or relation already exists, the system reads the old value, merges fields such as description, direct source set, and complete source path set, and recalculates or updates the corresponding vector. Through these idempotency criterion settings, the idempotency of the system operation process can be guaranteed.

[0051] In one implementation, when multiple text fragments concurrently contribute to the same entity aggregation object or the same relation aggregation object, the system can read the existing direct source set and complete source path set of the aggregation object before writing it, and verify whether the old values ​​in the current database are still consistent with those read during writing. If other processing tasks have updated the source field of the aggregation object before this write, the updated source field can be read again during this write, and deduplication aggregation can be re-executed before writing. Through the above optimistic verification and retry method, the risk of the direct source set or complete source path set being overwritten, lost, or repeatedly written during concurrent writes can be reduced.

[0052] Figure 3 The diagram illustrates the graph retrieval and hybrid search process. After a user initiates a search or question-and-answer request, the system first performs semantic analysis on the query request to extract high-level and low-level keywords. High-level keywords are used to represent thematic, conceptual, or relational expressions in the question, while low-level keywords are used to represent object-related, entity-related, or local semantic expressions in the question.

[0053] In one implementation, low-level keywords enter the entity retrieval branch. The system can perform vectorization processing on each low-level keyword and retrieve candidate entities from the entity vector index. After recalling candidate entities, the system can further query the connectivity of these entities in the graph database and use the connectivity as part of the ranking signal. A higher connectivity indicates that the entity is associated with more objects in the current knowledge graph, and therefore can be used as a reference for evaluating the importance of candidate entities.

[0054] In one implementation, high-level keywords enter the relation retrieval branch. The system can recall candidate relations in the relation vector index and rank them based on relation weight, semantic similarity, or other ranking factors. Subsequently, the system can expand neighbor nodes and associated edges in the graph database based on candidate entities and filter more important relations according to relation weight and node structural strength.

[0055] After obtaining candidate entities, candidate relations, and expanded neighbor objects, the system further retrieves the original text fragments corresponding to these objects, thus forming a source text set. The system can merge the results from the entity recall branch, relation recall branch, graph expansion branch, and source text retrieval branch, and perform deduplication and reordering on the merged results to generate entity results, relation results, and source results. The source results can be used as part of the final output or as evidence for subsequent answer generation or answer display.

[0056] In one implementation, when sorting the source text set, the ranking result can be determined by combining the graph structure importance, relation strength, and textual evidence contribution of the candidate objects. For example, in the entity recall stage, the connectivity of candidate entities in the currently visible graph can be used as the entity ranking factor; in the relation recall stage, the relation weight and the structural importance of the entities at both ends of the relation can be used as the relation ranking factor; in the source text candidate stage, the number of times the same text fragment is simultaneously associated with candidate entities, candidate relations, or entity pairs can be counted. The higher the number of associations, the more candidate objects the text fragment can simultaneously support, thus increasing its ranking priority. Furthermore, before finally outputting the source results, a re-ranking can be performed based on the relevance score between the query request and the candidate text fragments, and text fragments with a relevance score reaching a preset threshold can be output as source results.

[0057] Figure 4 A multi-data-plane permission filtering process is illustrated. Unlike filtering that is only performed uniformly at the final output stage, the filtering in this application can be performed across multiple intermediate processing stages. For example, source filtering can be performed at least two of the following stages: entity recall stage, relation recall stage, graph expansion stage, neighbor expansion stage, and source text lookup stage.

[0058] In one implementation, filtering in the entity recall stage can be based on the direct source set and the complete source path set of the entity aggregation object; filtering in the relationship recall stage can be based on the direct source set and the complete source path set of the relationship aggregation object; filtering in the graph expansion stage and the neighbor expansion stage can be based on the source information corresponding to the expanded object; and filtering in the source text lookup stage can be based on the source hierarchy identifier of the original text fragment. Through these filtering methods, the system can preemptively eliminate objects that do not meet the source visibility condition before candidate objects enter the next processing stage, reducing cross-source expansion or unauthorized propagation.

[0059] In one implementation, when only a portion of the sources of an aggregated entity or relationship satisfy the visibility conditions corresponding to the current query, the system does not directly output the complete aggregated object. Instead, it recalculates the object's visibility attributes based on the currently visible contribution details. Recalculated attributes include at least one of the following: object description, object keywords, object weight, direct source set, and complete source path set. If conflicts arise between filtering results from different stages, the system can adopt a conservative decision-making principle: if any dimension is determined to be invisible, the corresponding entity, relationship, path, neighbor, or reference will not be included in the final result set.

[0060] Figure 5This illustrates the process of deleting, retaining, and rebuilding by source. When the system receives an update request or deletion request for a target source, it first determines the set of sources to be processed corresponding to the request, then reads the set of sources associated with the target object, and determines whether to delete, retain, or rebuild the target object based on the relationship between the two. The target object can be a target entity aggregation object or a target relationship aggregation object.

[0061] In one implementation, the set of sources to be processed can be generated at the granularity of update or deletion requests. When the request is a text-level deletion request, the set of sources to be processed can consist of the source paths associated with the corresponding text identifiers; when the request is a source group-level or dataset-level deletion request, the system can first query all text identifiers under the corresponding source group or dataset from the text fragment table, and then form the set of sources to be processed.

[0062] For any target object, the system can read its direct source set and use it as the object's source set. If the object's source set consists only of sources from the source set to be processed, it means the target object is formed solely by contributions from the source set to be processed. The system deletes the target object and its corresponding graph data and vector index data. If the object's source set includes sources from the source set to be processed and other sources, it means the target object is formed by contributions from multiple sources. The system retains the target object and removes the source information corresponding to the source set to be processed. If the object's source set and the source set to be processed have no intersection, it means the target object is unaffected by this processing, and the system keeps the target object unchanged.

[0063] In one implementation, for shared objects that need to be retained but whose sources need to be partially removed, the system can read the detailed objects corresponding to the remaining sources from the entity detail table and the relationship detail table, and redetermine at least one attribute of the target object from the description, keywords, weight, direct source set, and complete source path set based on these detailed objects. For example, for a relationship aggregation object, the weight contribution information of multiple relationship detail objects corresponding to the remaining sources can be re-accumulated to obtain a new relationship weight; the keyword contribution information corresponding to the remaining sources can be deduplicated and concatenated to obtain new relationship keywords; and the description contribution information corresponding to the remaining sources can be recombine to obtain a new relationship description.

[0064] In one implementation, when the target object is an entity aggregation object, the system can reconstruct the entity description, direct source set, and complete source path set based on the entity detail objects corresponding to the remaining sources; when the target object is a relation aggregation object, the system can reconstruct the relation description, relation keywords, relation weights, direct source set, and complete source path set based on the relation detail objects corresponding to the remaining sources. Through the above processing, even if only some sources are stripped from an aggregation object, the system can regenerate the aggregation object attributes based on the detailed contributions of the remaining sources, thereby reducing the risk of object state distortion and source tracing distortion.

[0065] Figure 6 In the diagram, dashed arrows pointing from the log table to the graph database and vector index database respectively indicate dual-write status records and compensation / recovery relationships. Unlike solid arrows representing the generation, aggregation, writing, and vectorization of main business data, dashed arrows indicate auxiliary control flows, state maintenance flows, or descriptive relationships. For example, Figure 6 The dashed arrows pointing from the processing log table to the graph database and vector index represent dual-write status records, failure retries, and compensation recovery. In one implementation, the processing log table can also record information such as object type, object identifier, most recent exception reason, creation time, and update time. The object type distinguishes whether the currently processed object is a text fragment, entity object, or relational object; the object identifier records the unique identifier corresponding to the currently processed object; and the most recent exception reason records exception information such as graph database write failure, vector index write failure, network connection timeout, or service unavailability. The graph write status, vector write status, and overall status can be set to pending, graph write successful, vector write successful, partially successful, processing successful, retrying, compensation in progress, or processing failed, depending on the processing procedure. Therefore, the recovery task can locate objects in the partially successful, retrying, compensation in progress, or processing failed states based on the processing log table and perform targeted graph database rewriting, vector index rewriting, source field recalculation, or invalid object cleanup.

[0066] In one implementation, the processing log table can record operation identifiers, graph write status, vector write status, overall status, and the number of retries. The graph database and vector index database can maintain eventual consistency without relying on distributed transactions, instead using the processing log table and compensation mechanisms. If, during an update process, a graph write succeeds but a vector index write fails, or vice versa, the system can perform retries or compensation recovery based on the status information in the processing log table.

[0067] In one implementation, the system can generate the same business operation identifier for both graph database write operations and vector index database write operations, and write this business operation identifier, along with the object primary key, source field, and processing status, into the processing log table. When an update failure is detected on one side, the affected object can be relocated based on the business unique key, and the deletion, source stripping, or reconstruction operations can be repeated. Since deletion, source stripping, and reconstruction can all be performed based on source set relationships, repeated execution typically does not produce additional side effects on objects that have already been processed.

[0068] The following section uses a medical knowledge base scenario to illustrate the actual data flow process of this application.

[0069] In this embodiment, the system receives a data text belonging to the medical knowledge base, with its dataset identifier being ds_med_001, source group identifier being col_cardio_001, and text identifier being txt_heart_001. The original text content of this data text could be: "A cardiologist at a medical institution uses advanced cardiac CT technology to diagnose a patient with heart disease, develops a personalized drug treatment plan for the patient, and recommends regular follow-up. The patient has a history of hypertension, and there are also cases of coronary heart disease in their family." The system first performs fragmentation processing on the data text and writes the resulting text fragments into a text fragment table, along with the corresponding dataset identifier, source group identifier, and text identifier. Subsequently, the system calls a large language model to perform multi-round entity relation generation on the text fragments, obtaining entities such as "cardiologist," "medical institution," "cardiac CT technology," "patient," "heart disease," "drug treatment plan," "regular follow-up," "hypertension," and "coronary heart disease," as well as relations such as "cardiologist → cardiac CT technology," "cardiac CT technology → heart disease," "cardiologist → drug treatment plan," "cardiologist → regular follow-up," "patient → hypertension," "patient → coronary heart disease," and "coronary heart disease → family history."

[0070] For entities generated from the data text, the system can write the current text identifier into the direct source set and construct a complete source path based on the dataset identifier, source group identifier, and text identifier. For example, the entity description of the entity "patient" can be "the patient has a history of hypertension and there are also cases of coronary heart disease in the family," its direct source set can include txt_heart_001, and its complete source path set can include ds_med_001-col_cardio_001-txt_heart_001. The entity description of the entity "cardiac CT technology" can be "advanced cardiac CT technology for the diagnosis of heart disease," its direct source set can include txt_heart_001, and its complete source path set can include ds_med_001-col_cardio_001-txt_heart_001.

[0071] For relations generated from this data text, the system can write the relation start identifier, relation end identifier, relation description, keywords, weight, direct source set, and complete source path set into the relation details object. For example, the relation description for "patient → hypertension" can be "the patient has a history of hypertension," keywords can include "history" and "hypertension," the weight can be 1.0, the direct source set can include txt_heart_001, and the complete source path set can include ds_med_001-col_cardio_001-txt_heart_001. The relation description for "cardiac CT technology → heart disease" can be "cardiac CT technology is used for heart disease diagnosis," keywords can include "diagnosis" and "heart disease," the weight can be 1.0, the direct source set can include txt_heart_001, and the complete source path set can include ds_med_001-col_cardio_001-txt_heart_001.

[0072] For the entities and relationships extracted from this data text, the system can construct the following source fields: source_id = txt_heart_001, all_come_ids = ds_med_001-col_cardio_001-txt_heart_001.

[0073] If the same entity or relation is repeatedly hit in multiple rounds of extraction, the system will not create duplicate nodes or relation edges. Instead, it will perform deduplication and aggregation according to the entity merging rules and relation merging rules. For example, for the relation "patient → hypertension", the system can record the relation start point identifier, relation end point identifier, text identifier, direct source set, complete source path set, relation description contribution information, and weight contribution information in the relation details table.

[0074] Suppose the system subsequently receives another data text belonging to the same medical knowledge base. This data text has the dataset identifier ds_med_001, the source group identifier col_cardio_001, and the text identifier txt_heart_002. The original text content of this data text could be: "The patient was previously diagnosed with hypertension. During a recent follow-up visit, it is necessary to monitor blood pressure control and, based on the chest tightness symptoms, determine whether further cardiac examination is needed." After the system performs fragmentation processing and entity relation generation on this data text, it can also obtain entities such as "patient," "hypertension," "blood pressure control," "chest tightness symptoms," and "cardiac examination," as well as relations such as "patient → hypertension" and "chest tightness symptoms → cardiac examination."

[0075] When the system processes the second piece of data text, if the relation "Patient → Hypertension" is generated again, this relation will not create new relation edges; instead, it will be merged with the existing relation. Before merging, this relation can have the following fields: source_id = txt_heart_001, all_come_ids = ds_med_001-col_cardio_001-txt_heart_001, weight = 1.0.

[0076] After the merge, the relationship can be updated as follows: source_id = txt_heart_001 <sep>txt_heart_002, all_come_ids = ds_med_001-col_cardio_001-txt_heart_001 <sep>ds_med_001-col_cardio_001-txt_heart_002, weight = 2.0.

[0077] wherein, <sep>This represents a preset separator used to distinguish multiple sources. The above update indicates that the relationship "patient → hypertension" is formed by contributions from two text sources, and the relationship weight can also be accumulated based on the weight contribution information of the two sources.

[0078] In query scenarios, if a user requests access only to the dataset ds_med_001, the system can filter based on the dataset identifier and the complete set of source paths during entity retrieval, relation retrieval, graph expansion, and source text lookup stages. Only data with source paths belonging to ds_med_001 will be allowed to participate in retrieval and result return. For example, when a user enters the query "Does the patient have cardiovascular risk?", the system can retrieve candidate entities such as "patient", "hypertension", "chest tightness symptoms", "ECG examination", and "coronary heart disease risk", as well as candidate relations such as "patient → hypertension", "patient → chest tightness symptoms", "chest tightness symptoms → ECG examination", and "ECG examination → coronary heart disease risk". The system then determines whether a candidate object belongs to the currently visible range based on the complete set of source paths corresponding to that candidate object.

[0079] If an aggregate entity or aggregation relationship is formed across multiple datasets, the system only retains the visible contributions belonging to the authorized datasets. It then recalculates the description, keywords, and weights based on the visible details before outputting the visible result. This avoids cross-dataset information leakage due to aggregate objects containing invisible sources, and also prevents the unusable information from visible sources from being lost by simply deleting the entire aggregate object.

[0080] In a deletion scenario, if the system receives a deletion request for txt_heart_001, it first determines that the source set to be processed contains txt_heart_001. For the relation "Patient → Hypertension", if its current field is: source_id = txt_heart_001 <sep>txt_heart_002, all_come_ids = ds_med_001-col_cardio_001-txt_heart_001 <sep>ds_med_001-col_cardio_001-txt_heart_002, weight = 2.0 This indicates that the relationship is still being shared by txt_heart_002, therefore the system does not delete the relationship, but instead updates it as follows: source_id = txt_heart_002, all_come_ids = ds_med_001-col_cardio_001-txt_heart_002, weight = 1.0.

[0081] Simultaneously, the system can recalculate the relationship description and keywords based on the remaining detailed records in the relationship details table. If all sources of a certain entity or relationship fall into the pending source set, the system deletes the entity or relationship and simultaneously deletes the corresponding graph nodes, graph edges, and vector index records.

[0082] If, during the deletion process, the graph database is successfully updated but the vector index database is deleted, or the vector index database is successfully deleted but the graph database is updated, the system can relocate the affected object based on the text identifier, dataset identifier, source group identifier, direct source set, and complete source path set, and perform compensation and recovery based on the status in the processing log table until the graph database, vector index database, text fragment table, and processing log reach a consistent state.

[0083] Figure 7 This is a schematic diagram of the electronic device structure provided in this application. Figure 7 As shown, the electronic device may include a processor, a memory, a network interface, and an input / output interface. This electronic device can serve as an implementation of the systems described in the above embodiments. The memory may store program instructions, and when the processor executes these instructions, it can implement the knowledge graph construction and query methods described in the foregoing embodiments, including source-level identifier binding, text fragmentation, multi-round entity relationship generation, detailed object construction, aggregate object generation, dual writing to the graph database and vector index library, multi-stage permission filtering, deletion rollback by source, and anomaly compensation and recovery. Since the electronic device embodiments correspond to the aforementioned method embodiments, they will not be described in detail again.

[0084] The above are merely preferred embodiments of this application and are not intended to limit this application. Those skilled in the art can make various modifications or substitutions to the above embodiments without departing from the technical concept of this application, and all such modifications or substitutions should fall within the protection scope of this application.< / sep> < / sep> < / sep> < / sep> < / sep>

Claims

1. A method for constructing and querying knowledge graphs, characterized in that, include: Obtain the text to be processed and associate a source hierarchy identifier with the text to be processed. The source hierarchy identifier includes at least two of the following: dataset identifier, source group identifier, and text identifier. The text to be processed is fragmented to obtain text fragments, and the text fragments are associated with and stored with the source level identifier; Entity detail objects and relationship detail objects are generated based on the text fragments. The entity detail objects include a set of direct sources, a set of complete source paths, and descriptive contribution information. The relationship detail objects include a set of direct sources, a set of complete source paths, weight contribution information, keyword contribution information, and descriptive contribution information. An entity aggregation object is generated based on multiple entity detail objects, and a relationship aggregation object is generated based on multiple relationship detail objects; The entity aggregation object and the relation aggregation object are written into the graph database, and the text fragment, the entity aggregation object and the relation aggregation object are vectorized and written into the vector index library. In response to a query request, source filtering is performed in at least two retrieval processing stages based on at least one of the source hierarchy identifier, the direct source set, and the complete source path set. In response to an update or deletion request for a target source, a set of sources to be processed is determined, and based on the relationship between the source set associated with the target object and the set of sources to be processed, the target object is deleted, retained, or reconstructed based on the detailed objects corresponding to the remaining sources. The target object is an entity aggregation object or a relationship aggregation object. If the source set associated with the target object includes both the source in the source set to be processed and other sources, then the target object is retained, the source information corresponding to the source set to be processed is stripped, and the aggregate attribute of the target object is reconstructed based on the detailed objects corresponding to the remaining sources. When the target object is an entity aggregation object, at least one of the entity description, direct source set, and complete source path set is re-determined based on the entity detail object corresponding to the remaining sources; when the target object is a relationship aggregation object, at least one of the relationship description, keywords, weight, direct source set, and complete source path set is re-determined based on the relationship detail object corresponding to the remaining sources. The graph data and vector index data corresponding to the target object are updated in a coordinated manner.

2. The knowledge graph construction and query method according to claim 1, characterized in that, The complete source path in the complete source path set is formed by combining the dataset identifier, source group identifier, and text identifier in a preset order; When the dataset identifier, the source group identifier, or the text identifier contains a preset separator, the corresponding identifier is escaped or encoded before the complete source path is formed.

3. The knowledge graph construction and query method according to claim 1, characterized in that, The text fragment is associated with and stored in relation to the source hierarchy identifier, including: The text fragments are written into a text fragment table, which at least records the fragment identifier, text identifier, dataset identifier, source group identifier, and fragment content; The records in the text fragment table are used for entity relationship generation, source text lookup, and text fragment vectorization.

4. The knowledge graph construction and query method according to claim 1, characterized in that, Generate entity detail objects and relationship detail objects based on the text fragments, including: Perform multiple rounds of entity relation generation on the text fragment; Perform structured parsing on the results of multiple rounds of entity relationship generation to obtain the entity detail object and the relationship detail object; The entity detail object also includes an entity name and a text identifier, and the relationship detail object also includes a relationship start identifier, a relationship end identifier, and a text identifier.

5. The knowledge graph construction and query method according to claim 4, characterized in that, Perform multi-round entity relation generation on the text fragment, including: Perform the first round of entity relation generation to identify entities and relations in the text fragment; Perform a second round of entity relationship generation to supplement missing entities, missing relationships, or descriptive information; Before the start of subsequent rounds in the second round, determine whether to continue generating based on the already generated entity relationship results; If the judgment result indicates that there are still entities or relationships to be generated, then the entity and relationship generation process continues; if the judgment result indicates that there are no entities or relationships to be generated, then the entity and relationship generation process ends.

6. The knowledge graph construction and query method according to claim 1, characterized in that, An entity aggregation object is generated based on multiple entity detail objects, including: Merge multiple entity detail objects according to entity name; The descriptive contribution information in multiple entity detail objects is deduplicated and concatenated, the direct source set in multiple entity detail objects is deduplicated and aggregated, and the complete source path set in multiple entity detail objects is deduplicated and aggregated to generate the entity aggregation object.

7. The knowledge graph construction and query method according to claim 1, characterized in that, Generate a relationship aggregation object based on multiple relationship detail objects, including: Merge multiple relationship detail objects according to the relationship start identifier, relationship end identifier, and relationship type; The relationship description contribution information and keyword contribution information in multiple relationship detail objects are deduplicated and concatenated. The direct source set and complete source path set in multiple relationship detail objects are deduplicated and aggregated. The weight contribution information in multiple relationship detail objects is accumulated or recalculated to generate the relationship aggregation object.

8. The knowledge graph construction and query method according to claim 1, characterized in that, The at least two retrieval processing stages include at least two of the following: entity recall stage, relation recall stage, graph expansion stage, neighbor expansion stage, and source text lookup stage; In the entity recall phase, source filtering is performed based on the direct source set and the complete source path set of the entity aggregation object. During the relationship recall phase, source filtering is performed based on the direct source set and the complete source path set of the relationship aggregation object; During the graph expansion or neighbor expansion phase, source filtering is performed based on the source information corresponding to the expanded object. During the source text lookup phase, source filtering is performed based on the source hierarchy identifier corresponding to the text fragment.

9. The knowledge graph construction and query method according to claim 8, characterized in that, When performing source filtering, if only some sources of an entity aggregation object or relationship aggregation object meet the visibility conditions, then based on the detail objects corresponding to the sources that meet the visibility conditions, at least one visibility attribute of the entity aggregation object or relationship aggregation object is re-determined. The visible attributes include at least one of the following: description, keywords, weight, direct source set, and complete source path set.

10. The knowledge graph construction and query method according to claim 1, characterized in that, Responding to a query request includes: Semantic analysis is performed on the query request to obtain high-level keywords and low-level keywords; Candidate entities are recalled in the entity vector index based on the low-level keywords, and the connectivity of the candidate entities is determined based on the graph database. Candidate relations are recalled in the relation vector index based on the high-level keywords; Graph expansion and source text lookup are performed based on the candidate entities and / or the candidate relationships; The recall results and the return query results are merged, deduplicated, and reordered to obtain entity results, relation results, and source results.

11. The knowledge graph construction and query method according to claim 1, characterized in that, Based on the relationship between the source set associated with the target object and the source set to be processed, the method further includes deleting, retaining, or reconstructing the target object based on the detailed objects corresponding to the remaining sources. If the source set associated with the target object consists only of the sources in the source set to be processed, then delete the target object and delete the graph data and vector index data corresponding to the target object; If the source set associated with the target object does not intersect with the source set to be processed, then the target object remains unchanged.

12. The knowledge graph construction and query method according to claim 1, characterized in that, The system also includes the synchronized updating of graph data and vector index data corresponding to the target object, including: Generate the same operation identifier corresponding to both the graph database write operation and the vector index database write operation; Record the graph write status, vector write status, overall status, and number of retries in the processing log table; When either the graph write state or the vector write state indicates a write failure, a retry or compensation recovery is performed based on the processing log table.

13. The knowledge graph construction and query method according to claim 12, characterized in that, The processing log table also records the object's primary key and source field; When performing the retry or compensation recovery, the affected object is relocated based on the object primary key and the source field, and the deletion, source stripping, or reconstruction operations are repeated.

14. An electronic device, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor. When the computer program is executed by the at least one processor, the electronic device performs the knowledge graph construction and query method according to any one of claims 1 to 13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the knowledge graph construction and query method according to any one of claims 1 to 13.

Images