Knowledge base self-learning evolution method and system and storage medium

Abstract

The invention discloses a knowledge base self-learning evolution method and system and a storage medium. The method comprises the steps that a to-be-processed text containing entity mention is received, candidate standard entities are recalled in a knowledge graph, and enhanced knowledge sub-graphs are adaptively extracted based on ambiguity scores of the candidate standard entities; encoding a to-be-processed text, the enhanced knowledge sub-graph and the like into cue words, and inputting the cue words into the language model to obtain a standardized result; when the confidence coefficient of the standardization result is lower than a threshold value or any candidate standard entity is not recalled, a generative probe instruction containing a predefined ontology mode is submitted to the language model, and the language model is guided to generate structured candidate knowledge; calculating knowledge entropy based on the candidate knowledge; pushing the candidate knowledge to a man-machine collaborative auditing queue according to the knowledge entropy; and executing transaction-based atomization write-in operation on the candidate knowledge which is checked and confirmed, so as to realize a self-learning closed loop of the knowledge base. According to the method, a credible and automatic self-learning closed-loop method is constructed, so that the knowledge base can be dynamically evolved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a knowledge base self-learning evolution method, system and storage medium. Background Art

[0002] With the deepening of medical informatization, the volume of unstructured and semi-structured medical text data, such as electronic medical records, medical literature, and clinical trial reports, has exploded. Accurately and efficiently extracting and standardizing medical entities (such as diseases, drugs, procedures, and symptoms) from this massive volume of text is a key bottleneck for implementing downstream intelligent applications, such as clinical decision support systems (CDSS), medical record quality control, and drug development.

[0003] Existing entity standardization technologies can be mainly divided into three categories: Dictionary- and rule-based methods: This method relies on pre-built synonym dictionaries and manually written rules for matching. Traditional machine learning / deep learning-based methods: This method performs named entity recognition and linking by training models (such as BiLSTM-CRF, BERT, etc.) on a large amount of annotated corpus. Preliminary fusion methods based on retrieval-augmented generation (RAG): In recent years, with the development of large language models (LLMs), methods have emerged that use knowledge graphs (KGs) as external knowledge bases and enhance LLM context through retrieval for entity standardization.

[0004] The shortcomings of existing technologies include: Dictionary- and rule-based methods: Their fundamental limitation is that the knowledge base is static and cannot handle new entities or entity aliases outside the dictionary, resulting in high maintenance costs, delayed knowledge updates, and limited generalization capabilities.

[0005] Traditional machine learning / deep learning-based approaches rely heavily on large-scale, high-quality annotated data and are still ineffective at processing entities that are not seen in the training set, have complex contexts, or are highly ambiguous. Furthermore, the opaque "black box" nature of their internal decision-making processes makes the results difficult to interpret and trust.

[0006] Preliminary fusion method based on retrieval-augmented generation (RAG): The fusion method of existing technologies is relatively "shallow" and "unidirectional": Insufficient knowledge injection: Usually only simple entity name or description retrieval is performed, and the structured topological information of the knowledge graph is not effectively injected into the LLM reasoning process, resulting in LLM being prone to errors when facing complex ambiguities.

[0007] The knowledge base is static: Throughout the entire process, the KG serves only as a read-only database to be queried. The new knowledge and new relationships discovered by the LLM during the interaction cannot be fed back to the KG, resulting in the KG's inability to grow and the overall performance of the system being capped.

[0008] Lack of a trusted closed-loop mechanism: For entities that LLM cannot handle, existing technologies lack an automated, trusted closed-loop feedback mechanism for discovery, reasoning, verification, and ultimately writing back to the knowledge base. Summary of the Invention

[0009] In response to the above problems, the present invention provides a knowledge base self-learning evolution method, system and storage medium, aiming to deeply integrate the advantages of KG and LLM, and build a reliable and automated self-learning closed loop, so that the knowledge base itself can evolve dynamically, thereby overcoming the limitations of the above background technology.

[0010] According to a first aspect of an embodiment of the present disclosure, a method for self-learning and evolving a knowledge base is provided, the method comprising the following steps: Receive the text to be processed containing entity mentions and recall a set of candidate standard entities in the knowledge graph; Obtaining a fuzziness score of a candidate standard entity based on the candidate standard entity features; Adaptively extracting enhanced knowledge subgraphs of candidate standard entities from the knowledge graph based on fuzziness scores; A serialization method is used to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words, and the rich information prompt words are input into a large language model for processing to obtain standardized results and their confidence levels; When the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm does not recall any candidate standard entity, the following steps are performed: Submit a generative probe instruction containing an ontology schema to a large language model. The instruction contains the text to be processed, the unknown entity mention, and an ontology schema describing the properties that the unknown entity type should have. Large language models perform type inference, relationship extraction, and attribute filling for unknown entity mentions based on context and ontological patterns, generating structured candidate knowledge triples. Calculate the knowledge entropy based on the candidate knowledge triples; Based on knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification, or rejection. For confirmed or modified candidate knowledge, transaction-based atomic write operations are performed to complete the self-learning closed loop of the knowledge base.

[0011] In some embodiments, the normalization result is further normalized to obtain a final normalization result, and the confidence of the final normalization result is compared with a preset first threshold, wherein the further normalization processing includes: calculating the semantic consistency weighted scores of the normalization result and other identified entities in the text to be processed in multiple relationship paths in the knowledge graph, cross-validating the normalization result, and generating the final normalization result and its confidence.

[0012] In some embodiments, the fuzziness score is proportional to the depth and breadth of the extracted enhanced knowledge subgraph topology.

[0013] In some embodiments, ontology patterns are injected into generative probe instructions in a structured format to constrain the format and type of output content of large language models.

[0014] In some embodiments, based on the features of the candidate standard entity, the fuzziness score of the candidate standard entity is calculated using the Jaro-Winkler similarity. The specific expression is: , where N represents the number of candidate entities, represents the highest Jaro-Winkler similarity, 、 Represents the A weight.

[0015] In some embodiments, a fuzziness score is obtained by performing a weighted calculation based on the number of candidate standard entities and the distribution of text similarity scores between the candidate standard entities and entity mentions.

[0016] In some embodiments, the knowledge entropy is calculated based on the candidate knowledge triples. The specific expression is: ,in is the knowledge entropy, The confidence given for the large language model, Score the authority of the information source, Score conflicts or inconsistencies with existing knowledge in the knowledge graph, is the evidence redundancy count for knowledge, 、 、 、 is the preset weight value.

[0017] According to a second aspect of an embodiment of the present disclosure, a knowledge base self-learning evolution system is provided, the system comprising: The entity normalization module is used to receive the text to be processed containing entity mentions and recall a set of candidate standard entities in the knowledge graph; obtain the fuzziness score of the candidate standard entity based on its features; adaptively extract the enhanced knowledge subgraph of the candidate standard entity from the knowledge graph based on the fuzziness score; use a serialization method to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraph into rich information prompt words, and input the rich information prompt words into a large language model for processing to obtain the normalization result and its confidence level; The emerging entity discovery module is configured to, when the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm fails to recall any candidate standard entities, perform the following steps: submitting a generative probe instruction containing an ontology pattern to a large language model, the instruction containing the text to be processed, the unknown entity mention, and an ontology pattern describing the attributes that the unknown entity type should have; the large language model performs type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the ontology pattern to generate structured candidate knowledge triples; The knowledge evolution module is used to calculate the knowledge entropy based on candidate knowledge triples; based on the knowledge entropy, the candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification or rejection; for confirmed or modified candidate knowledge, transaction-based atomic write operations are performed to complete the self-learning closed loop of the knowledge base.

[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned knowledge base self-learning evolution method when executing the program.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the above-mentioned knowledge base self-learning evolution method are implemented.

[0020] The embodiments of the present disclosure provide a method, system, and storage medium for self-learning and evolving a knowledge base, the beneficial effects of which include: 1) Adaptive subgraph construction based on fuzziness scoring. This invention uses the F_score model, which quantifies entity ambiguity, to dynamically and on-demand adjust the amount of knowledge injected into the LLM. Compared to the static retrieval of existing RAGs, this invention provides precisely tailored context for entities of varying ambiguity, avoiding computational waste caused by redundant information while significantly improving the inference accuracy of the LLM in complex and ambiguous scenarios by providing the most discriminative structured information.

[0021] 2) Topological cross-validation and hallucination suppression: This invention innovatively introduces a semantic consistency verification mechanism based on graph embeddings (RotatE model) and text embeddings (SBERT model). Its advantage lies in establishing a closed loop of "fact-checking" for LLM outputs based on objective structured knowledge, rather than blindly trusting the LLM's own confidence. This provides a solid firewall against LLM hallucinations and greatly improves the credibility and robustness of the final output.

[0022] 3) Schema-constrained generative probes: This invention transforms the information extraction task into a constrained generative task by injecting JSON Schema into the prompt word and, in the preferred embodiment, employing guided decoding techniques. This advantage is that compared to traditional OIE or unconstrained LLM extraction, this invention ensures that the LLM output is naturally structured, conforms to predefined ontologies, and is machine-readable. This fundamentally guarantees the structural legitimacy of the output, completely eliminates high post-processing costs, and enables "plug-and-play" access to new knowledge, fundamentally solving the "last mile" problem in the automated construction of knowledge graphs.

[0023] 4) Knowledge Entropy Assessment Model: This paper proposes a multi-dimensional "Knowledge Entropy" (KES) concept to comprehensively assess the uncertainty and review value of new knowledge. Compared to existing technologies that rely solely on confidence thresholds, the KES model provides a more comprehensive and intelligent basis for decision-making. It enables intelligent scheduling and dynamic prioritization of valuable domain expert resources, focusing expert attention on the most valuable knowledge reviews. This is key to achieving large-scale, sustainable knowledge base evolution.

[0024] 5) Transaction-based atomic writes. This method emphasizes that all write operations must be completed within ACID-compliant database transactions. Its advantage lies in ensuring that every update to the knowledge base is "all or nothing," fundamentally eliminating the risk of data inconsistency caused by concurrency or failures. This is the engineering cornerstone for ensuring the long-term availability and reliability of an enterprise-level knowledge base that evolves over time and where data quality is paramount.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0027] Figure 1This is a flow chart of the knowledge base self-learning evolution method in an embodiment of the present invention; Figure 2 1 is a schematic diagram of a process for adaptively determining an enhanced knowledge subgraph in an embodiment of the present invention; Figure 3 Schematic diagram of knowledge entropy calculation in an embodiment of the present invention; Figure 4 Schematic diagram of the structure of the knowledge base self-learning evolution system in an embodiment of the present invention; Figure 5 It is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0029] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0030] The embodiments of the present invention aim to overcome the shortcomings of the background technology and provide a novel knowledge base self-learning evolution method, system and storage medium. The main purposes are: (1) Provide a method that deeply integrates the structured knowledge of knowledge graphs with the contextual reasoning capabilities of large language models to achieve high-precision and explainable entity normalization. This method quantitatively evaluates entity ambiguity and adaptively constructs knowledge subgraphs injected into LLMs. (2) Providing a cross-validation mechanism that uses the topology of the knowledge graph to perform secondary verification on the output of large language models, significantly reducing the risk of hallucinations; (3) Provide a mechanism to guide large language models to reason and extract structured new knowledge through preset ontological schema constraints when facing unknown entities, so as to ensure the structural legitimacy of the output; (4) Provide a human-machine collaborative closed loop that includes a quantitative evaluation model and a transactional writing mechanism to reliably, efficiently, and atomically inject confirmed new knowledge back into the knowledge graph, thereby achieving dynamic evolution of the knowledge base. Specific examples include the following: Knowledge base self-learning evolution method, such as Figure 1 As shown, the method includes the following steps: Step 1: Entity standardization steps for graph enhancement: receiving text to be processed containing entity mentions; Recall a set of candidate standard entities in the knowledge graph through a preset preliminary linking algorithm; Obtaining a fuzziness score for a candidate standard entity based on the candidate standard entity features. Specifically, calculating a fuzziness score for the candidate standard entity. The fuzziness score is used to characterize the degree of ambiguity of the entity mention. The fuzziness score is a function value calculated based on one or more preset features of the candidate standard entity. The preset features include but are not limited to the number of candidate entities and the distribution of text similarity scores between the candidate entities and entity mentions. Adaptively extracting an enhanced knowledge subgraph of a candidate standard entity from the knowledge graph based on the fuzziness score. Specifically, adaptively determining the topological depth and breadth of the enhanced knowledge subgraph extracted from the knowledge graph based on the fuzziness score. A higher fuzziness score indicates greater entity ambiguity, and the depth and breadth of the extracted subgraph also increase accordingly. A serialization method is used to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words, and the rich information prompt words are input into a large language model for processing to obtain standardized results and their confidence levels; Specifically, in a preferred embodiment, a predefined serialization method is used to encode the text to be processed, entity mentions, candidate entities and enhanced knowledge subgraphs into a rich information prompt word for processing by a large language model; the rich information prompt word is input into a preset large language model to obtain a preliminary normalization result, which includes the selected standard entities and their confidence levels; the preliminary normalization result is cross-validated by calculating the semantic consistency weighted scores of multiple relationship paths in the knowledge graph between other identified entities in the text and the preliminary normalization result to generate a final normalization result and confidence level.

[0031] Step 2: Discovery of new entities in the open domain: When the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm does not recall any candidate standard entity, the following steps are performed: Submitting a generative probe instruction containing an ontology schema to a large language model, the instruction containing a to-be-processed text, an unknown entity mention, and an ontology schema describing expected properties of the unknown entity type. Specifically, submitting a generative probe instruction containing a predefined ontology schema to the large language model, the instruction containing a to-be-processed text, an unknown entity mention, and a predefined ontology schema describing expected properties of the entity type. It's important to note that the core difference between "entity mentions" and "unknown entity mentions" is that the former are entities known to the knowledge base and have candidate matches, and the processing goal is "disambiguation." The latter are entities unknown to the knowledge base and have no matches at all, and the processing goal is "discovery." "Unknown entity mentions" are "knowledge gaps" identified by the system that are not covered by the existing knowledge base and serve as the key signal that triggers the knowledge base's "self-learning evolution."

[0032] Large-scale language models perform type inference, relationship extraction, and attribute filling for unknown entity mentions based on the context and ontological schema to generate structured candidate knowledge triples. Specifically, under the open world assumption, large-scale language models are guided to perform type inference, relationship extraction, and attribute filling for unknown entity mentions based on the context and the provided Schema to generate structured candidate knowledge triples with accompanying evidence text in the original text.

[0033] Step 3: Knowledge evolution steps of closed-loop feedback: Calculating the knowledge entropy of the candidate knowledge triple based on the candidate knowledge triple, specifically, calculating the knowledge entropy based on the confidence, information source authority, evidence redundancy and consistency with existing knowledge of the candidate knowledge triple, and the knowledge entropy is used to quantify the uncertainty and value of the candidate knowledge triple; Based on the knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification, or rejection. Specifically, based on the knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities, where knowledge with higher knowledge entropy is presented for review first, and confirmation, modification, or rejection instructions issued by domain experts through the review interface are received. For confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed, completing the knowledge base's self-learning closed loop. Specifically, for confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed... This atomic write operation ensures that a set of updates either all succeed or all fail, ensuring data consistency, thus completing the knowledge base's self-learning closed loop.

[0034] In a preferred embodiment, the normalization result is further normalized to obtain a final normalization result, and the confidence of the final normalization result is compared with a preset first threshold, wherein the further normalization processing includes: calculating the semantic consistency weighted scores of the normalization result and other identified entities in the text to be processed in multiple relationship paths in the knowledge graph, cross-validating the normalization result, and generating the final normalization result and its confidence.

[0035] In a preferred embodiment, the fuzziness score is proportional to the topological depth and breadth of the extracted enhanced knowledge subgraph.

[0036] In a preferred embodiment, ontological patterns are injected into generative probe instructions in a structured format to constrain the format and type of output content of a large language model.

[0037] In a preferred embodiment, based on the features of the candidate standard entity, the fuzziness score of the candidate standard entity is calculated using the Jaro-Winkler similarity. The specific expression is: , where N represents the number of candidate entities, represents the highest Jaro-Winkler similarity, 、 Represents weight.

[0038] In a preferred embodiment, a weighted calculation is performed based on the number of candidate standard entities and the distribution of text similarity scores between the candidate standard entities and entity mentions to obtain a fuzziness score.

[0039] In a preferred embodiment, the knowledge entropy is calculated based on the candidate knowledge triples. The specific expression is: ,in is the knowledge entropy, The confidence given for the large language model, Score the authority of the information source, Score conflicts or inconsistencies with existing knowledge in the knowledge graph, is the evidence redundancy count for knowledge, 、 、 、 is the preset weight value.

[0040] In a specific embodiment, Figure 1 As shown in the figure, a method for self-learning and evolving a knowledge base based on closed-loop feedback between a knowledge graph and a large model is provided. The following describes each step in detail using a specific medical text processing example.

[0041] Assume that the text to be processed is a sentence in a clinical trial abstract: "The study found that the new targeted drug Contral-inib showed significant efficacy in ALK-positive NSCLC patients who had previously been resistant to crizotinib." Among them, "Contral-inib" is a new drug that is not included in the knowledge base.

[0042] Step 1: Entity standardization step for map enhancement (taking "crizotinib" as an example) When the system processes the entity mentioning "crizotinib", it executes this step: (11) Preliminary linking and recall: After receiving the text containing "crizotinib", a preliminary linking algorithm such as vector search is used to query the knowledge graph database and recall a set of candidate entities. For example, the recall list is: [{id:KG00123, name: "crizotinib", score: 0.98}, {id: KG00456, name: "cladribine", score: 0.65}] (12) Calculate the ambiguity score: Calculate the ambiguity score of the entity mention. This score is used to quantify the difficulty of disambiguation. In this embodiment, the ambiguity score F_score is calculated by the formula Calculate, where N is the number of candidate entities, The highest Jaro-Winkler similarity is calculated as F_score of 0.8 in this example.

[0043] (13) Adaptively determine the subgraph range: The process is as follows Figure 2 As shown in the figure. Based on the fuzziness score calculated in the previous step, the system adaptively determines the scope and structure of the enhanced knowledge subgraph extracted from the knowledge graph database. For example, for a high fuzziness score, the system extracts a larger and deeper knowledge subgraph that contains more layers of associated entities and relationships to provide rich disambiguation context. Conversely, for a low fuzziness score, a smaller subgraph is extracted. In this example, based on (assuming a higher fuzziness), the system determines to extract a subgraph with a depth of 2.

[0044] (14) Rich information prompt word serialization: Encode the context, entity mentions, candidate entities and extracted subgraphs into a structured rich information prompt word.

[0045] (15) LLM preliminary normalization: The prompt word is sent to the preset LLM through the large language model interface for reasoning. LLM uses the structured knowledge provided by the subgraph to accurately determine that "crizotinib" is highly correlated with "ALK-positive NSCLC" in the context, and outputs the preliminary normalization result: {"best_candidate_id": "KG00123", "confidence": 0.99, ...}.

[0046] (16) Cross-validation: To suppress LLM hallucinations, the preliminary results are cross-validated. This uses the topological relationship between other recognized entities in the text (e.g., “ALK-positive NSCLC”) and the preliminary result (“crizotinib”) in the knowledge graph to calculate their semantic consistency score. This score is combined with the confidence of the LLM to obtain the final standardized result and confidence, which is 0.985 in the preferred embodiment.

[0047] Step 2: Discovering new entities in the open domain (taking "Contral-inib" as an example) When the system processes the entity mentioning "Contral-inib", the initial linking yields no results because it is a new drug. Figure 1 As shown, when the preliminary linking does not recall any candidate entity (the embodiment determines "no"), or when the confidence of the normalized result is lower than the preset threshold (the embodiment determines "yes"), the system will trigger the new entity discovery step.

[0048] Trigger condition: In this example, this step is triggered because the initial link yields no results.

[0049] Schema-constrained generative probes: Construct a probe instruction containing the text to be processed and a predefined JSON Schema and submit it to the LLM through the Large Language Model interface. The schema defines the properties of a new drug. Based on the instruction and context, the LLM generates structured candidate knowledge triples with accompanying supporting text. For example, [{"subject": "Contral-inib", "relation": "is_a", "object": "Targeted drug", ...}], along with LLM confidence scores and source document IDs.

[0050] Step 3: Knowledge evolution steps of closed-loop feedback After receiving the generated candidate knowledge, the system starts the closed-loop feedback step of knowledge evolution: (31) Calculating knowledge entropy: The calculation process is as follows Figure 3 As shown in Figure 2, the knowledge entropy calculation model is used to comprehensively evaluate inputs from multiple dimensions to quantify the uncertainty and value of new knowledge. These dimensions include the confidence level provided by the LLM, the authority of the information source, consistency with existing knowledge in the knowledge graph, and evidence redundancy. The final knowledge entropy value is calculated using the formula.

[0051] The system uses the following knowledge entropy model formula: , in is the knowledge entropy, The confidence given for the large language model, Score the authority of the information source, Score conflicts or inconsistencies with existing knowledge in the knowledge graph, is the evidence redundancy count for this knowledge, 、 、 、 The preset weights.

[0052] (32) The parameters in the knowledge entropy model formula KES are obtained from the LLM output, the source rating table, the KG conflict detection, and the evidence library. In this example, the KES is calculated to be 0.493.

[0053] (33) Human-machine collaborative review: Domain experts view all information, including candidate knowledge, evidence text, knowledge entropy, etc., through the human-machine collaborative review interface, and then issue instructions to confirm, modify or reject.

[0054] (34) Atomic Write: For knowledge confirmed by experts, the knowledge evolution module initiates a database transaction to write the new entity and all its relationships into the knowledge graph database at once, and simultaneously updates the associated vector index. If any step in the transaction fails, the entire transaction is rolled back, thus completing the self-learning closed loop of the knowledge base while ensuring data consistency.

[0055] Some other alternatives to the above embodiment include: Calculation of ambiguity score: In another embodiment, in order to more accurately measure the ambiguity of an entity, the ambiguity score may also take into account the statistical distribution characteristics of the candidate entity similarity scores, for example, by calculating the variance or information entropy of the scores. The more dispersed the scores are, the higher the information entropy is, indicating a greater ambiguity. In addition, the weight , etc. can be fixed values ​​pre-set by domain experts, or adaptive weights obtained through machine learning training on the validation set to optimize the accuracy of the scoring.

[0056] Implementation of Schema Constraints: To enhance the reliability of schema constraints and prevent large language models from generating output that does not conform to the predetermined format, in a preferred embodiment, the method for constraining the output of the large language model also includes: employing guided decoding or grammar-based sampling techniques. This technique filters out illegal candidate tokens based on a predefined grammar (e.g., context-free grammar) converted from the schema when each token (word) is generated by the large language model. This ensures that the final output strictly conforms to the preset structured format during the generation phase, eliminating the need for complex post-processing and verification.

[0057] Calculation of inconsistency score in knowledge entropy: In another embodiment, the conflict or inconsistency score with the existing knowledge in the knowledge graph is The calculation method may include: taking the candidate knowledge triples as hypotheses to be verified, and performing a series of predefined ontological constraint checks in the knowledge graph, such as checking whether the domain and value range of the new relationship are consistent with the schema, and checking whether there are functional dependency conflicts (for example, a drug can only have one molecular formula). The value of is positively correlated with the number of constraint rules violated and the preset severity level of each rule.

[0058] Determination of weights in the knowledge entropy formula: In another embodiment, the weights in the formula 、 、 、 It can be calibrated by domain experts based on experience, or automatically learned by training a classification or regression model (such as logistic regression) on a small dataset containing reviewed knowledge, so that the ranking results of knowledge entropy optimally match the expert's review priority.

[0059] Another embodiment is used to illustrate a knowledge base self-learning evolution system, such as Figure 4 As shown, the system includes: The entity normalization module 410 is configured to receive a text to be processed containing entity mentions, recall a set of candidate standard entities from the knowledge graph, obtain fuzziness scores for the candidate standard entities based on their features, adaptively extract an enhanced knowledge subgraph of the candidate standard entities from the knowledge graph based on the fuzziness scores, encode the text to be processed, the entity mentions, the candidate standard entities, and their enhanced knowledge subgraphs into information-rich prompt words using a serialization method, and input the information-rich prompt words into a large language model for processing to obtain a normalized result and its confidence level. The new entity discovery module 420 is configured to, when the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm fails to recall any candidate standard entities, perform the following steps: submitting a generative probe instruction containing an ontology model to a large language model, the instruction containing the text to be processed, the unknown entity mention, and an ontology model describing the attributes that the unknown entity type should have; the large language model performs type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the ontology model to generate structured candidate knowledge triples; The knowledge evolution module 430 is used to calculate the knowledge entropy based on the candidate knowledge triples; based on the knowledge entropy, the candidate knowledge triples are automatically pushed to the human-machine collaborative review queues of different priorities for manual review, confirmation, modification or rejection; for the confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed to complete the self-learning closed loop of the knowledge base.

[0060] In addition to the above modules, the system may also include other components, such as Figure 4As shown, its specific implementation is the same as the above-mentioned method embodiment. In addition, some components are irrelevant to the content of the embodiment of the present disclosure, so their illustration and description are omitted here.

[0061] The other specific working processes of the knowledge base self-learning and evolution system refer to the description of the above-mentioned knowledge base self-learning and evolution method embodiment, and will not be repeated here.

[0062] Another embodiment is used to illustrate that the system of the present invention can also be used with the help of Figure 5 The architecture of the computing device shown is implemented. Figure 5 The architecture of the computing device is shown in FIG. Figure 5 As shown, a computer system 510, a system bus 530, one or more CPUs 540, an input / output 520, a memory 550, etc. The memory 550 can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU including the embodiment of the knowledge base self-learning evolution method. Figure 5 The architecture shown is only exemplary and may be adjusted based on actual needs when implementing different devices. Figure 5 One or more components in the system. The memory 550, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules, such as the program instructions / modules corresponding to the knowledge base self-learning evolution method in the embodiment of the present invention (for example, the entity standardization module 410, the new entity discovery module 420, and the knowledge evolution module 430 in the knowledge base self-learning evolution system). One or more CPUs 540 execute various functional applications and data processing of the system of the present invention by running the software programs, instructions, and modules stored in the memory 550, that is, to implement the above-mentioned knowledge base self-learning evolution method, which includes the following steps: Receive the text to be processed containing entity mentions and recall a set of candidate standard entities in the knowledge graph; Obtaining a fuzziness score of a candidate standard entity based on the candidate standard entity features; Adaptively extracting enhanced knowledge subgraphs of candidate standard entities from the knowledge graph based on fuzziness scores; A serialization method is used to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words, and the rich information prompt words are input into a large language model for processing to obtain standardized results and their confidence levels; When the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm does not recall any candidate standard entity, the following steps are performed: Submit a generative probe instruction containing an ontology schema to a large language model. The instruction contains the text to be processed, the unknown entity mention, and an ontology schema describing the properties that the unknown entity type should have. Large language models perform type inference, relationship extraction, and attribute filling for unknown entity mentions based on context and ontological patterns, generating structured candidate knowledge triples. Calculate the knowledge entropy based on the candidate knowledge triples; Based on knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification, or rejection. For confirmed or modified candidate knowledge, transaction-based atomic write operations are performed to complete the self-learning closed loop of the knowledge base.

[0063] Of course, the processor of the server provided by the embodiment of the present invention is not limited to executing the method operations described above, but can also execute relevant operations in the knowledge base self-learning evolution method provided by any embodiment of the present invention.

[0064] The memory 550 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. Furthermore, the memory 550 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 550 may further include memory remotely located relative to one or more CPUs 540, and these remote memories may be connected to the device via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0065] The input / output 520 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The input / output 520 may also include a display device such as a display screen.

[0066] Embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge base self-learning evolution method described in the above embodiments. The computer-readable storage medium of the embodiments of the present invention may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0067] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0068] The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0069] In addition, other specific working processes of a non-transitory computer-readable storage medium refer to the description of the above-mentioned knowledge base self-learning evolution method embodiment and are not repeated here.

[0070] In this document, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a series of elements includes not only those elements, but also includes other elements not expressly listed, or also includes elements inherent to such step or method.

[0071] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A knowledge base self-learning evolution method, characterized in that: The method comprises the following steps: Receive the text to be processed containing entity mentions and recall a set of candidate standard entities in the knowledge graph; Obtaining a fuzziness score of a candidate standard entity based on the candidate standard entity features; Adaptively extracting enhanced knowledge subgraphs of candidate standard entities from the knowledge graph based on fuzziness scores; A serialization method is used to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words, and the rich information prompt words are input into a large language model for processing to obtain standardized results and their confidence levels; When the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm does not recall any candidate standard entity, the following steps are performed: Submit a generative probe instruction containing an ontology schema to a large language model. The instruction contains the text to be processed, the unknown entity mention, and an ontology schema describing the properties that the unknown entity type should have. Large language models perform type inference, relationship extraction, and attribute filling for unknown entity mentions based on context and ontological patterns, generating structured candidate knowledge triples. Calculate the knowledge entropy based on the candidate knowledge triples; Based on knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification, or rejection. For confirmed or modified candidate knowledge, transaction-based atomic write operations are performed to achieve a self-learning closed loop of the knowledge base.

2. The knowledge base self-learning evolution method according to claim 1, characterized in that: The normalization result is further normalized to obtain a final normalization result, and the confidence of the final normalization result is compared with a preset first threshold, wherein the further normalization processing includes: calculating the semantic consistency weighted scores of the normalization result and other identified entities in the text to be processed in multiple relationship paths in the knowledge graph, cross-validating the normalization result, and generating the final normalization result and its confidence.

3. The knowledge base self-learning evolution method according to claim 1, characterized in that: The fuzziness score is proportional to the topological depth and breadth of the extracted enhanced knowledge subgraph.

4. The knowledge base self-learning evolution method according to claim 1, characterized in that: The ontology schema is injected into the generative probe instructions in a structured format to constrain the format and type of the output content of the large language model.

5. The knowledge base self-learning evolution method according to claim 1, characterized in that: Based on the features of the candidate standard entity, the Jaro-Winkler similarity is used to calculate the fuzziness score of the candidate standard entity. The specific expression is: , where N represents the number of candidate entities, represents the highest Jaro-Winkler similarity, 、 Represents weight.

6. The knowledge base self-learning evolution method according to claim 1, characterized in that: The fuzziness score is obtained by weighted calculation based on the number of candidate standard entities and the distribution of text similarity scores between the candidate standard entities and the entity mentions.

7. The knowledge base self-learning evolution method according to claim 1, characterized in that: The knowledge entropy is calculated based on the candidate knowledge triples. The specific expression is: ,in is the knowledge entropy, The confidence given for the large language model, Score the authority of the information source, Score conflicts or inconsistencies with existing knowledge in the knowledge graph, is the evidence redundancy count for knowledge, 、 、 、 is the preset weight value.

8. A knowledge base self-learning evolution system, characterized in that: The system comprises: The entity normalization module is used to receive the text to be processed containing entity mentions and recall a set of candidate standard entities in the knowledge graph; obtain the fuzziness score of the candidate standard entity based on its features; adaptively extract the enhanced knowledge subgraph of the candidate standard entity from the knowledge graph based on the fuzziness score; use a serialization method to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraph into rich information prompt words, and input the rich information prompt words into a large language model for processing to obtain the normalization result and its confidence level; The emerging entity discovery module is configured to, when the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm fails to recall any candidate standard entities, perform the following steps: submitting a generative probe instruction containing an ontology pattern to a large language model, the instruction containing the text to be processed, the unknown entity mention, and an ontology pattern describing the attributes that the unknown entity type should have; the large language model performs type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the ontology pattern to generate structured candidate knowledge triples; The knowledge evolution module is used to calculate the knowledge entropy based on candidate knowledge triples; based on the knowledge entropy, the candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification or rejection; for confirmed or modified candidate knowledge, transaction-based atomic write operations are performed to complete the self-learning closed loop of the knowledge base.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the knowledge base self-learning evolution method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the knowledge base self-learning evolution method according to any one of claims 1 to 7 are implemented.

Images