Knowledge base self-learning evolution method and system, and storage medium

Abstract

The application discloses a knowledge base self-learning evolution method and system and a storage medium. The method comprises the following steps: receiving a to-be-processed text containing entity mentions, recalling candidate standard entities in a knowledge graph and adaptively extracting an enhanced knowledge subgraph based on the fuzziness score thereof; encoding the to-be-processed text, the enhanced knowledge subgraph and the like into prompt words and inputting the prompt words into a language model to obtain a standardized result; when the confidence of the standardized result is lower than a threshold value or no candidate standard entity is recalled, submitting a generative probe instruction containing a predefined ontology pattern to the language model to guide the language model to generate structured candidate knowledge; calculating the knowledge entropy of the candidate knowledge based on the candidate knowledge; pushing the candidate knowledge to a man-machine collaborative review queue according to the knowledge entropy; and for the candidate knowledge that is confirmed by review, performing a transaction-based atomic write operation to realize a self-learning closed loop of the knowledge base. The application constructs a reliable and automated self-learning closed loop method, and enables the knowledge base itself to dynamically evolve.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to a knowledge base self-learning evolution method and system and a storage medium. BACKGROUND

[0002] With the deepening of medical informatization, unstructured and semi-structured medical text data such as electronic medical records, medical literature, and clinical trial reports are growing explosively. Accurately and efficiently extracting and standardizing medical entities (such as diseases, drugs, surgeries, symptoms, etc.) from these massive texts is a key bottleneck to realizing downstream intelligent applications (such as clinical decision support systems CDSS, medical record quality control, and drug research and development).

[0003] Existing entity standardization technologies mainly fall into three categories: dictionary and rule-based methods: this method relies on pre-constructed synonym dictionaries and manually written rules for matching. Traditional machine learning / deep learning-based methods: this method performs named entity recognition and linking by training models (such as BiLSTM-CRF, BERT, etc.) on a large amount of annotated corpus. Preliminary fusion methods based on retrieval-augmented generation (RAG): in recent years, with the development of large language models (LLM), methods have emerged that use a knowledge graph (KG) as an external knowledge base to enhance the context of LLM for entity standardization.

[0004] The disadvantages of existing technologies include:

[0005] Dictionary and rule-based methods: the fundamental limitation is that the knowledge base is static and cannot handle new entities or alternative names outside the dictionary, resulting in high maintenance costs, outdated knowledge updates, and limited generalization capabilities.

[0006] Traditional machine learning / deep learning-based methods: heavily rely on large-scale, high-quality annotated data, and for entities that do not appear in the training set, have complex contexts, and have high ambiguity, their processing results are still not ideal. In addition, the "black box" nature of the internal decision-making process makes it difficult to explain and trust the results.

[0007] Preliminary fusion methods based on retrieval-augmented generation (RAG): the existing technology fusion method is relatively "shallow" and "one-way":

[0008] Insufficient knowledge injection: usually only simple entity name or description retrieval is performed, and the structured topological information of the knowledge graph is not effectively injected into the reasoning process of the LLM, resulting in LLM errors when facing complex ambiguities.

[0009] Knowledge base static: the entire process KG is only as a read-only database queried, the new knowledge and new relationships discovered by the LLM in the interaction cannot be fed back to the KG, resulting in the inability of the KG to grow, and the overall performance of the system has an upper limit.

[0010] Lack of a trusted closed-loop mechanism: for entities that the LLM cannot handle, the existing technology lacks an automated, trusted closed-loop feedback mechanism that discovers, reasons, verifies, and finally writes back to the knowledge base. SUMMARY

[0011] To solve the above problems, the present application provides a knowledge base self-learning evolution method and system and a storage medium, aiming to deeply integrate the advantages of KG and LLM, and build a trusted, automated self-learning closed loop, so that the knowledge base itself can dynamically evolve, thereby overcoming the limitations in the above background art.

[0012] According to a first aspect of an embodiment of the present disclosure, a knowledge base self-learning evolution method is provided, the method comprising the following steps:

[0013] Receiving a text containing entity mentions to be processed, recalling a set of candidate standard entities in a knowledge graph;

[0014] Based on the features of the candidate standard entities, the fuzziness scores of the candidate standard entities are obtained;

[0015] According to the fuzziness score, the enhanced knowledge subgraph of the candidate standard entity is adaptively extracted from the knowledge graph;

[0016] The to-be-processed text, entity mention, candidate standard entity and its enhanced knowledge subgraph are encoded into rich information prompt words using a serialization method, and the rich information prompt words are input into a large language model for processing to obtain a standardized result and its confidence;

[0017] When the confidence of the standardized result is lower than a preset first threshold, or the preliminary linking algorithm does not recall any candidate standard entity, the following steps are implemented:

[0018] Submit a generative probe instruction containing an ontology mode to the large language model, which contains the to-be-processed text, unknown entity mention, and an ontology mode describing the attributes of the unknown entity type;

[0019] The large language model performs type inference, relation extraction and attribute filling on the unknown entity mention based on the context and the ontology mode, and generates a structured candidate knowledge triple;

[0020] Based on the candidate knowledge triple, its knowledge entropy is calculated;

[0021] According to the knowledge entropy, the candidate knowledge triplets are automatically pushed to different priority human-machine collaborative review queues and manually reviewed, confirmed, modified or rejected;

[0022] For the confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed to complete the self-learning closed loop of the knowledge base.

[0023] In some embodiments, the standardized result is further standardized to obtain a final standardized result, and the confidence of the final standardized result is compared with a preset first threshold, wherein the further standardization processing includes: cross verifying the standardized result by calculating the semantic consistency weighted score of the standardized result and other identified entities in the knowledge graph in the plurality of relationship paths in the text to be processed, to generate the final standardized result and the confidence thereof.

[0024] In some embodiments, the ambiguity score is proportional to the depth and breadth of the extracted enhanced knowledge subgraph topology.

[0025] In some embodiments, the ontological mode is injected into the generative probe instruction in a structured format, for constraining the format and type of the large language model output content.

[0026] In some embodiments, based on the candidate standard entity features, the Jaro-Winkler similarity is used to calculate the ambiguity score of the candidate standard entity, and the specific expression is: , wherein N represents the number of candidate entities, represents the highest Jaro-Winkler similarity, , represents the weight of A.

[0027] In some embodiments, based on the number of candidate standard entities and the distribution of the text similarity scores of the candidate standard entities and the entity mentions, the ambiguity score is obtained by weighted calculation.

[0028] In some embodiments, the knowledge entropy of the candidate knowledge triplet is calculated based on the knowledge entropy, and the specific expression is: , wherein is the knowledge entropy, is the confidence given by the large language model, is the authority score of the information source, is the conflict or inconsistency score with the existing knowledge of the knowledge graph, is the evidence redundancy count of the knowledge, , , , is a preset weight value.

[0029] According to a second aspect of the embodiments of the present disclosure, a knowledge base self-learning evolution system is provided, and the system comprises:

[0030] An entity standardization module is configured to receive a text to be processed containing an entity mention, recall a set of candidate standard entities in a knowledge graph, obtain a fuzziness score of the candidate standard entities based on the features of the candidate standard entities, adaptively extract an enhanced knowledge subgraph of the candidate standard entities from the knowledge graph according to the fuzziness score, encode the text to be processed, the entity mention, the candidate standard entities and the enhanced knowledge subgraph of the candidate standard entities into rich information prompt words using a serialization method, input the rich information prompt words into a large language model for processing, and obtain a standardization result and a confidence thereof;

[0031] A new entity discovery module is configured to perform the following steps when the confidence of the standardization result is lower than a preset first threshold or when no candidate standard entity is recalled by a preliminary linking algorithm: submit a generative probe instruction containing an ontological schema to the large language model, the instruction containing the text to be processed, an unknown entity mention, and an ontological schema describing the properties of the unknown entity type; the large language model performs type inference, relation extraction and attribute filling on the unknown entity mention according to the context and the ontological schema, and generates a structured candidate knowledge triple;

[0032] A knowledge evolution module is configured to calculate the knowledge entropy of the candidate knowledge triple based on the candidate knowledge triple, automatically push the candidate knowledge triple to a human-machine collaborative review queue of different priorities according to the knowledge entropy, and perform manual review, confirmation, modification or rejection; for the confirmed or modified candidate knowledge, perform a transaction-based atomic write operation to complete the self-learning closed loop of the knowledge base.

[0033] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the knowledge base self-learning evolution method when executing the program.

[0034] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, and the storage medium stores computer instructions, and the instructions are executed by a processor to implement the steps of the knowledge base self-learning evolution method.

[0035] The knowledge base self-learning evolution method, system and storage medium provided by the embodiments of the present disclosure have the following beneficial effects:

[0036] 1) Adaptive subgraph construction based on ambiguity score, the invention realizes the dynamic and on-demand control of the injected LLM knowledge quantity by quantifying the F_score model of entity ambiguity. The advantage is that compared with the static search of existing RAG, the invention can provide the right context for entities with different ambiguity, avoiding the waste of calculation caused by information redundancy, and significantly improving the reasoning accuracy of LLM in complex ambiguity scenarios.

[0037] 2) Topology cross-validation and hallucination suppression, the invention creatively introduces a semantic consistency verification mechanism based on graph embedding (RotatE model) and text embedding (SBERT model). The advantage is that it establishes a "fact checking" closed loop based on objective structured knowledge for the output of LLM, and does not blindly trust the confidence of LLM itself. This provides a solid firewall for suppressing LLM hallucinations, greatly improving the credibility and robustness of the final output results.

[0038] 3) Schema-constrained generative probe, the invention converts the information extraction task into a constrained generation task by injecting JSON Schema into the prompt, and uses guided decoding technology in the preferred embodiment. The advantage is that compared with traditional OIE or unconstrained LLM extraction, the invention ensures that the output of LLM is naturally structured, conforms to the predefined ontology, and is machine-readable, fundamentally ensuring the structural legality of the output, completely eliminating the high post-processing cost, and making new knowledge "plug and play", fundamentally solving the "last mile" problem in automatic construction of knowledge graph.

[0039] 4) Knowledge Entropy (Knowledge Entropy) evaluation model, the invention proposes a multi-dimensional "Knowledge Entropy" (KES) concept to comprehensively evaluate the uncertainty and review value of new knowledge. The advantage is that compared with the single confidence threshold judgment of existing technology, the KES model provides a more comprehensive and intelligent decision basis, can realize the intelligent scheduling and dynamic priority sorting of valuable field expert resources, and focuses the attention of experts on the highest value knowledge review, which is the key to realizing large-scale and sustainable knowledge base evolution.

[0040] 5) Atomic writing based on transactions. The invention emphasizes that all write operations must be completed in an ACID-supported database transaction. The advantage is that it ensures that the knowledge base is "all or nothing" every time it is updated, fundamentally eliminating the risk of data inconsistency caused by concurrency or faults. This is an engineering cornerstone for building a long-term evolving, data quality critical enterprise-level knowledge base to ensure its long-term availability and credibility.

[0041] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application.

[0043] Figure 1 is a knowledge base self-learning evolution method flow chart in embodiments of the present application;

[0044] Figure 2 is a self-adaptive determination of enhanced knowledge subgraph process schematic diagram in embodiments of the present application;

[0045] Figure 3 is a knowledge entropy calculation schematic diagram in embodiments of the present application;

[0046] Figure 4 is a knowledge base self-learning evolution system structure schematic diagram in embodiments of the present application;

[0047] Figure 5 is a schematic diagram of an electronic device in embodiments of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the part related to the present application is shown in the drawings, but not all the structures.

[0049] Before the example embodiments are discussed in more detail, it should be mentioned that some of the example embodiments are described as processes or methods depicted as flow charts. While the steps of the processes are depicted as sequential processes, many of the steps can be performed in parallel, concurrently or at the same time. In addition, the order of the steps can be rearranged. The processes can be terminated when their operations are completed, but can also have additional steps not included in the figure. The processes can correspond to methods, functions, routines, subroutines, subprograms, etc.

[0050] Embodiments of the present application aim to overcome the defects in the background art, and provide a novel knowledge base self-learning evolution method, system and storage medium. The main purpose is:

[0051] (1) A method for deeply integrating knowledge graph structured knowledge and large language model context reasoning ability is provided to realize high-precision and interpretable entity standardization. The method quantitatively evaluates entity ambiguity and adaptively constructs a knowledge subgraph injected with LLM;

[0052] (2) Providing a cross-validation mechanism that uses the topology of the knowledge graph to perform secondary verification on the output of large language models, significantly reducing the risk of hallucinations;

[0053] (3) Provide a mechanism to guide large language models to reason and extract structured new knowledge through preset ontological schema constraints when facing unknown entities, so as to ensure the structural legitimacy of the output;

[0054] (4) Provide a human-machine collaborative closed loop that includes a quantitative evaluation model and a transactional writing mechanism to reliably, efficiently, and atomically inject confirmed new knowledge back into the knowledge graph, thereby achieving dynamic evolution of the knowledge base. Specific examples include the following:

[0055] Knowledge base self-learning evolution method, such as Figure 1 As shown, the method includes the following steps:

[0056] Step 1: Entity standardization steps for graph enhancement:

[0057] receiving text to be processed containing entity mentions;

[0058] Recall a set of candidate standard entities in the knowledge graph through a preset preliminary linking algorithm;

[0059] Obtaining a fuzziness score for a candidate standard entity based on the candidate standard entity features. Specifically, calculating a fuzziness score for the candidate standard entity. The fuzziness score is used to characterize the degree of ambiguity of the entity mention. The fuzziness score is a function value calculated based on one or more preset features of the candidate standard entity. The preset features include but are not limited to the number of candidate entities and the distribution of text similarity scores between the candidate entities and entity mentions.

[0060] Adaptively extracting an enhanced knowledge subgraph of a candidate standard entity from the knowledge graph based on the fuzziness score. Specifically, adaptively determining the topological depth and breadth of the enhanced knowledge subgraph extracted from the knowledge graph based on the fuzziness score. A higher fuzziness score indicates greater entity ambiguity, and the depth and breadth of the extracted subgraph also increase accordingly.

[0061] A serialization method is used to encode the text to be processed, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words, and the rich information prompt words are input into a large language model for processing to obtain standardized results and their confidence levels;

[0062] Specifically, in the preferred embodiment, a predefined serialization method is adopted to encode the text to be processed, entity mentions, candidate entities, and enhanced knowledge sub-graph into an informative prompt for large language models; the informative prompt is input into a pre-set large language model to obtain a preliminary standardization result, which includes selected standard entities and their confidence levels; the result is cross-validated by calculating the semantic consistency weighted score of the preliminary standardization result and multiple relationship paths of other recognized entities in the knowledge graph, to generate the final standardization result and confidence level.

[0063] Step two, open domain new entity discovery step:

[0064] When the confidence level of the standardization result is lower than the pre-set first threshold, or the preliminary linking algorithm does not recall any candidate standard entity, the following steps are implemented:

[0065] Submit a generative probe instruction containing an ontological schema to the large language model, which contains the text to be processed, unknown entity mentions, and an ontological schema describing the attributes of the unknown entity type, specifically, submit a generative probe instruction containing a predefined ontological schema (Schema) to the large language model, which contains the text to be processed, unknown entity mentions, and a predefined ontological schema (Schema) describing the attributes of the entity type;

[0066] It should be noted that the core difference between "entity mention" and "unknown entity mention" is that the former is an entity known to the knowledge base and can find candidate matches, and the processing goal is "disambiguation"; while the latter is an entity unknown to the knowledge base and cannot find any matches at all, and the processing goal is "discovery" of new knowledge. "Unknown entity mention" is a "knowledge gap" that the existing knowledge base cannot cover, and is a key signal to trigger the knowledge base to "self-learn and evolve".

[0067] The large language model performs type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the ontological schema, generating structured candidate knowledge triples, specifically, guiding the large language model to perform type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the provided Schema under the assumption of an open world, generating structured candidate knowledge triples with their evidence text in the original text.

[0068] Step three, knowledge evolution step of closed-loop feedback:

[0069] Calculating the knowledge entropy of the candidate knowledge triple based on the candidate knowledge triple, specifically, calculating the knowledge entropy based on the confidence, information source authority, evidence redundancy and consistency with existing knowledge of the candidate knowledge triple, and the knowledge entropy is used to quantify the uncertainty and value of the candidate knowledge triple;

[0070] Based on the knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities for manual review, confirmation, modification, or rejection. Specifically, based on the knowledge entropy, candidate knowledge triples are automatically pushed to human-machine collaborative review queues of different priorities, where knowledge with higher knowledge entropy is presented for review first, and confirmation, modification, or rejection instructions issued by domain experts through the review interface are received.

[0071] For confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed, completing the knowledge base's self-learning closed loop. Specifically, for confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed... This atomic write operation ensures that a set of updates either all succeed or all fail, ensuring data consistency, thus completing the knowledge base's self-learning closed loop.

[0072] In a preferred embodiment, the normalization result is further normalized to obtain a final normalization result, and the confidence of the final normalization result is compared with a preset first threshold, wherein the further normalization processing includes: calculating the semantic consistency weighted scores of the normalization result and other identified entities in the text to be processed in multiple relationship paths in the knowledge graph, cross-validating the normalization result, and generating the final normalization result and its confidence.

[0073] In a preferred embodiment, the fuzziness score is proportional to the topological depth and breadth of the extracted enhanced knowledge subgraph.

[0074] In a preferred embodiment, ontological patterns are injected into generative probe instructions in a structured format to constrain the format and type of output content of a large language model.

[0075] In a preferred embodiment, based on the features of the candidate standard entity, the fuzziness score of the candidate standard entity is calculated using the Jaro-Winkler similarity. The specific expression is: , where N represents the number of candidate entities, represents the highest Jaro-Winkler similarity, 、 Represents weight.

[0076] In a preferred embodiment, a weighted calculation is performed based on the number of candidate standard entities and the distribution of text similarity scores between the candidate standard entities and entity mentions to obtain a fuzziness score.

[0077] In a preferred embodiment, the knowledge entropy of a candidate knowledge triple is calculated based on the candidate knowledge triple, and the specific expression is:

[0078] wherein is the knowledge entropy, is the confidence given by the large language model, is the information source authority score, is the conflict or inconsistency score with the existing knowledge of the knowledge graph, is the evidence redundancy count of the knowledge, , , , is a preset weight value.

[0079] In one specific embodiment, as shown in Figure 1 , a knowledge base self-learning evolution method based on knowledge graph and large model closed-loop feedback is provided. The following will be described in detail through a specific medical text processing example.

[0080] Suppose the text to be processed is a sentence in a clinical trial abstract: "The study found that the new targeted drug Contral-inib showed significant efficacy in ALK-positive NSCLC patients who were previously resistant to crizotinib." Among them, "Contral-inib" is a new drug not included in the knowledge base.

[0081] Step 1: Entity standardization step enhanced by graph (take "crizotinib" as an example)

[0082] When the system processes the entity mention "crizotinib", this step is executed:

[0083] (11) Preliminary linking and recall: After receiving the text containing "crizotinib", through preliminary linking algorithms such as vector retrieval, a query is initiated to the knowledge graph database, and a set of candidate entities are recalled. For example, the recall list is: [{id: KG00123, name: "crizotinib", score: 0.98}, {id: KG00456, name: "cladribine", score: 0.65}]

[0084] (12) Calculate ambiguity score: Calculate the ambiguity score of the entity mention. This score is used to quantify the difficulty of disambiguation. In this embodiment, the ambiguity score F_score is calculated by the formula , where N is the number of candidate entities, is the highest Jaro-Winkler similarity. In this example, the F_score is calculated as 0.8.

[0085] (13) Adaptive subgraph range determination: The process is illustrated in Figure 2 According to the ambiguity score calculated in the previous step, the system adaptively determines the range and structure of the enhanced knowledge subgraph to be extracted from the knowledge graph database. For example, for a high ambiguity score, the system extracts a larger and deeper knowledge subgraph that contains more layers of related entities and relationships to provide rich disambiguation context. Conversely, for a low ambiguity score, a smaller subgraph is extracted. In this example, according to the (assumed higher ambiguity), the system determines to extract a subgraph with a depth of 2.

[0086] (14) Rich information prompt word serialization: The context, entity mention, candidate entity, and extracted subgraph are encoded into a structured rich information prompt word.

[0087] (15) LLM preliminary standardization: The prompt word is sent to the preset LLM through the large language model interface for reasoning. The LLM accurately determines that "crizotinib" is highly related to "ALK-positive NSCLC" in the context using the structured knowledge provided by the subgraph, and outputs the preliminary standardization result: {"best_candidate_id": "KG00123", "confidence": 0.99,...}.

[0088] (16) Cross-validation: To suppress LLM hallucinations, the preliminary result is cross-validated. It uses the topological relationship between other recognized entities (such as "ALK-positive NSCLC") and the preliminary result ("crizotinib") in the knowledge graph to calculate their semantic consistency score. After fusing this score with the LLM's confidence, the final standardization result and confidence are obtained, which are 0.985 in the preferred embodiment.

[0089] Step two: Open domain newborn entity discovery step (example with "Contral-inib")

[0090] When the system processes the entity mention "Contral-inib", there is no preliminary link result because it is a new drug. As shown in Figure 1 When the preliminary link does not recall any candidate entity (the embodiment determines "no"), or when the confidence of the standardization result is lower than the preset threshold (the embodiment determines "yes"), the system will trigger this newborn entity discovery step.

[0091] Triggering condition: In this example, since the preliminary link has no result, this step is triggered.

[0092] Schema-constrained generative probe: build a probe instruction containing the text to be processed and a predefined JSON Schema, and submit it to the LLM through the large language model interface. The Schema defines the attributes that a new drug should have. The LLM generates structured candidate knowledge triples according to the instructions and context, and attaches evidence text. For example: [{"subject": "Contral-inib", "relation": "is_a", "object": "targeted drug",...}], with LLM confidence and source document ID.

[0093] Step three: knowledge evolution step of closed-loop feedback

[0094] After the system receives the generated candidate knowledge, it starts the closed-loop feedback step of knowledge evolution:

[0095] (31) Calculate knowledge entropy: the calculation process is shown in Figure 3 . Use the knowledge entropy calculation model to comprehensively evaluate multiple dimensions of input to quantify the uncertainty and value of new knowledge. These dimensions include: confidence given by LLM, authority of information source, consistency with existing knowledge of knowledge graph, and evidence redundancy. According to the formula, the final knowledge entropy value is calculated.

[0096] The system uses the knowledge entropy model formula as follows:

[0097] ,

[0098] Where is the knowledge entropy, is the confidence given by the large language model, is the authority score of the information source, is the conflict or inconsistency score with the existing knowledge of the knowledge graph, is the evidence redundancy count of the knowledge, , , , is the preset weight.

[0099] (32) Each parameter in the knowledge entropy model formula KES is obtained from LLM output, source rating table, KG conflict detection, and evidence base. The KES calculation in this example is 0.493.

[0100] (33) Human-machine collaborative review: domain experts review all information, including candidate knowledge, evidence text, knowledge entropy value, etc. through the human-machine collaborative review interface, and then issue confirmation, modification or rejection instructions.

[0101] (34) Atomic Write: For the expert-confirmed knowledge, the knowledge evolution module initiates a database transaction to write the new entity and all its relationships into the knowledge graph database at once, and synchronously update the associated vector index. If any step in the transaction fails, it will be rolled back, thus completing the knowledge base self-learning closed loop under the premise of ensuring data consistency.

[0102] As some alternatives to the above embodiments, including:

[0103] Calculation of ambiguity score: In another embodiment, to more accurately measure the ambiguity of entities, the ambiguity score can also consider the statistical distribution characteristics of the candidate entity similarity score, for example, by calculating the variance or information entropy of the score. The more dispersed the score, the higher the information entropy, indicating greater ambiguity. In addition, the weights , may be fixed values set by domain experts in advance, or adaptive weights obtained by machine learning training on the validation set to optimize the accuracy of the score.

[0104] Implementation of Schema constraints: To enhance the reliability of Schema constraints and prevent large language models from generating outputs that do not conform to the predetermined format, in a preferred embodiment, the method of constraining the large language model output also includes: using guided decoding or grammar-based sampling techniques. This technique filters out illegal candidate tokens according to the predefined grammar (such as context-free grammar) converted from the Schema when the large language model generates each token, thus ensuring that the final output strictly conforms to the pre-set structured format during the generation phase, without the need for complex post-processing and verification.

[0105] Calculation of inconsistency score in knowledge entropy: In another embodiment, the method for calculating the conflict or inconsistency score of the existing knowledge of the knowledge graph can include: taking the candidate knowledge triple as a hypothesis to be verified, and performing a series of predefined ontology constraint checks in the knowledge graph, such as checking whether the definition domain and value domain of the new relationship are consistent with the Schema, checking whether there is a functional dependency conflict (such as a drug can only have one molecular formula), etc. The value of the inconsistency score is positively related to the number of violated constraint rules and the pre-set severity level of each rule.

[0106] Determination of weights in knowledge entropy formula: In yet another embodiment, the weights , , , It can be calibrated by domain experts based on experience, or automatically learned by training a classification or regression model (such as logistic regression) on a small dataset containing reviewed knowledge, so that the ranking results of knowledge entropy optimally match the expert's review priority.

[0107] Another embodiment is used to illustrate a knowledge base self-learning evolution system, such as Figure 4 As shown, the system includes:

[0108] The entity normalization module 410 is configured to receive a text to be processed containing entity mentions, recall a set of candidate standard entities from the knowledge graph, obtain fuzziness scores for the candidate standard entities based on their features, adaptively extract an enhanced knowledge subgraph of the candidate standard entities from the knowledge graph based on the fuzziness scores, encode the text to be processed, the entity mentions, the candidate standard entities, and their enhanced knowledge subgraphs into information-rich prompt words using a serialization method, and input the information-rich prompt words into a large language model for processing to obtain a normalized result and its confidence level.

[0109] The new entity discovery module 420 is configured to, when the confidence level of the normalization result is lower than a preset first threshold, or when the preliminary linking algorithm fails to recall any candidate standard entities, perform the following steps: submitting a generative probe instruction containing an ontology model to a large language model, the instruction containing the text to be processed, the unknown entity mention, and an ontology model describing the attributes that the unknown entity type should have; the large language model performs type inference, relationship extraction, and attribute filling on the unknown entity mention based on the context and the ontology model to generate structured candidate knowledge triples;

[0110] The knowledge evolution module 430 is used to calculate the knowledge entropy based on the candidate knowledge triples; based on the knowledge entropy, the candidate knowledge triples are automatically pushed to the human-machine collaborative review queues of different priorities for manual review, confirmation, modification or rejection; for the confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed to complete the self-learning closed loop of the knowledge base.

[0111] In addition to the above modules, the system may also include other components, such as Figure 4 As shown, its specific implementation is the same as the above-mentioned method embodiment. In addition, some components are irrelevant to the content of the embodiment of the present disclosure, so their illustration and description are omitted here.

[0112] The other specific working processes of the knowledge base self-learning and evolution system refer to the description of the above-mentioned knowledge base self-learning and evolution method embodiment, and will not be repeated here.

[0113] Another embodiment is used to illustrate that the system of the present invention can also be used with the help of Figure 5 The architecture of the computing device shown is implemented. Figure 5 The architecture of the computing device is shown in FIG.Figure 5 The illustrated architecture is exemplary only, and in implementing different devices, one or more components of the architecture can be adjusted as necessary Figure 5 The illustrated architecture is exemplary only, and in implementing different devices, one or more components of the architecture can be adjusted as necessary Figure 5 The memory 550, as a kind of computer readable storage medium, can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the knowledge base self-learning evolution method in the embodiment of the present application (for example, the entity standardization module 410, the new entity discovery module 420 and the knowledge evolution module 430 in the knowledge base self-learning evolution system). One or more CPUs 540 execute various functional applications and data processing of the system of the present application by running the software programs, instructions and modules stored in the memory 550, that is, implement the knowledge base self-learning evolution method described above, which includes the following steps:

[0114] receiving a to-be-processed text containing entity mentions, recalling a set of candidate standard entities in a knowledge graph;

[0115] obtaining a fuzziness score of the candidate standard entity based on the features of the candidate standard entity;

[0116] adaptively extracting an enhanced knowledge subgraph of the candidate standard entity from the knowledge graph according to the fuzziness score;

[0117] encoding the to-be-processed text, entity mentions, candidate standard entities and their enhanced knowledge subgraphs into rich information prompt words using a serialization method, and inputting the rich information prompt words into a large language model for processing to obtain a standardization result and its confidence;

[0118] When the confidence of the standardization result is lower than a preset first threshold, or the preliminary linking algorithm does not recall any candidate standard entity, the following steps are implemented:

[0119] submitting a generative probe instruction containing an ontological schema to the large language model, which contains the to-be-processed text, unknown entity mentions, and an ontological schema describing the attributes of the unknown entity type;

[0120] The large language model performs type inference, relation extraction and attribute filling on the unknown entity mentions according to the context and the ontological schema, and generates structured candidate knowledge triples;

[0121] calculating the knowledge entropy of the candidate knowledge triples based on the candidate knowledge triples;

[0122] According to the knowledge entropy, the candidate knowledge triples are automatically pushed to different priority human-computer collaborative review queues and manually reviewed, confirmed, modified or rejected;

[0123] For the confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed to complete the self-learning closed loop of the knowledge base.

[0124] Of course, the processor of the server provided by the embodiments of the present application is not limited to performing the method operations as described above, but can also perform related operations in the knowledge base self-learning evolution method provided by any embodiment of the present application.

[0125] The memory 550 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 550 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 550 can further include a memory remotely arranged with respect to the one or more CPUs 540, which can be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0126] The input / output 520 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device. The input / output 520 can also include a display device such as a display screen.

[0127] The embodiment of the present application further provides a non-temporary computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the knowledge base self-learning evolution method described in the above embodiment. The computer readable storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, device or component.

[0128] The computer readable signal medium can include a data signal propagated in a baseband or propagated as a carrier wave in a propagated data signal, in which the computer readable program code is embodied. Such propagated data signal can take various forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, device or component.

[0129] The program code contained in the storage medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0130] In addition, the other specific working processes of the non-temporary computer readable storage medium are described with reference to the above embodiment of the knowledge base self-learning evolution method, and will not be described again.

[0131] In this document, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the steps, methods comprising a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such steps, methods.

[0132] The above is further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed as limitation of the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, and all of them shall be deemed as falling within the protection scope of the present application.

Claims

1. A knowledge base self-learning evolution method, characterized in that, The method comprises the following steps: receiving a text containing an entity mention to be processed, recalling a set of candidate standard entities in a knowledge graph; obtaining a fuzziness score of the candidate standard entity based on the features of the candidate standard entity; adaptively extracting an enhanced knowledge subgraph of the candidate standard entity from the knowledge graph according to the fuzziness score; encoding the text to be processed, the entity mention, the candidate standard entity and its enhanced knowledge subgraph into a rich information prompt word using a serialization method, and inputting the rich information prompt word into a large language model for processing to obtain a standardized result and its confidence; when the confidence of the standardized result is lower than a preset first threshold value, or when the preliminary linking algorithm does not recall any candidate standard entity, the following steps are implemented: submitting a generative probe instruction containing an ontological mode to the large language model, which contains the text to be processed, the unknown entity mention, and an ontological mode describing the attributes of the unknown entity type; the large language model performs type inference, relation extraction and attribute filling on the unknown entity mention based on the context and the ontological mode, and generates a structured candidate knowledge triple; calculating the knowledge entropy of the candidate knowledge triple based on the candidate knowledge triple; according to the knowledge entropy, the candidate knowledge triple is automatically pushed to different priority human-computer collaborative review queues and manually reviewed, confirmed, modified or rejected; for the confirmed or modified candidate knowledge, a transaction-based atomic writing operation is performed to realize the self-learning closed loop of the knowledge base.

2. The knowledge base self-learning evolution method of claim 1, wherein, further standardizing the standardized result to obtain a final standardized result, and comparing the confidence of the final standardized result with the preset first threshold value, wherein the further standardization processing includes: cross-verifying the standardized result by calculating the semantic consistency weighted score of the standardized result and other recognized entities in the text to be processed in multiple relationship paths in the knowledge graph, generating the final standardized result and its confidence.

3. The knowledge base self-learning evolution method of claim 1, wherein, The fuzziness score is proportional to the depth and breadth of the extracted enhanced knowledge subgraph.

4. The knowledge base self-learning evolution method of claim 1, wherein, The ontological mode is injected into the generative probe instruction in a structured format to constrain the format and type of the output content of the large language model.

5. The knowledge base self-learning evolution method of claim 1, wherein, Based on the candidate standard entity features, the Jaro-Winkler similarity is used to calculate the fuzziness score of the candidate standard entity, and the specific expression is as follows: , wherein N represents the number of candidate entities, , wherein the highest Jaro-Winkler similarity is represented by , , and the weight is represented by 6. The knowledge base self-learning evolution method of claim 1, wherein, The fuzziness score is obtained by weighted calculation based on the number of candidate standard entities and the distribution of the text similarity score of the candidate standard entity and the entity mention.

7. The knowledge base self-learning evolution method of claim 1, wherein, The knowledge entropy of the candidate knowledge triple is calculated based on the candidate knowledge triple, and the specific expression is: wherein is a knowledge entropy, is a confidence level given by a large language model, is an information source authority score, is a score of conflict or inconsistency with existing knowledge of a knowledge graph, is a count of evidence redundancy of knowledge, , , , is a preset weight value.

8. A knowledge base self-learning evolution system, characterized by, The system comprises: an entity standardization module for receiving a text containing an entity mention to be processed, recalling a set of candidate standard entities in a knowledge graph, obtaining a fuzziness score of the candidate standard entity based on the features of the candidate standard entity, adaptively extracting an enhanced knowledge subgraph of the candidate standard entity from the knowledge graph according to the fuzziness score, and encoding the text to be processed, the entity mention, the candidate standard entity and its enhanced knowledge subgraph into a rich information prompt word using a serialization method, and inputting the rich information prompt word into a large language model for processing to obtain a standardized result and its confidence; The new entity discovery module is configured to implement the following steps when the confidence of the standardization result is lower than a preset first threshold value or the preliminary linking algorithm does not recall any candidate standard entity: submitting a generative probe instruction containing an ontology pattern to a large language model, the instruction containing the text to be processed, an unknown entity mention, and an ontology pattern describing the attributes of the unknown entity type; the large language model performs type inference, relation extraction and attribute filling on the unknown entity mention according to the context and the ontology pattern, and generates a structured candidate knowledge triple; The knowledge evolution module is configured to calculate the knowledge entropy of the candidate knowledge triple based on the candidate knowledge triple; according to the knowledge entropy, the candidate knowledge triple is automatically pushed to different priority human-computer collaborative review queues and subjected to manual review, confirmation, modification or rejection; for the confirmed or modified candidate knowledge, a transaction-based atomic write operation is performed to complete the self-learning closed loop of the knowledge base.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the knowledge base self-learning evolution method according to any one of claims 1 to 7 when executing the program.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The instructions implement the steps of the knowledge base self-learning evolution method according to any one of claims 1 to 7 when executed by the processor.

Images