Knowledge graph construction method based on fine-tuning large language model

Abstract

The invention discloses a knowledge graph construction method based on a fine-tuning large language model, and the method comprises the following steps: S1), data preprocessing: S2), extracting knowledge points and relationships in a data source, and obtaining preprocessed knowledge points and relationships; s3) disambiguating knowledge points and relationships; and S4) constructing a knowledge graph, forming a dynamic extensible knowledge graph, regularly using the AI model to re-analyze the text data in the later use process, and updating the knowledge graph. According to the method, the knowledge points are automatically extracted by utilizing the knowledge point extraction fine tuning model to determine the entity and the relation edge, the knowledge points are further identified through multiple disambiguation means, and the complex relation between the knowledge points is constructed, so that the efficiency and the accuracy of data processing are effectively improved; and finally, storing the data subjected to ambiguity elimination processing into a graph database model to construct a knowledge graph, thereby realizing automatic construction and updating of the knowledge graph.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a knowledge graph construction method based on fine-tuning a large language model. Background Art

[0002] In knowledge management systems, the extraction and organization of knowledge points often rely on manual operations, which is labor-intensive, inefficient, and easily affected by subjective judgments. With the development of artificial intelligence technology, large AI models such as BERT and GPT have made significant progress in the field of natural language processing, providing the possibility for automated knowledge extraction. Although some models are trained through deep learning algorithms, they can understand and process large amounts of text data and identify entities, concepts, attributes, and the relationships between them in the text. The existing technical solutions have the following problems:

[0003] 1. Currently, most knowledge management systems use keyword matching and expert systems to extract and organize knowledge points.

[0004] 2. The keyword matching method relies on a predefined keyword list and identifies relevant knowledge points by searching for keywords in documents. However, this method often fails to understand the context and easily ignores information that does not contain keywords but is still important.

[0005] 3. Expert systems rely on the knowledge and rules of domain experts to extract and organize knowledge points through a series of customized rules. However, this method requires a lot of manual participation and is difficult to adapt to the ever-changing knowledge system.

[0006] 4. Another method is to use traditional relational databases to store knowledge points, but this method has limitations in dealing with complex relationships and dynamically updating knowledge graphs because it cannot effectively represent complex relationships and attributes between entities.

[0007] At the same time, there are still some limitations in the existing technology in data processing:

[0008] Existing technologies often require a lot of manual intervention when processing large amounts of unstructured data, which not only increases costs but also reduces efficiency.

[0009] It is difficult for existing technologies to accurately capture the complex relationships between knowledge points, especially in cross-domain and cross-document situations. The construction and maintenance of knowledge graphs often require professional knowledge and skills, which limits their application in a wider range of fields; existing technical solutions perform poorly in the dynamic update of knowledge graphs and are difficult to adapt to the rapidly changing information environment. In summary, the extraction of knowledge points in the existing technical data processing process relies heavily on manual labor, with low work efficiency and high costs; and the relationship between knowledge points is difficult to capture accurately, which increases the difficulty of knowledge point extraction and processing. Even if some existing technologies use knowledge graphs, the subsequent dynamic update and maintenance are more difficult, so the existing technologies have problems such as inaccurate knowledge point extraction, time-consuming knowledge graph construction and difficulty in maintenance. Summary of the invention

[0010] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide a knowledge graph construction method based on a fine-tuned large language model, obtain processed data from data preprocessing, and then automatically extract knowledge points to determine entities and relationship edges by utilizing the knowledge point extraction fine-tuning model, further identify and optimize the extracted knowledge points through a variety of disambiguation means, identify and construct complex relationships between knowledge points, and effectively improve the efficiency and accuracy of data processing. Finally, the disambiguated data is stored in a graph database model to construct a knowledge graph, thereby realizing the automatic construction and update of the knowledge graph.

[0011] Technical solution: In order to achieve the above purpose, the present invention provides a knowledge graph construction method based on fine-tuning a large language model, comprising the following steps:

[0012] S1): Data preprocessing, i.e. processing the data source of the knowledge points to be extracted, wherein the data source of the knowledge points to be extracted includes structured data and unstructured data, the structured data is directly input into the mapper, and the text data in the unstructured data is cleaned and sentence-separated;

[0013] S2): Extract knowledge points and relationships from the data source, first build a dedicated knowledge point extraction fine-tuning model, input the pre-processed data into the knowledge point extraction fine-tuning model, automatically identify entities in the text through the knowledge point extraction fine-tuning model, and extract the relationships between the entities to obtain pre-processed knowledge points and relationships;

[0014] S3): Knowledge point and relationship disambiguation, that is, identification and mapping of different descriptions of the same knowledge point; the specific disambiguation process is as follows:

[0015] S31): Establishing a standard knowledge point library, that is, first constructing a standard set of knowledge point entities, and using the established standard set of knowledge point entities as a mapping target set to disambiguate knowledge points extracted from the text;

[0016] S32): Similarity calculation and mapping, i.e., using text embedding technology in natural language processing to convert knowledge points and related texts into vectors; then similarity calculation is performed to determine whether the extracted different expressions point to the same knowledge point by calculating the cosine similarity or Euclidean distance between the text embedding vectors. Cosine similarity is used to measure the angle between vectors to determine the similarity between the two.

[0017] S33): Semantic disambiguation based on context: To ensure the accuracy of the disambiguation process, the sliding window technology is used to take context information into consideration, so that in the process of identifying knowledge points, not only the current sentence but also the previous and next sentences are considered to extract the knowledge point relationship from the pre-processed knowledge points;

[0018] S34): Iterative verification and feedback optimization: After disambiguation, the knowledge points and relationships extracted in the previous step will be mapped to the knowledge point entity standard set, and the two will be matched to find possible corresponding relationships. Further verification will be performed through the extracted relationships (inclusion, subordination, synonymy, etc.). If there is no match, the entity will be deleted;

[0019] S4): Knowledge graph construction: First, establish a graph database model, innovate entity nodes, and create relationships between entity nodes to form relationship edges. Then, store the disambiguated entity nodes and relationship edges into the graph database model to form a dynamic and scalable knowledge graph. In the later use process, use the AI ​​model regularly to re-analyze the text data and update the knowledge graph to ensure that it reflects the latest knowledge points and their changes in a timely manner.

[0020] In the knowledge graph construction method based on fine-tuning a large language model described in the present invention, the specific method of preprocessing text materials in unstructured data in S1) is as follows:

[0021] S11): Text cleaning: First, clean the text data in the unstructured data, use the deep learning model to classify different text areas in the text, and distinguish the irrelevant content from the main text through the context and semantic features of the text to automatically identify the main text and irrelevant content, and remove the irrelevant content through regular expressions; eliminate the noise in the text data, and delete the extra spaces, special characters, irrelevant HTML tags, and repeated text;

[0022] S12): Sentence processing: In order to provide a clear context for subsequent knowledge point extraction, the cleaned text needs to be sentence-separated;

[0023] S13): Extract repeated content, extract repeated content in the text data by detecting repeated sentences and repeated paragraphs, specifically:

[0024] Detect duplicate sentences, using text similarity detection algorithms to identify and merge similar or duplicate sentences;

[0025] Detect duplicate paragraphs: Detect longer texts through text comparison and clustering algorithms, and merge duplicate paragraphs;

[0026] S14): Context preservation: In order to preserve the original context of the text material, the following measures are taken:

[0027] Maintain paragraph structure: While dividing sentences, maintain the paragraph structure of the original text so that the complete paragraph information can be referenced when extracting knowledge points;

[0028] Use sliding windows: In order to capture a wider range of contextual information, a sliding window technique can be used when extracting knowledge points, which allows the model to consider the information of a sentence and its surrounding sentences at the same time.

[0029] The knowledge graph construction method based on fine-tuning the large language model described in the present invention, the specific process of sentence processing in S12) is as follows:

[0030] S121): Based on punctuation: using periods, question marks, exclamation marks and other punctuation marks as sentence separators;

[0031] S122): Sentence segmentation using AI models or natural language processing libraries based on natural language processing tools that can more accurately identify sentence boundaries even in the absence of obvious punctuation;

[0032] S123): Dependency-based syntactic analysis: By analyzing the dependency relationship in a sentence, the main and subordinate structures of the sentence are determined, so as to divide the sentence more accurately; its core idea is to decompose the sentence into a "dependency relationship" graph between words, so as to determine the predicate verb, subject, object and other elements in the sentence and their mutual relationships.

[0033] In the knowledge graph construction method based on fine-tuning a large language model described in the present invention, the specific analysis process of the dependency syntactic analysis in S123) is as follows:

[0034] S1231): Input preprocessing, which includes word segmentation, denoising, and normalization of the input text; the details are as follows:

[0035] Text cleaning: First, the input text data needs to be cleaned and preprocessed, specifically removing noise, symbols, irrelevant HTML tags, and normalizing complex text, such as converting it to lowercase and correcting typos;

[0036] Word segmentation: Before analyzing the syntax, use word segmentation tools to split the sentence into individual words or phrases;

[0037] S1232): Part-of-speech tagging

[0038] First, use natural language processing tools to mark the part of speech for each word. That is, use the part-of-speech tagger to tag each word or phrase after word segmentation and determine the grammatical role of each word or phrase in the sentence, such as noun, verb, or adjective. Part-of-speech tagging provides basic information for subsequent dependency analysis.

[0039] Then simplify the parts of speech: simplify the parts of speech according to the specific application scenarios of words or phrases, such as simplifying detailed verb forms into a single "verb" category;

[0040] S1233): Dependency relationship analysis, using the dependency syntax analysis model to generate a dependency relationship tree, as follows:

[0041] First, sentence component analysis is performed: dependency syntactic analysis is used to analyze the dependency relationship between words in a sentence and obtain a dependency tree. The goal of dependency syntactic analysis is to find the subordinate relationship between the "core word" and other words in a sentence.

[0042] Then determine the core components of the sentence: Analyze the predicate verb in the sentence and the related subject, object, and adverbial components. For example, `subject` depends on `verb`, and `object` also depends on `verb`. These components constitute the basic structure of the sentence.

[0043] Model selection: Select the appropriate dependency syntax model according to the actual application and build dependency relationships;

[0044] S1234): Extraction of main structure, i.e., analyzing the subject, predicate, object, attributive, adverbial and complement in the sentences in the text to identify the main components of the sentences;

[0045] S1235): Dependency tree construction, using visualization tools to display dependency relationships, so as to analyze sentence structure more intuitively, as follows:

[0046] Dependency tree structured representation: A dependency tree is generated for each sentence. The root node of the tree is the predicate verb of the sentence. Other words are attached to the trunk of the dependency tree in sequence according to the dependency relationship. The dependency tree makes it easy to see the master-slave relationship of the components in the sentence, thereby achieving grammatical analysis and sentence reorganization.

[0047] Dependency tree visualization: Use spaCy's displaCy tool or other dependency tree visualization tools to visualize the parsed dependency relationships, which helps to understand the sentence structure more intuitively.

[0048] S1236): Syntax rule verification and optimization

[0049] Dependency tree trunk and subordinate structure identification: By analyzing the dependency tree, the trunk structure of the sentence can be determined, that is, the predicate verb and its main modifying components. The subject and object directly depend on the predicate verb, while the subordinate components such as attributives and adverbials depend on the core components.

[0050] Optimization of syntactic rules: According to the needs of the field, some specific syntactic rules can be set to correct dependency relationships. For example, for complex sentence patterns such as compound sentences and parallel sentences, conjunctions can be processed through dependency syntax to clarify the position and function of conjunctions in the sentence structure. Conjunctions can be specially processed to ensure the accuracy of semantic dependency relationships.

[0051] In the knowledge graph construction method based on fine-tuning a large language model described in the present invention, a context embedding model is used in the knowledge point and relationship disambiguation in S3), and the extracted knowledge points and their contexts are converted into high-dimensional vectors, thereby achieving more accurate semantic matching. Compared with traditional word vectors, context embedding can better capture the different meanings of the same word in different contexts, thereby improving the robustness of disambiguation.

[0052] The details are as follows:

[0053] For the input text t, use the contextual embedding model BERT to convert the input text into a vector representation v;

[0054] υ t =BERT(t)

[0055] Among them, υ t is the context embedding vector of text t,

[0056] Let cosine similarity calculate the angle between two vectors, defined as:

[0057]

[0058] Among them, υ1 and υ2 are the contextual embedding vector representations of the two texts;

[0059] In the dynamic weighted cosine similarity, considering that different dimensions contribute differently to the similarity, a

[0060] Weight w i , so that the similarity calculation can more flexibly reflect the important information in the text:

[0061]

[0062] Where the dimension is d, usually d is the hidden layer size of the BERT model;

[0063] Among them, w iis the weight used for weighted cosine similarity, which is a weight dynamically calculated based on context or category, and is used to measure the contribution of different dimensions to similarity. If two knowledge points are consistent in a certain category, the weight w of the dimension related to the category can be increased. i ;

[0064] The w i Based on the following factors

[0065] Term frequency (TF): the frequency with which a word appears in a text, reflecting its importance;

[0066] Inverse Document Frequency (IDF): how rare a word is in the entire corpus;

[0067] w i =TFi·IDFi

[0068] It can also be calculated by semantic level similarity, as follows:

[0069] Assume two sentences s1 and s2, their semantic vector representations are and

[0070] SBERT uses cosine similarity to measure the similarity between two sentences s1 and s2:

[0071]

[0072] Compared with traditional sentence representation methods, SBERT can capture more fine-grained semantic information, making the similarity calculation between two sentences at the semantic level more accurate;

[0073] Multi-level similarity:

[0074] In order to match semantics more precisely, the similarity of knowledge points at multiple levels can be calculated; the details are as follows:

[0075] First, the similarity based on the word level is calculated, and then the similarity at the sentence level or paragraph level is calculated. The final similarity can be the weighted average of the similarities at each level:

[0076] total_similarity(x1, x2)

[0077] =α·word_similarity(x1,x2)+β·sentence_similarity(x1,x2)+γ·document_similarity(x1,x2)

[0078] Among them, α, β, and γ are the weights of similarity at different levels, which are adjusted according to the specific application scenario;

[0079] In the processing process, if there are multiple representations of the same entity, we fine-tune the large model to determine whether they are the same knowledge point, and then perform mapping recognition to obtain the final knowledge points and relationships;

[0080] Mapped to the knowledge point entity standard set through similarity calculation,

[0081] It also includes mapping methods, and introduces context-aware semantic matching in the matching process by capturing the context of knowledge points in the text;

[0082] Context analysis combines the contextual information of surrounding words and sentences to enhance the ability to distinguish synonyms or related knowledge points;

[0083] The knowledge graph construction method based on fine-tuning a large language model described in the present invention, the specific construction process of the knowledge point extraction fine-tuning model is as follows:

[0084] (1) Data preparation, that is, preparing enough domain-related text data, including annotations of knowledge points. For different expressions of knowledge points, it is necessary to prepare a standardized set of knowledge points to facilitate subsequent mapping and verification. The data source can be manually annotated text or use existing knowledge bases, such as Wiki or expert-annotated domain data sets;

[0085] (2) Model selection

[0086] Choose a pre-trained large language model (such as BERT, GPT, etc.) as the pre-training model. This model has been pre-trained with massive general data and has good language understanding capabilities. For the task of knowledge point extraction, you can choose a model based on the Transformer architecture. These models perform well in natural language processing tasks.

[0087] (3) Label and clean the text

[0088] Preprocess the data, including text cleaning (removing noise and irrelevant content) and sentence segmentation. Then, use annotation tools or write rules to mark entities and relationships in the text. This step can use tools such as SpaCy or NLTK for automatic sentence segmentation and part-of-speech tagging.

[0089] (4) Fine-tune the pre-trained model to obtain a fine-tuned model.

[0090] For the knowledge point extraction task, the pre-trained model is fine-tuned to obtain a fine-tuned model. The training process of the pre-trained model is completed through the following stages:

[0091] Entity recognition: Use models to automatically identify entities in text, such as concepts, facts, and principles;

[0092] Relationship extraction: extracting relationships between entities, such as inclusion, subordination, and predecessor;

[0093] Disambiguation processing: For different descriptions of the same knowledge point, vectorization technology is used to calculate similarity and map similar expressions to the same entity;

[0094] 5. Model validation and optimization

[0095] The fine-tuned model is tested on the validation set to evaluate its accuracy in knowledge point recognition and relationship extraction tasks. The similarity algorithm is used to verify the accuracy and consistency of the extracted entities and relationships. If the accuracy and consistency meet the requirements, it means that the model can meet the needs.

[0096] The knowledge graph construction method based on fine-tuning the large language model described in the present invention, the specific method of model verification and optimization is as follows:

[0097] Adaptive threshold adjustment: During the similarity calculation process, the similarity threshold is dynamically adjusted by continuously learning the accuracy of historical data;

[0098] The specific process is as follows:

[0099] Data initialization: Collect the initial historical data of similarity calculation, including the results of each similarity calculation and the correctness of actual verification;

[0100] Initial threshold setting: Set the similarity threshold based on common empirical values, such as the default threshold of cosine similarity is 0.8;

[0101] Similarity calculation: For each knowledge point to be processed, a preliminary similarity score is obtained based on cosine similarity, SBERT or other similarity algorithms;

[0102] Feedback collection: Based on the results of previous processing, the system collects feedback data from users or experts and marks whether each resolution is correct;

[0103] Adaptive learning: The system uses machine learning algorithms to dynamically adjust the similarity thresholds based on accumulated feedback data and errors in disambiguation results. The adjustment range is usually between 0.05 and 0.10. The thresholds can be adaptively changed based on different fields, text length, and semantic complexity.

[0104] Apply dynamic threshold: In a new round of similarity calculation, a dynamically adjusted threshold is used to determine whether two knowledge points are considered the same.

[0105] The knowledge graph construction method based on fine-tuning the large language model described in the present invention can be combined with symbolic reasoning in the disambiguation process. Symbolic reasoning can further correct the disambiguation results through known rules and logic. The specific process is as follows:

[0106] (1) Knowledge point extraction: Extract potential similar knowledge points from the original document data and obtain preliminary semantic similarity through a deep learning model;

[0107] (2): Semantic disambiguation: Use deep learning methods to perform preliminary similarity calculations and disambiguate some basic knowledge points;

[0108] (3): Application of symbolic reasoning rules: For knowledge points that cannot be disambiguated or have insufficient similarity, symbolic reasoning is combined with defined rule sets or logical reasoning to further correct the results based on known knowledge in the field;

[0109] (4): Reasoning optimization: The result of symbolic reasoning in the previous step is fed back to the deep learning model to further adjust the weights or thresholds of the deep learning model to reduce errors;

[0110] (5): Output the final result: Combine the final results of symbolic reasoning and deep learning disambiguation to determine whether two knowledge points are the same.

[0111] The knowledge graph construction method based on fine-tuning the large language model described in the present invention can also combine expert knowledge or rules in a specific field to perform disambiguation in addition to general semantic matching and similarity calculation in a specific field during the disambiguation process.

[0112] Implementation process:

[0113] A. Knowledge point semantic matching: Use deep learning models to perform general semantic matching on the extracted knowledge points;

[0114] B. Introduction of domain expert knowledge: Introduce expert rules, definitions, vocabularies or knowledge graphs in specific fields to further verify and correct the semantic matching results through these domain knowledge;

[0115] C. Domain rule disambiguation: more finely classify or exclude matching results based on a domain-specific rule set;

[0116] D. Output disambiguation results: Combine the domain knowledge disambiguation results with the general semantic matching results, and output the final disambiguated knowledge point matching.

[0117] The knowledge graph construction method based on fine-tuning a large language model described in the present invention uses the following multiple means to ensure the accuracy of the inserted knowledge points and relationships during the knowledge graph construction process, as follows:

[0118] (1) Multi-level verification

[0119] Rule-based verification: Different verification rules are defined for different fields to detect knowledge points and relationships. If data that does not meet the verification rules is matched, potential errors will be marked, and the knowledge points and relationships marked as not wrong will be deleted;

[0120] Context consistency verification: By analyzing the context in which the knowledge point is located, ensure that the extracted entities and relationships are consistent with the logic in the text;

[0121] Domain expert review: For knowledge in a specific field, a knowledge base annotated by experts can be introduced for comparison to ensure that the extracted knowledge points are consistent with the information in the existing knowledge base;

[0122] (2) Semantic similarity matching

[0123] Use semantic similarity calculation to verify the extracted knowledge points. For example, use language models such as BERT and SBERT to vectorize the extracted knowledge points and standard knowledge points, and then calculate the similarity to determine whether the knowledge points are accurately extracted. If the similarity is lower than the set threshold, it is marked as a potential error and deleted.

[0124] (3) Relational logic verification

[0125] Relationship type restrictions: ensure that the relationship type conforms to the logic within the domain, and perform logical verification on the relationship type through the domain rule library; for example, in the education field, the relationship between "course" and "chapter" should be "included" instead of "preceded";

[0126] Loop detection: Use graph algorithms to detect whether there are unreasonable loops in the knowledge graph to avoid unreasonable knowledge graph structure caused by incorrect relationships.

[0127] In the knowledge graph construction method based on fine-tuning a large language model described in the present invention, the means and process of verifying the accuracy of extracting knowledge points and relationships in the data source in S2) are specifically as follows:

[0128] Before storing the knowledge points and relationships in the graph database, use the following steps to verify the accuracy of the extracted knowledge points and relationships:

[0129] (1): Automated test dataset

[0130] Build a standard annotated data set and input it into the AI ​​model as a benchmark. Then compare the knowledge points extracted by the model with the standard answers. If the extracted knowledge points and relationships are consistent with those in the annotated data set, the verification is passed; otherwise, the model needs to be further debugged or the algorithm needs to be adjusted.

[0131] Accuracy: number of correctly extracted knowledge points / total number of extracted knowledge points;

[0132] Recall rate: the number of correctly extracted knowledge points / the number of knowledge points in the standard set;

[0133] (2): Manual review and expert feedback

[0134] Knowledge points in specific fields are manually reviewed by experts to ensure that the extracted content meets industry standards; for example, in the medical field, doctors can review the extracted terms and relationships;

[0135] (3): Context verification

[0136] After knowledge points are extracted, secondary verification is performed in combination with their context to ensure that the extracted knowledge points are consistent with their meanings in the context. For example, "Java" as a knowledge point of programming language may also represent other concepts in different contexts. This error can be reduced through context verification.

[0137] (4): Model adaptive optimization

[0138] By continuously collecting error cases extracted by the model, analyzing and providing feedback, adding these feedbacks to the training data, and continuously optimizing the performance of the model, this circular feedback mechanism can effectively improve the accuracy of extraction. In the process of building the knowledge graph, by inserting the extracted knowledge points and relationships into the graph database, using tools such as Neo4j and JanusGraph, a complex knowledge graph can be built. At the same time, the accuracy and logic of the inserted data are ensured through context consistency checks, rule-based verification, semantic similarity matching, and loop detection. In order to further verify the accuracy of knowledge points and relationships, a variety of methods such as automated dataset testing, expert review, and context verification can be used to ensure the high quality and high precision of the knowledge graph.

[0139] It can be seen from the above technical solution that the present invention has the following beneficial effects:

[0140] 1. The method for constructing a knowledge graph based on a fine-tuned large language model described in the present invention obtains processed data from data preprocessing, and then automatically extracts knowledge points to determine entities and relationship edges by utilizing a knowledge point extraction fine-tuning model, further identifies and optimizes the extracted knowledge points through a variety of disambiguation methods, identifies and constructs complex relationships between knowledge points, and effectively improves the efficiency and accuracy of data processing. Finally, the data after disambiguation processing is stored in a graph database model to construct a knowledge graph, thereby realizing automatic construction and updating of the knowledge graph, and effectively solving the problems of large workload and time-consuming data processing.

[0141] 2. The data preprocessing process described in the present invention effectively improves the accuracy of data processing, effectively removes invalid data, and improves the efficiency of data processing through text cleaning, sentence processing, repeated content extraction, and context retention.

[0142] 3. Extract knowledge points and relationships in the data source by building a special extraction fine-tuning model to accurately identify the relationship between entities, which effectively solves the problem of difficulty in identifying the relationship between entities in current data processing.

[0143] 4. The present invention uses a large AI model combined with vector embedding, semantic similarity calculation, and symbolic reasoning to efficiently identify different representations of the same knowledge point and disambiguate. The accuracy and flexibility of knowledge point disambiguation are further improved by dynamically updating the standard set and combining domain knowledge. This method is particularly effective in building and maintaining large-scale knowledge graphs, and can ensure the high accuracy and robustness of knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0144] Figure 1 It is a structural schematic diagram of the knowledge graph construction method based on fine-tuning the large language model according to the present invention;

[0145] Figure 2 Flow chart of knowledge point and relationship disambiguation in the present invention. DETAILED DESCRIPTION

[0146] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0147] Example

[0148] like Figure 1 A knowledge graph construction method based on fine-tuning a large language model is shown, comprising the following steps:

[0149] S1): Data preprocessing, i.e. processing the data source of the knowledge points to be extracted, wherein the data source of the knowledge points to be extracted includes structured data and unstructured data, the structured data is directly input into the mapper, and the text data in the unstructured data is cleaned and sentence-separated;

[0150] S2): Extract knowledge points and relationships from the data source, first build a dedicated knowledge point extraction fine-tuning model, input the pre-processed data into the knowledge point extraction fine-tuning model, automatically identify entities in the text through the knowledge point extraction fine-tuning model, and extract the relationships between the entities to obtain pre-processed knowledge points and relationships;

[0151] S3): Knowledge point and relationship disambiguation, that is, identification and mapping of different descriptions of the same knowledge point; the specific disambiguation process is as follows:

[0152] S31): Establishing a standard knowledge point library, that is, first constructing a standard set of knowledge point entities, and using the established standard set of knowledge point entities as a mapping target set to disambiguate knowledge points extracted from the text;

[0153] S32): Similarity calculation and mapping, i.e., using text embedding technology in natural language processing to convert knowledge points and related texts into vectors; then similarity calculation is performed to determine whether the extracted different expressions point to the same knowledge point by calculating the cosine similarity or Euclidean distance between the text embedding vectors. Cosine similarity is used to measure the angle between vectors to determine the similarity between the two.

[0154] S33): Semantic disambiguation based on context: To ensure the accuracy of the disambiguation process, the sliding window technology is used to take context information into consideration, so that in the process of identifying knowledge points, not only the current sentence but also the previous and next sentences are considered to extract the knowledge point relationship from the pre-processed knowledge points;

[0155] S34): Iterative verification and feedback optimization: After disambiguation, the knowledge points and relationships extracted in the previous step will be mapped to the knowledge point entity standard set, and the two will be matched to find possible corresponding relationships. Further verification will be performed through the extracted relationships. If there is no match, the entity will be deleted;

[0156] S4): Knowledge graph construction: First, establish a graph database model, innovate entity nodes, and create relationships between entity nodes to form relationship edges. Then, store the disambiguated entity nodes and relationship edges into the graph database model to form a dynamic and scalable knowledge graph. In the later use process, use the AI ​​model regularly to re-analyze the text data and update the knowledge graph to ensure that it reflects the latest knowledge points and their changes in a timely manner.

[0157] In the knowledge graph construction method based on fine-tuning the large language model described in this embodiment, the specific method of preprocessing the text materials in the unstructured data in S1) is as follows:

[0158] S11): Text cleaning: First, clean the text data in the unstructured data, use the deep learning model to classify different text areas in the text, and distinguish the irrelevant content from the main text through the context and semantic features of the text to automatically identify the main text and irrelevant content (headers, footers, advertisements, copyright notices), and remove irrelevant content through regular expressions; eliminate the noise in the text data, and delete extra spaces, special characters, irrelevant HTML tags (for web page data), and repeated text;

[0159] S12): Sentence processing: In order to provide a clear context for subsequent knowledge point extraction, the cleaned text needs to be sentence-separated;

[0160] S13): Extract repeated content, extract repeated content in the text data by detecting repeated sentences and repeated paragraphs, specifically:

[0161] Detect duplicate sentences, using text similarity detection algorithms to identify and merge similar or duplicate sentences;

[0162] Detect duplicate paragraphs: Detect longer texts through text comparison and clustering algorithms, and merge duplicate paragraphs;

[0163] S14): Context preservation: In order to preserve the original context of the text material, the following measures are taken:

[0164] Maintain paragraph structure: While dividing sentences, maintain the paragraph structure of the original text so that the complete paragraph information can be referenced when extracting knowledge points;

[0165] Use sliding windows: In order to capture a wider range of contextual information, a sliding window technique can be used when extracting knowledge points, which allows the model to consider the information of a sentence and its surrounding sentences at the same time.

[0166] The knowledge graph construction method based on fine-tuning the large language model described in this embodiment, the specific process of sentence processing in S12) is as follows:

[0167] S121): Based on punctuation: using periods, question marks, exclamation marks and other punctuation marks as sentence separators;

[0168] S122): Sentence segmentation based on natural language processing tools, using AI models or natural language processing libraries (such as SpaCy, NLTK, etc.), which can more accurately identify sentence boundaries even in the absence of obvious punctuation;

[0169] S123): Dependency-based syntactic analysis: By analyzing the dependency relationship in a sentence, the main and subordinate structures of the sentence are determined, so as to divide the sentence more accurately; its core idea is to decompose the sentence into a "dependency relationship" graph between words, so as to determine the predicate verb, subject, object and other elements in the sentence and their mutual relationships.

[0170] In the knowledge graph construction method based on fine-tuning the large language model described in this embodiment, the specific analysis process of the dependency syntactic analysis in S123) is as follows:

[0171] S1231): Input preprocessing, which includes word segmentation, denoising, and normalization of the input text; the details are as follows:

[0172] Text cleaning: First, the input text data needs to be cleaned and preprocessed, specifically removing noise, symbols, irrelevant HTML tags, and normalizing complex text, such as converting it to lowercase and correcting typos;

[0173] Word segmentation: Before analyzing the syntax, the sentence is divided into individual words or phrases. The accuracy of word segmentation is the basis of dependency syntax analysis. Using mature word segmentation tools such as `SpaCy`, `Stanford NLP`, and `HanLP` can greatly improve the accuracy of word segmentation;

[0174] S1232): Part-of-Speech Tagging (POS),

[0175] First, use natural language processing tools to mark the part of speech for each word, that is, use the part-of-speech tagger to tag each word or phrase after word segmentation, and determine the grammatical role of each word or phrase in the sentence, such as noun, verb, adjective; part-of-speech tagging provides basic information for subsequent dependency analysis. Commonly used tools include `spaCy`, `NLTK`, `Stanford POS Tagger`, etc.;

[0176] Then simplify the parts of speech: simplify the parts of speech according to the specific application scenarios of words or phrases, such as simplifying detailed verb forms into a single "verb" category;

[0177] S1233): Dependency Parsing, using the dependency syntactic analysis model to generate a dependency tree, so as to determine the predicate verb, subject and object elements in the sentence and their mutual relationship, as follows: First, sentence component analysis: through dependency syntactic analysis, the dependency relationship between words in the sentence is analyzed to obtain a dependency tree. The goal of dependency syntactic analysis is to find out the subordinate relationship between the "core word" (such as the predicate verb) and other words in the sentence;

[0178] Then determine the core components of the sentence: Analyze the predicate verb (core component) in the sentence and the related subject, object, and adverbial components. For example, `subject` depends on `verb`, and `object` also depends on `verb`. These components constitute the basic structure of the sentence.

[0179] Model selection: Select a suitable dependency syntax model according to the actual application and build dependency relationships; commonly used dependency syntax models include the `Transition-based` model and the `Graph-based` model. The former gradually builds dependency relationships through serialization, while the latter optimizes global dependency relationships through graph structures;

[0180] S1234): Extraction of the main structure, i.e., analyzing the subject, predicate, object, attributive, adverbial and complement in the sentences in the text to identify the main components of the sentences; S1235): Construction of dependency tree, using visualization tools to display dependency relationships, so as to analyze the sentence structure more intuitively, specifically: Dependency tree structured representation: Each sentence will generate a dependency tree, the root node of the tree is the predicate verb of the sentence, and other words are sequentially attached to the main trunk of the dependency tree according to the dependency relationship. Through the dependency tree, it is easy to see the master-slave relationship of the components in the sentence, thereby realizing grammatical analysis and sentence reorganization;

[0181] Dependency tree visualization: Use spaCy's displaCy tool or other dependency tree visualization tools to visualize the parsed dependency relationships, which helps to understand the sentence structure more intuitively.

[0182] S1236): Grammatical rule verification and optimization: Dependency tree trunk and subordinate structure identification: By analyzing the dependency tree, the trunk structure of the sentence can be determined, that is, the predicate verb and its main modifying components (such as subject, object, etc.). The subject and object directly depend on the predicate verb, while the subordinate components such as attributives and adverbials depend on the core components; Syntactic rule optimization: According to the needs of the field, some specific syntactic rules can be set to correct the dependency relationship. For example, for compound sentences and parallel sentences,

[0183] For complex sentences, conjunctions can be processed through dependency syntax to clarify the position and function of conjunctions in the sentence structure.

[0184] It should be noted that the subordinate relationship between core words and other words mainly follows the following principles:

[0185] 1. Take the predicate verb as the core: The predicate verb is the root node of the dependency tree, and other words establish dependency relationships around it.

[0186] 2. Dependency of the main components: The subject and object directly depend on the predicate verb, while subordinate components such as attributives and adverbials depend on the core components.

[0187] 3. Hierarchical structure: Through the form of a dependency tree, the master-slave relationship of each component in the sentence is clearly displayed, which facilitates grammatical analysis and sentence reorganization.

[0188] 4. Grammatical rules: Specific syntactic rules will be set according to the needs of specific fields, especially when dealing with complex sentence patterns such as compound sentences and parallel sentences, to ensure the accuracy of semantic dependencies.

[0189] The words in S1235) are linked based on grammatical relations and hierarchical structures, and the linking process follows the following principles:

[0190] -First, use the predicate verb as the root node and trunk of the dependency tree.

[0191] -The main grammatical components (such as subject and object) are directly dependent on the predicate verb, while other modifying components (such as attributives and adverbials) are dependent on the core components they modify.

[0192] -The whole forms a hierarchical structure, clearly showing the master-slave relationship of each component in the sentence.

[0193] In the specific process of linking, the main structure of the sentence is identified by analyzing the subject, predicate, object, attributive, adverbial and complement components of the sentence, following specific grammatical rules, especially when dealing with complex sentences, parallel sentences and other complex sentence patterns, to ensure the accuracy of semantic dependencies. It should be noted that the predicate verb is the core, the predicate verb is the root node of the dependency tree, and is an important part of the main structure of the entire sentence;

[0194] The core components and their subordinate relationships are as follows:

[0195] The subject and object directly depend on the predicate verb, while the attributive, adverbial, etc. are subordinate components, which depend on the core component. 3. The concept of core components:

[0196] - "Core word" refers specifically to the word that is the dominant word in a sentence, usually a predicate verb

[0197] - "Core components of a sentence" is a broader concept, including the predicate verb and the subject, object, adverbial and other sentence components directly related to it;

[0198] This hierarchical structure clearly displays the master-slave relationship of each component in the sentence in the form of a dependency tree.

[0199] Application Scenario

[0200] Machine translation: Dependency syntactic analysis can help machine translation systems better understand the structure of complex sentences and ensure the accuracy of translation.

[0201] Text Summarization: Dependency parsing can be used to generate concise text summaries by extracting the backbone structure of sentences.

[0202] Question answering system: Dependency parsing can help question answering systems understand the core components of user questions and generate more accurate answers.

[0203] As an effective method to parse the internal structure of a sentence, dependency parsing has been widely used in text analysis, information extraction and other fields.

[0204] In the process of extracting knowledge points and relationships in the data source (S2) described in this embodiment, an AI big model is used to identify entities and concepts in the text as preprocessing knowledge points. For the application of the big model, knowledge points are mainly extracted through Prompt and mounting a limited knowledge set. It should be noted that the AI ​​big model uses an existing pre-trained model, Ali qwen-max, and it can also select other models that can meet the requirements according to actual needs. Mounting a limited knowledge set is to clarify the limited knowledge point set within the vertical range. This part of the content comes from an external knowledge base, such as Wikipedia or annotations by domain experts within the company, such as a computer teacher defining 408 subject examination knowledge points and a custom rule set: according to specific needs, define a set of your own limited knowledge sets as constraints for entity and relationship extraction.

[0205] The following is a prompt text:

[0206] You are a knowledge graph expert in the field of online education, specializing in entity extraction and relationship extraction. You can extract entities from text and the relationships between entities.

[0207] The entity types are limited to the following:

[0208] Facts: Information describing observed facts, knowledge of specific things, such as capitals, dates, formulas, etc.

[0209] Composition: refers to the understanding of the components of things, such as human body structure, engine parts, computer hardware, etc.

[0210] Concept: refers to the abstract and general definition of things, such as democracy, evolution, force, etc.

[0211] Principle: refers to the understanding of how things work, such as Newton's laws of motion, circuit principles, programming languages, etc., including subcategories such as rules and formulas.

[0212] Procedure: refers to the steps and methods of how to complete a task, such as solving math problems, doing experiments, writing essays, etc.

[0213] Core courses: In an education system, basic and key courses are set up to achieve specific educational goals. Core courses usually cover a wide range of basic knowledge and skills. For example: "Advanced Mathematics: A required basic course for university science and engineering students."

[0214] Entity relationships are limited to the following:

[0215] Contains: One entity contains another entity, that is, a larger entity contains a smaller part or subset. For example: "The mathematics course contains algebra, geometry, and calculus."

[0216] Belong to: knowledge point B is part of knowledge point A. For example: "Hardware is part of the computer system."

[0217] Precedence: One entity is the previous state or stage of another entity, usually an earlier stage in a process. For example: "Algebra is a prerequisite course for learning calculus."

[0218] Synonymy: Two entities are identical or very similar in meaning and can be used interchangeably. For example: "Differential and derivative are synonymous in some contexts."

[0219] Reference: reference, related relationship, there is some kind of association or connection between two entities, such as "the instruction word length depends on the length of the opcode and the length and number of the operand addresses."

[0220] Application: apply to, knowledge point A is a kind of ability, method and approach to apply knowledge point B. For example: "Through learning and practice, theoretical knowledge can be transformed into practical skills."

[0221] Correlation: There is some association or connection between two entities, but it is not necessarily a causal relationship. For example: "There is a correlation between study time and test scores."

[0222] Follow these steps:

[0223] Step 1: High-precision entity recognition and entity extraction

[0224] - Deeply understand the entity context and accurately identify the entity attributes and meaning.

[0225] -Entity types are limited to the provided types, and new categories cannot be generated arbitrarily.

[0226] Step 2: Complex relationship parsing and extracting relationships between entities

[0227] - Ensure that the parsing logic is rigorous and complies with the construction principles and standards of the knowledge graph.

[0228] -Entity relationships are limited to the pre-provided relationship range, and new relationships cannot be generated arbitrarily.

[0229] Step 3: The final output data format is as follows:

[0230] "entities":[

[0231] {

[0232] "name":"Entity name",

[0233] "type":"Entity type"

[0234] }

[0235] ],

[0236] "relations":[

[0237] {

[0238] "head":"Relationship head entity name",

[0239] "tail": "Relationship tail entity name",

[0240] "type":"Relationship type"

[0241] } ]

[0243] }

[0244] It should be noted that the data format of json type contains array keys such as "entities".

[0245] Example:

[0246] Content: Recursion and iteration are two common methods for solving data structure problems. Recursion is breaking down a problem into smaller sub-problems, while iteration is repeatedly executing a set of instructions until a condition is met.

[0247] result:

[0248] {

[0249] "entities":[

[0250] {

[0251] "name":"Semiconductor Random Access Memory",

[0252] "type":"Concept"

[0253] },

[0254] {

[0255] "name":"Computer performance indicators",

[0256] "type":"Concept"

[0257] },

[0258] {

[0259] "name":"Basic principles of multiplication / division operations",

[0260] "type":"Principle"

[0261] },

[0262] {

[0263] "name":"Calculation method",

[0264] "type":"Principle"

[0265] },

[0266] {

[0267] "name":"Command",

[0268] "type":"Concept"

[0269] },

[0270] {

[0271] "name":"Basic Principles of Cache",

[0272] "type":"Principle"

[0273] },

[0274] {

[0275] "name":"Replacement algorithm for main memory blocks in cache",

[0276] "type":"Principle"

[0277] },

[0278] {

[0279] "name":"Instruction format",

[0280] "type":"Concept"

[0281] },

[0282] {

[0283] "name":"Data Structure",

[0284] "type":"Core Courses"

[0285] }

[0286] ],

[0287] "relations":[

[0288] {

[0289] "head":"Calculation method",

[0290] "tail":"Basic principles of multiplication / division operations",

[0291] "type":"include"

[0292] },

[0293] {

[0294] "head":"Instructions",

[0295] "tail":"Instruction format",

[0296] "type":"include"

[0297] },

[0298] {

[0299] "head":"Replacement algorithm of main memory blocks in cache",

[0300] "tail":"Basic Principles of Cache",

[0301] "type":"Related"

[0302] } ]

[0304] }

[0305] Here is an example scenario:

[0306] enter:

[0307] Advanced mathematics is a basic course for university science and engineering students, including calculus, linear algebra and probability theory. Calculus is a prelude to further study of physics, while linear algebra is closely related to engineering mathematics. Probability theory is mainly used in the field of data analysis.

[0308] Output:

[0309]

[0310]

[0311]

[0312] The knowledge point and relationship disambiguation uses cosine similarity to measure the angle between two vectors to determine whether they represent the same concept. The calculation formula of cosine similarity is:

[0313] Among them, a and b are two vectors, ||a|| and ||b|| are their moduli.

[0314] In this way, similar knowledge points can be mapped to the same entity, thereby building accurate entities and relationships in the knowledge graph. This method can improve the accuracy and robustness of the knowledge graph, allowing the knowledge graph to better reflect the knowledge and information in the text material.

[0315] The knowledge point entity standard set is the basis of knowledge disambiguation, which usually comes from authoritative public resources (such as Wikipedia, DBpedia), specialized knowledge bases annotated by domain experts, or knowledge bases within enterprises. In this process, it is crucial to ensure the accuracy and coverage of the standard set.

[0316] Authoritative public resources: such as structured data provided by WikiData, which can cover basic concepts and entities in multiple fields.

[0317] Expert-labeled specialized data sources: Combined with knowledge in specific fields, enterprises can build customized sets of standard knowledge points based on their business needs, which are particularly suitable for vertical scenario applications (such as mathematics, education, etc.).

[0318] Dynamic update mechanism: During use, the knowledge point entity standard set is regularly updated from the latest literature and databases to continuously introduce new knowledge points, thereby ensuring the real-time nature of the system's knowledge.

[0319] Standardization and naming conventions:

[0320] Name and standardize each knowledge point to ensure uniform naming. For example, a course can be unified as "Linear Algebra" instead of "Linear Algebra", "Linear Algebra Class" and other variants. At the same time, these standardized knowledge points need to contain unique identifiers (such as knowledge point IDs) for subsequent mapping and query.

[0321] In the knowledge graph construction method based on fine-tuning a large language model described in this embodiment, a context embedding model (such as BERT and RoBERTa) is used in the knowledge point and relationship disambiguation in S3) to convert the extracted knowledge points and their contexts into high-dimensional vectors, thereby achieving more accurate semantic matching. Compared with traditional word vectors (such as Word2Vec), context embedding can better capture the different meanings of the same word in different contexts and improve the robustness of disambiguation.

[0322] The details are as follows:

[0323] For the input text t, use the contextual embedding model BERT to convert the input text into a vector representation v;

[0324] υ t =BERT(t)

[0325] Among them, tis the context embedding vector of text t, with dimension d, where d is usually the hidden layer size of the BERT model (e.g. 768 dimensions);

[0326] Compared with the formula of traditional word embedding models (such as Word2Vec), the latter's word vector is expressed as:

[0327] υ w =Word2Vec(w)

[0328] Among them, w is a word, v_w is a fixed vector of the word, Word2Vec does not consider the context, and the representation of a word in different contexts is fixed.

[0329] The context embedding model maps the meaning of the same word in different scenarios to different vectors through contextual information, greatly improving the accuracy of similarity calculation.

[0330] Let cosine similarity calculate the angle between two vectors, defined as:

[0331] Among them, t is the context embedding vector of text t,

[0332] Let cosine similarity calculate the angle between two vectors, defined as:

[0333]

[0334] Among them, υ1 and υ2 are the contextual embedding vector representations of the two texts;

[0335] In the dynamic weighted cosine similarity, considering that different dimensions contribute differently to the similarity, a

[0336] Weight w i , so that the similarity calculation can more flexibly reflect the important information in the text:

[0337]

[0338] Where the dimension is d, usually d is the hidden layer size of the BERT model;

[0339] Among them, w i is the weight used for weighted cosine similarity, which is a weight dynamically calculated based on context or category, and is used to measure the contribution of different dimensions to similarity. If two knowledge points are consistent in a certain category, the weight w of the dimension related to the category can be increased. i ;

[0340] The w i Based on the following factors

[0341] Term frequency (TF): the frequency with which a word appears in a text, reflecting its importance;

[0342] Inverse Document Frequency (IDF): how rare a word is in the entire corpus;

[0343] w i =TFi·IDFi

[0344] It can also be calculated by semantic level similarity, as follows:

[0345] Assume two sentences s1 and s2, their semantic vector representations are and

[0346] SBERT uses cosine similarity to measure the similarity between two sentences s1 and s2:

[0347]

[0348] Compared with traditional sentence representation methods, SBERT can capture more fine-grained semantic information, making the similarity calculation between two sentences at the semantic level more accurate;

[0349] Multi-level similarity:

[0350] In order to match semantics more precisely, the similarity of knowledge points at multiple levels can be calculated; the details are as follows:

[0351] First, the similarity based on the word level is calculated, and then the similarity at the sentence level or paragraph level is calculated. The final similarity can be the weighted average of the similarities at each level:

[0352] total_similarity(x1, x2)

[0353] =α·word_similarity(x1,x2)+β·sentence_similarity(x1,x2)+γ·document_similarity(x1,x2)

[0354] Among them, α, β, and γ are weights of similarity at different levels, which are adjusted according to specific application scenarios; among them, α, β, and γ can be artificially set according to the degree of influence of similarity at different levels on the final goal. For example, if you pay more attention to word-level similarity, you can increase α; if the context and overall meaning are more important, you can increase β and γ.

[0355] In the processing process, if there are multiple representations of the same entity, we fine-tune the large model to determine whether they are the same knowledge point, and then perform mapping recognition to obtain the final knowledge points and relationships;

[0356] Mapped to the knowledge point entity standard set through similarity calculation,

[0357] It also includes mapping methods, and introduces context-aware semantic matching in the matching process by capturing the context of knowledge points in the text;

[0358] Contextual analysis combines the contextual information of surrounding words and sentences to enhance the ability to distinguish synonyms or related knowledge points; for example, if "quantum mechanics" is mentioned in the context of physics, it can be inferred that the statement is consistent with the relevant knowledge points in this field.

[0359] It should be noted that dynamic weighted cosine similarity is suitable for scenarios where the importance of different dimensions needs to be considered. By assigning weights w_i to different dimensions, the calculation results can more flexibly reflect important information. In particular, when two knowledge points are consistent in a certain category, the weight of the relevant dimension of the category can be increased;

[0360] Semantic Hierarchical Similarity (SBERT): Suitable for scenarios that require more fine-grained semantic analysis. Compared with traditional sentence representation methods, SBERT can capture more fine-grained semantic information, making the similarity calculation between sentences more accurate.

[0361] Multi-level similarity: Suitable for scenarios that require more comprehensive semantic matching; this method calculates similarities at multiple levels simultaneously.

[0362] The knowledge graph construction method based on fine-tuning the large language model described in this embodiment, the specific construction process of the knowledge point extraction fine-tuning model is as follows:

[0363] 1. Data preparation, that is, preparing enough domain-related text data, including annotations of knowledge points. For different expressions of knowledge points, it is necessary to prepare standardized knowledge point sets to facilitate subsequent mapping and verification. The data source can be manually annotated text or use existing knowledge bases, such as Wiki or expert-annotated domain data sets;

[0364] 2. Model selection

[0365] Choose a pre-trained large language model (such as BERT, GPT, etc.) as the pre-training model. This model has been pre-trained with massive general data and has good language understanding capabilities. For the task of knowledge point extraction, you can choose a model based on the Transformer architecture. These models perform well in natural language processing tasks.

[0366] 3. Label and clean the text

[0367] Preprocess the data, including text cleaning (removing noise and irrelevant content) and sentence segmentation. Then, use annotation tools or write rules to mark entities and relationships in the text. This step can use tools such as SpaCy or NLTK for automatic sentence segmentation and part-of-speech tagging.

[0368] 4. Fine-tune the pre-trained model to obtain a fine-tuned model.

[0369] For the knowledge point extraction task, the pre-trained model is fine-tuned to obtain a fine-tuned model. The training process of the pre-trained model is completed through the following stages:

[0370] Entity recognition: Use models to automatically identify entities in text, such as concepts, facts, and principles;

[0371] Relation extraction: The extraction model automatically identifies the relationships between entities in the text, such as containment, subordination, and predecessor;

[0372] Disambiguation processing: For different descriptions of the same knowledge point, vectorization technology is used to calculate similarity and map similar expressions to the same entity;

[0373] 5. Model validation and optimization

[0374] The fine-tuned model is tested on the validation set to evaluate its accuracy in knowledge point recognition and relationship extraction tasks. The similarity algorithm is used to verify the accuracy and consistency of the extracted entities and relationships. If the accuracy and consistency meet the requirements, it means that the model can meet the needs.

[0375] The specific process of fine-tuning the pre-trained model to obtain the fine-tuned model is as follows:

[0376] (401) First, determine the task objectives: that is, determine the task objectives of entity recognition, relationship extraction, and disambiguation processing respectively;

[0377] (402) Prepare the data set: label the entities and relations (BI0 labeling, entity pairs and relation categories);

[0378] (403) Model selection: that is, according to actual needs, select a suitable model from “BERT-type models (suitable for classification and sequence labeling), GPT-type models (suitable for generation tasks), and graph neural networks (suitable for complex relationship modeling)”;

[0379] (404) Design fine-tuning tasks: that is, setting fine-tuning tasks for the selected model, specifically, performing sequence labeling (BI0) for entity recognition and fine-tuning using a linear layer; Relationship extraction: extracting relationships from entities and contexts, extracting and outputting relationships through a classifier; for disambiguation processing, performing entity vectorization description, and training a similarity classifier or clustering model for fine-tuning;

[0380] (405): Train and optimize the fine-tuned model using the AdamW optimizer and set the learning rate to 1e -5 To 5e -5 ;

[0381] (406): Evaluate the fine-tuned training model, specifically: evaluate the entity recognition extraction through the F1 score performance indicator; evaluate the accuracy of relationship extraction by judging the text classification accuracy and F1 score performance indicator during the relationship extraction process; evaluate the effect of disambiguation processing through Top-k accuracy and similarity evaluation.

[0382] The knowledge graph construction method based on fine-tuning the large language model described in this embodiment, the specific method of model verification and optimization is as follows:

[0383] Adaptive threshold adjustment: During the similarity calculation process, the similarity threshold is dynamically adjusted by continuously learning the accuracy of historical data; the specific process is as follows:

[0384] Data initialization: Collect the initial historical data of similarity calculation, including the results of each similarity calculation and the correctness of actual verification;

[0385] Initial threshold setting: Set the similarity threshold based on common empirical values, such as the default threshold of cosine similarity is 0.8;

[0386] Similarity calculation: For each knowledge point to be processed, a preliminary similarity score is obtained based on cosine similarity, SBERT or other similarity algorithms;

[0387] Feedback collection: Based on the results of previous processing, the system collects feedback data from users or experts and marks whether each resolution is correct;

[0388] Adaptive learning: The system uses machine learning algorithms to dynamically adjust the similarity thresholds based on accumulated feedback data and errors in disambiguation results. The adjustment range is usually between 0.05 and 0.10. The thresholds can be adaptively changed based on different fields, text length, and semantic complexity.

[0389] Apply dynamic threshold: In a new round of similarity calculation, a dynamically adjusted threshold is used to determine whether two knowledge points are considered the same.

[0390] For example, when the system disambiguates “quantum physics” and “quantum mechanics,” based on similar data previously processed, the system can adaptively adjust the threshold to ensure that these statements are considered the same knowledge point.

[0391] In this embodiment, the extracted knowledge points and relationships are stored in the graph database. The specific process of forming the knowledge graph is as follows: 1. Insert the knowledge points and relationships into the graph database

[0392] Graph databases (such as Neo4j, JanusGraph, ArangoDB, etc.) store data in the form of nodes and edges. Nodes represent entities (knowledge points) and edges represent the relationship between entities. The process of building a knowledge graph mainly involves the following steps:

[0393] (1) Design of graph data model

[0394] Entity Nodes: Each extracted knowledge point is inserted into the graph database as a node. Nodes usually have multiple attributes, such as entity name, type, unique ID, etc.

[0395] Edges: The relationship between each two entities is represented as an edge. The attributes of the edge include the relationship type (such as "contains", "related", "predecessor", etc.), and the direction of the relationship.

[0396] (2) Insertion process using Neo4j as an example

[0397] Connect to the graph database: Use the Neo4j Python client (such as `neo4j-driver`) to connect to the Neo4j database:

[0398]

[0399]

[0400] (3) Insertion process using JanusGraph

[0401] For distributed graph databases such as JanusGraph, Gremlin is often used as the query language, similar to Neo4j's Cypher. To batch insert nodes and relationships through the Gremlin interface:

[0402]

[0403] The knowledge graph construction method based on fine-tuning the large language model described in this embodiment can be combined with symbolic reasoning in the disambiguation process. Symbolic reasoning can further correct the disambiguation results through known rules and logic. The specific process is as follows: (1) Knowledge point extraction: Extract potential similar knowledge points from the original document data and obtain preliminary semantic similarity through a deep learning model (such as BERT or SBERT);

[0404] (2): Semantic disambiguation: Use deep learning methods to perform preliminary similarity calculations and disambiguate some basic knowledge points;

[0405] (3): Application of symbolic reasoning rules: For knowledge points that cannot be disambiguated or have insufficient similarity, symbolic reasoning is combined with defined rule sets or logical reasoning (such as reasoning of premises and conclusions) to further correct the results based on known knowledge in the field;

[0406] (4): Reasoning optimization: The result of symbolic reasoning in the previous step is fed back to the deep learning model to further adjust the weights or thresholds of the deep learning model to reduce errors;

[0407] (5): Output the final result: Combine the final results of symbolic reasoning and deep learning disambiguation to determine whether two knowledge points are the same.

[0408] The known rules include logical rules based on logical expressions (mutual exclusivity, transitivity), ontology rules for defining the hierarchy, attributes and relationships of concepts, data consistency rules for verifying the consistency of data, semantic rules for reasoning through context or defined semantic relationships, mathematical or physical laws and priority rules.

[0409] The deep learning model (such as BERT or SBERT) obtains preliminary semantic similarity and generates fixed word vectors, which can be used to calculate semantic similarity as follows:

[0410] Word2Vec: Generates word vectors based on a context window and uses cosine similarity of the vectors to compare semantic similarity;

[0411] GloVe: Generates word embeddings based on global statistical information, suitable for semantic similarity calculation.

[0412] FastText: Extends Word2Vec to support subword representation and better handle word form changes. The rule set defined above is the core of symbolic reasoning. In specific implementations, its definition usually needs to be combined with domain knowledge. The following are some general principles and examples for defining rule sets:

[0413] The ruleset is structured as follows:

[0414] (1) Static rules, which are deterministic rules based on known facts and domain constraints. For example:

[0415] Classification rules: If a knowledge point belongs to a certain category, it may have attributes common to that category.

[0416] Association rule: Two knowledge points are equivalent or related under certain conditions.

[0417] (2) Dynamic rules, which involve data-driven patterns and can usually be combined with statistical analysis or frequent item set discovery: Conditional probability rules, if knowledge points A and B appear frequently at the same time, it can be inferred that they may be the same or related. Context-sensitive rules, that is, the meaning of knowledge points changes with the context.

[0418] The specific rule set examples are as follows:

[0419] (1) Field: Mathematical knowledge points

[0420] Equivalent rule: If the description of a knowledge point contains "equivalent to" or "also known as", they may be the same concept. TextContains(x,"also known as")→Equivalent(x,y)

[0421] If two formulas have the same transformed form, they may be equivalent, such as

[0422] FormulaSimilar(A,B)∧Transform(A,B)→Equivalent(A,B)

[0423] Derivation rule: If a formula can be transformed into another formula through simple algebraic transformation, they may be the same knowledge point;

[0424] AlgebraicTransform(Equation1,Equation2)→SamePoint(Equation1,

[0425] Equation2)

[0426] (2) Field: Biological terminology

[0427] Synonym rule: If the spelling of two terms differs only in the suffix (such as British and American spellings), they may be synonyms. Hyponym rule: If one term is a hyponym of another term and meets certain conditions, they may be partially related: (3) Cross-domain rule

[0428] Temporal correlation: If knowledge points A and B are frequently cited in the same time period, they may be related.

[0429] 4. The rule set is constructed as follows:

[0430] Based on expert knowledge: Domain experts summarize the relationships between concepts and manually define rules.

[0431] Based on data mining: extracting potential pattern generation rules from the corpus.

[0432] Use tools such as the Apriori algorithm to generate frequent itemsets.

[0433] Using knowledge graphs to mine term relationships.

[0434] Automatic generation and optimization: Combine deep learning and reinforcement learning to generate new rules and optimize them through verification feedback.

[0435] 5. Rule verification and optimization

[0436] Verification: Test the defined rules and verify the correctness of the reasoning using known data.

[0437] Optimization: Adjust rule weights based on feedback from actual applications, or design meta-rules to prioritize conflicting rules.

[0438] By combining these rule sets, symbolic reasoning can effectively correct ambiguous semantics during the disambiguation process and further improve the accuracy and interpretability of the final results.

[0439] The specific logic of reasoning, specifically, the reasoning logic can be divided into the following key steps:

[0440] 1. Formalization of input data. In order for symbolic reasoning to handle problems, the input data (knowledge points, semantic information) must first be formalized into a logical language. Common forms include:

[0441] Predicate logic: Describe the relationship between knowledge points through logical predicates.

[0442] Term("Optical diffraction") → Related("Light wave interference")

[0443] Graph structure: Knowledge points can be represented by a knowledge graph, where nodes are knowledge points and edges are relationships.

[0444] 2. Application of knowledge point matching rules, using predefined rule sets to perform pattern matching on knowledge points and their contextual information to extract potential logical relationships.

[0445] The rule matching method is as follows:

[0446] 2.1 Static matching: directly matching the conditions in the rules and comparing them with known knowledge points or context;

[0447] For example, the rule:

[0448] Synonym(a,b)→Equivalent(a,b)

[0449] If Synonym("Optical Diffraction","Diffraction") holds, then it follows that:

[0450] Equivalent("Optical Diffraction","Diffraction")

[0451] 2.2 Dynamic conditional deduction: for uncertain knowledge points, conditional judgment is performed based on the logical rules of the context;

[0452] Context("optics")∧Related("light wave","interference")→Related("optics","diffraction")

[0453] 2.3 Fuzzy rule processing: For knowledge points with insufficient semantic similarity, fuzzy logic is used to define the credibility threshold:

[0454] For example: Rule:

[0455] Similarity(a,b)>0.7→PossibleEquivalent(a,b)

[0456] If Similarity("Optical Diffraction", "Diffraction") = 0.75, then it is inferred that they are likely equivalent.

[0457] 3. Application of the inference engine. Using the inference engine to execute the rule set and combine the logic to deduce the relationship between knowledge points, the recommendation engine can achieve the following reasoning:

[0458] 3.1 Forward reasoning, that is, starting from known conditions, triggering rules in sequence to derive new knowledge point relationships;

[0459] Known:

[0460] Synonym("Optical Diffraction","Diffraction")

[0461] rule:

[0462] Equivalent(a,b)→Related(a,b)

[0463] Derivation:

[0464] Related("Optical diffraction","Diffraction").

[0465] 3.2 Backward reasoning: starting from the goal, looking backwards for the conditions that meet the goal;

[0466] 3.3 Fuzzy logic reasoning, which introduces confidence between conditions and conclusions.

[0467] 4. Conflict resolution and priority processing. In the symbolic reasoning process, multiple rule conflicts may occur, and the priority needs to be clarified:

[0468] 4.1. Priority based on weight: Define weights for rules, and rules with higher weights will be executed first.

[0469] 4.2. Meta-rule based conflict resolution: Conflict resolution strategies are defined through meta-rules.

[0470] During the reasoning process, the rule Synonym(a,b) is used to match the text, check whether the context supports it, and finally merge the reasoning results to draw a conclusion.

[0471] If RuleA∧RuleB conflict→Prefer(RuleA)

[0472] 5. Example Process

[0473] Question: Are "optical diffraction" and "interference" equivalent knowledge points?

[0474] Known:

[0475] rule:

[0476] Synonym(a,b)→Equivalent(a,b)

[0477] Context("optics")∧Related(a,b)→Equivalent(a,b)

[0478] data:

[0479] Synonym("Optical Diffraction","Diffraction")

[0480] Context("Optics")

[0481] Related("light wave","interference")

[0482] Reasoning process:

[0483] 1. Use the rule Synonym(a,b) to match:

[0484] Synonym("Optical Diffraction","Diffraction")→Equivalent("Optical Diffraction","Diffraction")

[0485] 2. Check if the context supports:

[0486] Context("optics")∧Related("light wave","interference")→Equivalent("optical diffraction","interference")

[0487] 3. Combine the inference results:

[0488] Equivalent("Optical Diffraction","Interference")∧Equivalent("Optical Diffraction","Diffraction")→True

[0489] Finally, it is concluded that "optical diffraction" is equivalent to "interference".

[0490] 5. Combined with the feedback of the deep learning model, that is, after the symbolic reasoning is completed, the results can be used as feedback to further adjust the similarity calculation or knowledge representation of the deep learning model. For example:

[0491] Update the semantic similarity model: If the symbolic reasoning result is True, adjust the similarity weight to make it closer to the corrected standard.

[0492] Retrain the semantic embedding model: add new disambiguation results to the corpus and update word vectors or relationship graphs.

[0493] In this way, symbolic reasoning and deep learning can optimize each other and improve the accuracy and robustness of overall disambiguation. The knowledge graph construction method based on fine-tuning the large language model described in this embodiment, in the disambiguation process, the disambiguation mechanism based on domain knowledge, in addition to general semantic matching and similarity calculation in a specific field, can also be combined with expert knowledge or rules in a specific field to perform disambiguation;

[0494] Implementation process:

[0495] A. Knowledge point semantic matching: Use deep learning models to perform general semantic matching on the extracted knowledge points;

[0496] B. Introduction of domain expert knowledge: Introduce expert rules, definitions, vocabularies or knowledge graphs in specific fields to further verify and correct the semantic matching results through these domain knowledge;

[0497] C. Domain rule disambiguation: According to the rule set of a specific field (such as the precise definition of different disease names in the medical field), the matching results are more finely classified or excluded; for example, for the distinction between "myocardial infarction" and "heart attack", the strict definition in the field is combined to ensure the distinction between the two;

[0498] D. Output disambiguation results: Combine the domain knowledge disambiguation results with the general semantic matching results, and output the final disambiguated knowledge point matching.

[0499] In knowledge point disambiguation, the combination method of part D is not a simple direct superposition, but through weight distribution, multi-layer decision logic or collaborative optimization, the results of domain knowledge disambiguation and general semantic matching are integrated to improve the accuracy and robustness of the final result. The following is an analysis of the specific combination method, as well as a detailed explanation of the purpose and function of the combination:

[0500] The core process of knowledge point semantic matching is as follows:

[0501] (1) The semantic representation of knowledge points transforms knowledge points into comparable semantic vectors through a deep learning model.

[0502] There are the following ways to embed word vectors:

[0503] Use pre-trained word vector models (such as Word2Vec, GloVe) to convert knowledge points into vectors.

[0504] Use language models (such as BERT, RoBERTa) to generate contextual vector representations.

[0505] If the knowledge point contains multimodal information (such as text, formula, image), a joint embedding representation can be generated through a multimodal model (such as CLIP).

[0506] (2) Semantic similarity calculation, that is, calculating the similarity between the vector representations of two knowledge points.

[0507] (3) Setting the matching threshold: Based on the similarity score, define a matching threshold:

[0508] Exact match: if similarity > 0.9

[0509] Possible match: 0.7≤similarity≤0.9

[0510] No match: Similarity < 0.7

[0511] It should be noted that the setting of the threshold can be adjusted according to the actual field.

[0512] 3. Tools and models for implementing general semantic matching can use the following models and tools:

[0513] (1) Pre-trained language model: Use a model pre-trained on large-scale corpus to generate semantic embeddings, for example:

[0514] BERT, RoBERTa: suitable for text semantic matching; SciBERT: optimized for scientific literature; Sentence-BERT: generates sentence-level semantic vectors, suitable for calculating similarity.

[0515] (2) Knowledge base and dictionary: use general knowledge base (such as WordNet, ConceptNet) to expand semantic information and use domain-related dictionaries to supplement terminology.

[0516] (3) Semantic similarity calculation library, including Python tools, scikit-learn: provides cosine similarity and Euclidean distance calculation; sentence-transformers directly calculates embedding vector similarity.

[0517] Example: The complete process of general semantic matching

[0518] Question: Are “optical diffraction” and “wave interference” the same knowledge point?

[0519] 1 Generate embeddings using BERT:

[0520] "Optical Diffraction" → [0.32, 0.45, 0.76]

[0521] "Interference of waves" → [0.33, 0.44, 0.78]

[0522] 2. Calculate cosine similarity: CosineSimilarity = 0.98

[0523] 3. Matching rule verification: Verify by checking spelling variants:

[0524] If "optical diffraction", "interference of waves"

[0525] EditDistance("Optical Diffraction","Wave Interference")>3→NoMatchEditDistance(a,b): represents the edit distance between strings a and b. The edit distance (Levenshtein Distance) is a common metric for measuring the similarity between two strings. It represents the minimum number of edit operations required to transform one string into another.

[0526] These edits typically include:

[0527] Insert a character;

[0528] Delete a character;

[0529] Replaces a character.

[0530] >3: Indicates that the edit distance is greater than 3; that is, the number of edit operations between strings a and b is greater than 3.

[0531] NoMatch: If the condition is met, that is, the edit distance is greater than 3, a and b are considered to be unmatched (NoMatch).

[0532] Check context overlap: If the context overlap is greater than or equal to 0.7, then it is a possible match;

[0533] 4. Matching threshold judgment: judge as "possible match" based on similarity and rule results;

[0534] In the process of combining the domain knowledge disambiguation results with the general semantic matching results, weighted fusion is used to combine them.

[0535] Different weights are assigned to the domain knowledge disambiguation results and the general semantic matching results, and the two are weighted according to the importance of the actual task;

[0536] The details are as follows:

[0537] General semantic similarity score: S1 = 0.8

[0538] Domain rule disambiguation score: S2 = 0.9

[0539] Weighted formula: S final =w1·S1+w2·S2

[0540] Among them, w1 and w2 are weight coefficients, which can be adjusted through experiments;

[0541] Assume the weights are w1=0.4, w2=0.6

[0542] S final =0.4·0.8+0.6·0.9=0.86

[0543] Just determine whether it matches based on the set threshold;

[0544] The combination of weighted fusion can effectively overcome the limitations of a single method and make the final disambiguation result more comprehensive and accurate. The details are as follows:

[0545] (1) Make up for the shortcomings of a single method

[0546] Disadvantages of general semantic matching: General semantic matching is weak in processing domain-specific terms or rare knowledge points, which may lead to misjudgment or omission. For example, a general model may mistake "optical diffraction" and "wave interference" as unrelated.

[0547] Disadvantages of domain rule disambiguation: Domain rules are highly dependent on known expert knowledge and may be powerless for unknown or uncovered situations;

[0548] By combining the broad coverage of general semantic matching and the high precision of domain rules, the disambiguation results are made more reliable.

[0549] (2) Handling special situations,

[0550] Some matching pairs may have low scores in general semantic matching, but domain knowledge shows that the two are closely related;

[0551] For example: General matching result: Similarity = 0.6

[0552] Domain rules: rules indicate that there is a causal relationship between the two;

[0553] After combination: determined as a match;

[0554] (3) Improve confidence and explainability. The introduction of domain rules provides a clear logical basis, making the matching results more explainable, while the general semantic matching results provide quantitative support.

[0555] (4) Optimize model performance,

[0556] Combined with the feedback from domain rules, the parameters of the semantic model can be dynamically adjusted to gradually adapt it to domain requirements.

[0557] Although the domain rule disambiguation in Part C has classified and excluded the common semantic matching results, the following reasons make the combination more necessary:

[0558] (1) Improve matching coverage,

[0559] The coverage of domain rules is limited and cannot cover all knowledge points. Combined with general semantic matching, it can handle parts that are not defined or covered by the rules;

[0560] (2) Handling low confidence matches

[0561] In semantic matching and rule disambiguation, some knowledge points may be in a boundary state at the same time:

[0562] For example, the semantic similarity score is close to the threshold or the domain rules are contradictory or incomplete;

[0563] By combining the results from both, these edge cases can be treated more comprehensively;

[0564] (3) Verification and calibration results, combined with the results of general semantic matching, can verify the judgment of domain rules, thereby reducing errors;

[0565] (4) Improve scalability. Combined with semantic matching, the system can still output valid results even when domain rules are insufficiently covered, making the model more general and flexible.

[0566] The following are some examples:

[0567] Determine whether "interference of waves" and "optical diffraction" are the same

[0568] 1. General semantic matching:

[0569] Semantic similarity score: 0.6 (below the matching threshold);

[0570] Initial judgment: mismatch;

[0571] 2. Domain rule disambiguation:

[0572] Rule 1: If the knowledge point contains "wave" and "diffraction", the correlation is high;

[0573] Rule application results: relevance score 0.9;

[0574] 3. Weighted fusion:

[0575] Weight distribution: w1=0.3, w2=0.7;

[0576] Final score:

[0577] S final =0.3·0.6+0.7·0.9=0.78

[0578] From the above results, it can be seen that the match threshold is exceeded and it is determined to be a match.

[0579] 4. Output result: Knowledge point matching: "Wave interference" = "Optical diffraction"

[0580] Therefore, combining general semantic matching and domain rule disambiguation not only improves coverage and confidence, but also provides the model with the ability to dynamically adjust and expand. This combined strategy can output more accurate and reliable disambiguation results when a single method is insufficient.

[0581] For example, although "myocardial infarction" and "heart attack" in the medical field are similar in general semantics, they have more subtle differences in some literature, so they need to be strictly distinguished in combination with domain rules.

[0582] In the knowledge graph construction method based on fine-tuning a large language model described in this embodiment, in order to ensure the accuracy of the inserted knowledge points and relationships during the knowledge graph construction process, the following multiple means are used to ensure the accuracy, as follows:

[0583] (1) Multi-level verification

[0584] Rule-based verification: Different verification rules are defined for different fields to detect knowledge points and relationships. If data that does not meet the verification rules is matched, potential errors will be marked, and the knowledge points and relationships marked as not wrong will be deleted. For example, in the field of education, "course" and "chapter" should have a containment relationship. If the relationship type between them is detected to be "related", it will be marked as a potential error.

[0585] Contextual consistency verification: By analyzing the context of the knowledge point, it ensures that the extracted entities and relationships are consistent with the logic in the text. For example, the relationship between "calculus" and "advanced mathematics" is "included" rather than "preceded". Contextual verification can automatically discover these logical inconsistencies.

[0586] Domain expert review: For knowledge in a specific field, a knowledge base annotated by experts can be introduced for comparison to ensure that the extracted knowledge points are consistent with the information in the existing knowledge base;

[0587] (2) Semantic similarity matching

[0588] Use semantic similarity calculation to verify the extracted knowledge points. For example, use language models such as BERT and SBERT to vectorize the extracted knowledge points and standard knowledge points, and then calculate the similarity to determine whether the knowledge points are accurately extracted. If the similarity is lower than the set threshold, it is marked as a potential error and deleted.

[0589] It should be noted that the threshold is usually adjusted dynamically according to specific task requirements and application scenarios, rather than a fixed range.

[0590] Semantic similarity models such as BERT and SBERT: The output similarity is usually in the interval [0,1], representing the cosine similarity or dot product result of two vectors:

[0591] High threshold (0.8-0.9): Applicable to scenarios with high matching requirements, such as building knowledge graphs in highly sensitive fields such as medicine and law; Purpose: To avoid introducing erroneous knowledge points as much as possible, even if it will sacrifice a certain recall rate.

[0592] Medium threshold (0.6-0.8): Applicable to generalized scenarios, such as education and general domain knowledge. Purpose: To ensure accuracy while taking into account knowledge point coverage.

[0593] Low threshold (0.4-0.6): Applied to exploratory tasks, such as generating new knowledge or knowledge expansion in unlabeled domains. Purpose: Allowing certain fuzzy matching to discover potential related knowledge;

[0594] In this case, the threshold is preferably [0.4, 0.9].

[0595] (3) Relational logic verification

[0596] Relationship type restrictions: ensure that the relationship type conforms to the logic within the domain, and perform logical verification on the relationship type through the domain rule library; for example, in the education field, the relationship between "course" and "chapter" should be "included" instead of "preceded";

[0597] Loop detection: Use graph algorithms to detect whether there are unreasonable loops in the knowledge graph to avoid unreasonable knowledge graph structure caused by incorrect relationships.

[0598] In the knowledge graph construction method based on fine-tuning a large language model described in this embodiment, the means and process of verifying the accuracy of extracting knowledge points and relationships in the data source in S2) are as follows:

[0599] Before storing the knowledge points and relationships in the graph database, use the following steps to verify the accuracy of the extracted knowledge points and relationships:

[0600] (1): Automated test dataset

[0601] Build a standard annotated data set and input it into the AI ​​model as a benchmark. Then compare the knowledge points extracted by the model with the standard answers. If the extracted knowledge points and relationships are consistent with those in the annotated data set, the verification is passed; otherwise, the model needs to be further debugged or the algorithm needs to be adjusted.

[0602] Accuracy: number of correctly extracted knowledge points / total number of extracted knowledge points;

[0603] Recall rate: the number of correctly extracted knowledge points / the number of knowledge points in the standard set;

[0604] (2): Manual review and expert feedback

[0605] Knowledge points in specific fields are manually reviewed by experts to ensure that the extracted content meets industry standards; for example, in the medical field, doctors can review the extracted terms and relationships;

[0606] (3): Context verification

[0607] After knowledge points are extracted, secondary verification is performed in combination with their context to ensure that the extracted knowledge points are consistent with their meanings in the context. For example, "Java" as a knowledge point of programming language may also represent other concepts in different contexts. This error can be reduced through context verification.

[0608] (4): Model adaptive optimization

[0609] By continuously collecting error cases extracted by the model, analyzing and providing feedback, adding this feedback to the training data, and continuously optimizing the performance of the model, this circular feedback mechanism can effectively improve the accuracy of extraction.

[0610] In the process of building the knowledge graph, complex knowledge graphs can be built by inserting the extracted knowledge points and relationships into the graph database using tools such as Neo4j and JanusGraph. At the same time, the accuracy and logic of the inserted data can be ensured through context consistency checks, rule-based validation, semantic similarity matching, and loop detection. In order to further verify the accuracy of knowledge points and relationships, various methods such as automated data set testing, expert review, and context verification can be used to ensure the high quality and accuracy of the knowledge graph.

Claims

1. A knowledge graph construction method based on fine-tuning a large language model, characterized by: include: The following steps: S1): data preprocessing, that is, processing the data source of the knowledge points to be extracted, the data source of the knowledge points to be extracted includes structured data and unstructured data, the structured data is directly input into the mapper, and the text data in the unstructured data is cleaned; S2): Extract knowledge points and relationships from the data source, first build a dedicated knowledge point extraction fine-tuning model, input the pre-processed data into the knowledge point extraction fine-tuning model, automatically identify entities in the text through the knowledge point extraction fine-tuning model, and extract the relationships between the entities to obtain pre-processed knowledge points and relationships; S3): Knowledge point and relationship disambiguation, that is, identification and mapping of different descriptions of the same knowledge point; the specific disambiguation process is as follows: S31): Establishing a standard knowledge point library, that is, first constructing a standard set of knowledge point entities, and using the established standard set of knowledge point entities as a mapping target set to disambiguate knowledge points extracted from the text; S32): Similarity calculation and mapping, i.e., using text embedding technology in natural language processing to convert knowledge points and related texts into vectors; then similarity calculation is performed to determine whether the extracted different expressions point to the same knowledge point by calculating the cosine similarity or Euclidean distance between the text embedding vectors. The cosine similarity is used to measure the angle between the vectors to determine the similarity between the two. S33): Semantic disambiguation based on context: To ensure the accuracy of the disambiguation process, the sliding window technology is used to take context information into consideration, so that in the process of identifying knowledge points, not only the current sentence but also the previous and next sentences are considered to extract the knowledge point relationship from the pre-processed knowledge points; S34): Iterative verification and feedback optimization: After disambiguation, the knowledge points and relationships extracted in the previous step will be mapped to the knowledge point entity standard set, and the two will be matched to find possible corresponding relationships. Further verification will be performed through the extracted relationships. If there is no match, the entity will be deleted; S4): Knowledge graph construction: First, establish a graph database model, innovate entity nodes, and create relationships between entity nodes to form relationship edges. Then, store the disambiguated entity nodes and relationship edges into the graph database model to form a dynamic and scalable knowledge graph. In the later use process, use the AI ​​model regularly to re-analyze text materials and update the knowledge graph to ensure that it reflects the latest knowledge points and their changes in a timely manner.

2. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 1, characterized in that: The specific method for performing data cleaning on text data in unstructured data in S1) is as follows: S11): Text cleaning: First, clean the text data in the unstructured data, use the deep learning model to classify different text areas in the text, and distinguish the irrelevant content from the main text through the context and semantic features of the text to automatically identify the main text and irrelevant content, and remove the irrelevant content through regular expressions; eliminate the noise in the text data, and delete the extra spaces, special characters, irrelevant HTML tags, and repeated text; S12): Sentence processing: In order to provide a clear context for subsequent knowledge point extraction, the cleaned text needs to be sentence-separated; S13): Extract repeated content, extract repeated content in the text data by detecting repeated sentences and repeated paragraphs, specifically: Detect duplicate sentences, using text similarity detection algorithms to identify and merge similar or duplicate sentences; Detect duplicate paragraphs: Detect longer texts through text comparison and clustering algorithms, and merge duplicate paragraphs; S14): Context preservation: In order to preserve the original context of the text material, the following measures are taken: Maintain paragraph structure: While dividing sentences, maintain the paragraph structure of the original text so that the complete paragraph information can be referenced when extracting knowledge points; Use sliding windows: In order to capture a wider range of contextual information, a sliding window technique can be used when extracting knowledge points, which allows the model to consider the information of a sentence and its surrounding sentences at the same time.

3. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 2, characterized in that: The specific process of sentence processing in S12) is as follows: S121): Based on punctuation: using periods, question marks, exclamation marks and other punctuation marks as sentence separators; S122): Sentence segmentation using AI models or natural language processing libraries based on natural language processing tools that can more accurately identify sentence boundaries even in the absence of obvious punctuation; S123): Dependency-based syntactic analysis: By analyzing the dependency relationship in the sentence, the main and subordinate structures of the sentence are determined, so as to divide the sentence more accurately; The core idea is to decompose a sentence into a "dependency" graph between words, so as to determine the predicate verb, subject, object elements in the sentence and their relationships.

4. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 3, characterized in that: The specific analysis process of the dependency syntax analysis in S123) is as follows: S1231): Input preprocessing, which includes word segmentation, denoising, and normalization of the input text; the details are as follows: Text cleaning: First, the input text data needs to be cleaned and preprocessed, specifically removing noise, symbols, irrelevant HTML tags, and normalizing complex text, such as converting it to lowercase and correcting typos; Word segmentation: Before analyzing the syntax, use word segmentation tools to split the sentence into individual words or phrases; S1232): Part-of-speech tagging First, use natural language processing tools to mark the part of speech for each word. That is, use the part-of-speech tagger to tag each word or phrase after word segmentation and determine the grammatical role of each word or phrase in the sentence, such as noun, verb, or adjective. Part-of-speech tagging provides basic information for subsequent dependency analysis. Simplify the parts of speech: Simplify the parts of speech according to the specific application scenarios of words or phrases, such as simplifying detailed verb forms into a single "verb" category; S1233): Dependency relationship analysis, using the dependency syntax analysis model to generate a dependency relationship tree, as follows: First, sentence component analysis is performed: through dependency syntactic analysis, the dependency relationship between words in the sentence is analyzed to obtain a dependency tree. The goal of dependency syntactic analysis is to find the subordinate relationship between the "core word" and other words in the sentence; Then determine the core components of the sentence: Analyze the predicate verb in the sentence and the related subject, object, and adverbial components. For example, `subject` depends on `verb`, and `object` also depends on `verb`. These components constitute the basic structure of the sentence. Model selection: Select the appropriate dependency syntax model according to the actual application and build dependency relationships; S1234): Extraction of main structure, i.e., analyzing the subject, predicate, object, attributive, adverbial and complement in the sentences in the text to identify the main components of the sentences; S1235): Dependency tree construction, using visualization tools to display dependency relationships, so as to analyze sentence structure more intuitively, as follows: Dependency tree structured representation: A dependency tree is generated for each sentence. The root node of the tree is the predicate verb of the sentence. Other words are attached to the trunk of the dependency tree in sequence according to the dependency relationship. The dependency tree makes it easy to see the master-slave relationship of the components in the sentence, thereby achieving grammatical analysis and sentence reorganization. Dependency tree visualization: Use spaCy's displaCy tool or other dependency tree visualization tools to visualize the parsed dependency relationships, which helps to understand the sentence structure more intuitively. S1236): Syntax rule verification and optimization Dependency tree trunk and subordinate structure identification: By analyzing the dependency tree, the trunk structure of the sentence can be determined, that is, the predicate verb and its main modifying components. The subject and object directly depend on the predicate verb, while the attributive and adverbial subordinate components depend on the core component. Optimization of syntactic rules: According to the needs of the field, some specific syntactic rules are set to correct dependency relationships. For example, for complex sentence patterns such as compound sentences and parallel sentences, conjunctions can be processed through dependency syntax to clarify the position and function of conjunctions in the sentence structure and ensure the accuracy of semantic dependency relationships.

5. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 1, characterized in that: In the S3), the context embedding model is used in the knowledge point and relationship disambiguation to convert the extracted knowledge points and their contexts into high-dimensional vectors, thereby achieving more accurate semantic matching. Compared with traditional word vectors, context embedding can better capture the different meanings of the same word in different contexts and improve the robustness of disambiguation. The details are as follows: For the input text t, use the contextual embedding model BERT to convert the input text into a high-dimensional vector representation v; υ t =BERT(t) Among them, t is the context embedding vector of text t, Let cosine similarity calculate the angle between two vectors, defined as: Among them, υ1 and υ2 are the contextual embedding vector representations of the two texts; In dynamic weighted cosine similarity, a weight w is assigned to each dimension, considering that different dimensions contribute differently to the similarity. i , so that the similarity calculation can more flexibly reflect the important information in the text: Where d is the dimension, usually d is the hidden layer size of the BERT model; Among them, w i is the weight used for weighted cosine similarity, which is a weight dynamically calculated based on the context or category, It is used to measure the contribution of different dimensions to similarity. If two knowledge points are consistent in a certain category, they can be added. The weight w of the dimension related to the category i ; The w i Based on the following factors Term frequency (TF): the frequency with which a word appears in a text, reflecting its importance; Inverse Document Frequency (IDF): how rare a word is in the entire corpus; <h2 style=";text-align:left;direction:ltr">w<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =TFi·IDFi It can also be calculated by semantic level similarity, as follows: Assume two sentences s1 and s2, their semantic vector representations are and SBERT uses cosine similarity to measure the similarity between two sentences s1 and s2: Compared with traditional sentence representation methods, SBERT can capture more fine-grained semantic information, making the similarity calculation between two sentences at the semantic level more accurate; Multi-level similarity: In order to match semantics more precisely, the similarity of knowledge points at multiple levels can be calculated; the details are as follows: First, the similarity based on the word level is calculated, and then the similarity at the sentence level or paragraph level is calculated. The final similarity can be the weighted average of the similarities at each level: total_similarity(x1,x2)=α·word_similarity(x1,x2)+β·sentence_similarity(x1,x2)+γ·documentt_similarity(x1,x2) Among them, α, β, and γ are the weights of similarity at different levels, which are adjusted according to the specific application scenario; In the processing process, if there are multiple representations of the same entity, we fine-tune the large model to determine whether they are the same knowledge point, and then perform mapping recognition to obtain the final knowledge points and relationships; Mapped to the knowledge point entity standard set through similarity calculation, It also includes a mapping method, and introduces context-aware semantic matching in the matching process by capturing the context of knowledge points in the text; context analysis combines the context information of surrounding words and sentences to enhance the ability to distinguish synonyms or related knowledge points.

6. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 5, characterized in that: The specific construction process of the knowledge point extraction fine-tuning model in S2) is as follows: (1) Data preparation, that is, preparing enough domain-related text data, including annotations of knowledge points. For different expressions of knowledge points, it is necessary to prepare a standardized set of knowledge points to facilitate subsequent mapping and verification. The data source can be manually annotated text or use existing knowledge bases, such as Wiki or expert-annotated domain data sets; (2) Model selection Choose a pre-trained large language model (such as BERT, GPT, etc.) as the pre-training model. This model has been pre-trained with massive general data and has good language understanding capabilities. For the task of knowledge point extraction, you can choose a model based on the Transformer architecture. These models perform well in natural language processing tasks. (3) Label and clean the text Preprocess the data, including text cleaning (removing noise and irrelevant content) and sentence segmentation. Then, use annotation tools or write rules to mark entities and relationships in the text. This step can use tools such as SpaCy or NLTK for automatic sentence segmentation and part-of-speech tagging. (4) Fine-tune the pre-trained model to obtain a fine-tuned knowledge point extraction fine-tuning model. For the knowledge point extraction task, the pre-trained model such as BERT or GPT is fine-tuned to obtain the fine-tuned model. The training process of the pre-trained model is completed through the following stages: Entity recognition: Use pre-trained models to automatically identify entities in text, such as concepts, facts, and principles; Relationship extraction: extract the relationships between text entities automatically identified by the pre-trained model, such as inclusion, subordination, and predecessor; Disambiguation processing: For different descriptions of the same knowledge point, vectorization technology is used to calculate similarity and map similar expressions to the same entity; (5) Model verification and optimization The fine-tuned model is tested on the validation set to evaluate its accuracy in knowledge point recognition and relationship extraction tasks. The similarity algorithm is used to verify the accuracy and consistency of the extracted entities and relationships. If the accuracy and consistency meet the requirements, it means that the model can meet the needs.

7. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 6, characterized in that: The specific methods for model verification and optimization are as follows: Adaptive threshold adjustment: During the similarity calculation process, the similarity threshold is dynamically adjusted by continuously learning the accuracy of historical data; The specific process is as follows: Data initialization: Collect the initial historical data of similarity calculation, including the results of each similarity calculation and the correctness of actual verification; Initial threshold setting: Set the similarity threshold based on common empirical values, such as the default threshold of cosine similarity is 0.8; Similarity calculation: For each knowledge point to be processed, a preliminary similarity score is obtained based on cosine similarity, SBERT or other similarity algorithms; Feedback collection: Based on the results of previous processing, the system collects feedback data from users or experts and marks whether each resolution is correct; Adaptive learning: The system uses machine learning algorithms to dynamically adjust the similarity thresholds based on accumulated feedback data and errors in disambiguation results. The adjustment range is usually between 0.05 and 0.

10. The thresholds can be adaptively changed based on different fields, text length, and semantic complexity. Apply dynamic threshold: In a new round of similarity calculation, a dynamically adjusted threshold is used to determine whether two knowledge points are considered the same.

8. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 6, characterized in that: In the process of disambiguation, symbolic reasoning can be combined. Symbolic reasoning can further correct the disambiguation results through known rules and logic. The specific process is as follows: (1) Knowledge point extraction: Extract potential similar knowledge points in the original data from the literature and obtain preliminary semantic similarity through the deep learning model; (2): Semantic disambiguation: Use deep learning methods to perform preliminary similarity calculations. Those with similarities lower than the target value will be deleted to disambiguate some basic knowledge points; (3): Application of symbolic reasoning rules: For knowledge points that cannot be disambiguated or have insufficient similarity, symbolic reasoning is combined with defined rule sets or logical reasoning to further correct the results based on known knowledge in the field; (4): Reasoning optimization: The result of symbolic reasoning in the previous step is fed back to the deep learning model to further adjust the weights or thresholds of the deep learning model to reduce errors; (5): Output the final result: Combine the final results of symbolic reasoning and deep learning disambiguation to determine whether two knowledge points are the same.

9. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 6, characterized in that: In the process of disambiguation, the disambiguation mechanism based on domain knowledge can combine expert knowledge or rules in a specific field to disambiguate in addition to general semantic matching and similarity calculation in a specific field; Implementation process: A. Knowledge point semantic matching: Use deep learning models to perform general semantic matching on the extracted knowledge points; In the disambiguation task, the general semantic matching of Part A mainly relies on the deep learning model to compare knowledge points at the semantic level. This process involves specific semantic embedding representation, similarity calculation and matching rules. The following are the specific implementation methods and corresponding matching rules: B. Introduction of domain expert knowledge: Introduce expert rules, definitions, vocabularies or knowledge graphs in specific fields to further verify and correct the semantic matching results through these domain knowledge; C. Domain rule disambiguation: more finely classify or exclude matching results based on a domain-specific rule set; D. Output disambiguation results: Combine the domain knowledge disambiguation results with the general semantic matching results, and output the final disambiguated knowledge point matching. In knowledge point disambiguation, the combination method of part D is not a simple direct superposition, but the results of domain knowledge disambiguation and general semantic matching are integrated through methods such as weight distribution, multi-layer decision logic or collaborative optimization to improve the accuracy and robustness of the final result.

10. The method for constructing a knowledge graph based on a fine-tuned large language model according to claim 6, characterized in that: The means and process of verifying the accuracy of extracting knowledge points and relationships in the data source in S2) are as follows: Before storing the knowledge points and relationships in the graph database, use the following steps to verify the accuracy of the extracted knowledge points and relationships: (1): Automated test dataset Build a standard annotated data set and input it into the AI ​​model as a benchmark. Then compare the knowledge points extracted by the knowledge point extraction and fine-tuning model with the standard answers. If the extracted knowledge points and relationships are consistent with those in the annotated data set, the verification is passed; otherwise, the knowledge point extraction and fine-tuning model needs to be further debugged or the algorithm needs to be adjusted. Accuracy: number of correctly extracted knowledge points / total number of extracted knowledge points; Recall rate: the number of correctly extracted knowledge points / the number of knowledge points in the standard set; (2): Manual review and expert feedback Knowledge points in specific fields are manually reviewed by experts to ensure that the extracted content meets industry standards; for example, in the medical field, doctors can review the extracted terms and relationships; (3): Context verification After knowledge points are extracted, secondary verification is performed in combination with their context to ensure that the extracted knowledge points are consistent with their meanings in the context. For example, "Java" as a knowledge point of programming language may also represent other concepts in different contexts. This error can be reduced through context verification. (4): Model adaptive optimization By continuously collecting error cases extracted by the knowledge point extraction fine-tuning model, analyzing and providing feedback, adding this feedback to the training data, and continuously optimizing the performance of the model, this circular feedback mechanism can effectively improve the accuracy of extraction.

Images