A knowledge extraction method based on a large language model and dynamic pattern induction

Abstract

The application discloses a knowledge extraction method based on a large language model and dynamic mode induction, and belongs to the field of artificial intelligence and knowledge graph construction. The method of the application obtains unstructured text data in a target application scenario, uses a large language model to progressively extract entity pairs and semantic relationships between entities, events and entities participating in the events, and time sequences or causal relationships between events from the text; then, abstract-level concept phrases are generated for each entity, event and relationship, semantic embedding vector representation and clustering are performed, core labels and cluster abstracts are generated for each concept cluster, and a hierarchical knowledge graph output is formed, with concept clusters as upper semantic nodes and entities, events and relationships as lower instance nodes. The method of the application realizes cross-domain adaptive knowledge extraction, can automatically identify entities and relationships in different fields without predefining a schema, greatly improves the efficiency of knowledge graph construction, and improves the completeness and timeliness of knowledge extraction.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and knowledge graph construction, specifically involving a knowledge extraction method based on large language models and dynamic pattern induction. Background Technology

[0002] With the deepening development of scientific research, the volume of knowledge in various fields is growing exponentially, and the knowledge system presents a complex network structure, leading to an increasingly serious problem of information overload. Knowledge graphs, as a structured knowledge representation method, can effectively organize and associate massive amounts of information and have been widely applied in fields such as search engine optimization and intelligent question-answering systems. Knowledge graph construction is a complex process involving data collection, processing, integration, and knowledge extraction and storage. Triples, as an important foundation for knowledge graph construction, refer to information about things in the natural world extracted in the form of (head entity, relation, tail entity), and their extraction quality directly determines the completeness and reliability of the knowledge graph. Traditional methods for constructing knowledge graphs have significant shortcomings in the data collection, entity labeling, and relation extraction stages: early unsupervised knowledge extraction methods directly extracted triples through clustering or dependency parsing, but these methods usually require manually customized rules, consuming a lot of tedious human labor, and have poor ability to interpret semantically similar relations such as "capital" and "city," resulting in blurred classification boundaries. Therefore, the field of knowledge extraction has gradually shifted towards supervised training methods.

[0003] Supervised knowledge extraction algorithms can be divided into two categories: pipelined algorithms and joint extraction models. Traditional knowledge extraction methods are pipelined methods, which first identify entities and their types from the original text, and then classify relationships for each pair of candidate entities. While simple and intuitive, this approach doesn't fully utilize the correlation between entities and relationships, and is prone to cascading errors and exposure bias between the two subtasks. To address this issue, researchers have proposed joint entity-relationship extraction models, achieving good results. Current joint extraction models are diverse: annotation-based methods determine the start and end positions of entities or relationships between entities through sequence labeling; graph-based methods apply GCNs (Graph Convolutional Networks) to extraction tasks, proposing graph-based joint extraction models. Although supervised methods have made significant progress compared to unsupervised methods, they still require a large amount of manually labeled data, resulting in high costs and long processing times. Furthermore, data sources are limited; the data in each existing data source is fixed, and even without considering extraction efficiency, the knowledge that can be extracted has a definite upper limit.

[0004] Existing methods, whether supervised pipeline methods or joint extraction models, all suffer from the following core problems:

[0005] a. Reliance on predefined schemas: Entity types and relationship types need to be defined in advance, lacking cross-domain adaptability;

[0006] b. High data annotation cost: It requires a large amount of manually annotated training data, and the annotation process is time-consuming and requires the participation of domain experts;

[0007] c. Poor timeliness of knowledge: The training data is fixed and it is difficult to reflect changes and updates in new knowledge in a timely manner;

[0008] d. High technical threshold: Building and training deep learning models is quite difficult for non-computer professionals.

[0009] In recent years, large language models, exemplified by GPT, have demonstrated powerful semantic understanding and knowledge generation capabilities. Through pre-training on massive datasets, they have built-in extensive cross-domain knowledge. Unlike traditional knowledge bases, large language models store knowledge implicitly (through parameter weight distribution), providing a new technical path for knowledge extraction. Current research has attempted to guide large language models to generate knowledge using prompting engineering, but a systematic extraction framework and dynamic pattern construction mechanism are lacking. Furthermore, this approach of using prompting engineering to guide large language models in knowledge generation suffers from reliance on prior knowledge, insufficient generality, and potential omission of unknown information.

[0010] Chinese invention patent application CN121351975A, published on January 16, 2026, discloses a method and system for constructing a trauma knowledge graph based on a three-level fault-tolerant mechanism. This method pre-establishes a trauma injury rule base, dynamically generates target prompt words based on the rule base using a large language model, processes the target text based on the prompt words using the large language model to generate candidate files, verifies the candidate files to obtain verification files, and constructs a trauma knowledge graph based on the verification files. However, this technical solution relies on a core knowledge framework and rule base specific to a particular domain for knowledge graph construction, resulting in poor cross-domain adaptability. Chinese invention patent application CN121351972A, also published on January 16, 2026, discloses a method for constructing a knowledge graph based on active learning and incremental learning. This method preprocesses heterogeneous data sources and extracts entities, relationships, and attributes to form an initial knowledge network. After vectorization and alignment, an initial graph is obtained. Active learning is used to select candidate triples for annotation and model optimization. Then, incremental learning is used to incorporate new data and resolve knowledge conflicts. The knowledge graph construction in this technical solution requires a large amount of user-annotated data, resulting in high construction costs and long cycles. All of the above technical solutions lack a dynamic concept summarization mechanism, have insufficient hierarchical structure and scalability of the knowledge system, and cannot adapt to changes in new knowledge in a timely manner.

[0011] Existing knowledge extraction technologies suffer from the following drawbacks, which limit their generalization ability across different fields and their promotion and effectiveness in practical applications:

[0012] First, existing methods heavily rely on predefined schemas, resulting in extremely poor cross-domain adaptability. Traditional methods require the pre-definition of fixed entity and relation types, i.e., schemas. This design necessitates redesigning the entire schema and re-annotating large amounts of data when applied to new domains. It cannot automatically adapt to the semantic features of knowledge from different domains, and it struggles to capture domain-specific complex relation types, thus limiting the transferability of the technology.

[0013] Secondly, manual annotation is costly and time-consuming. Supervised learning methods rely on a large amount of high-quality labeled data as the training foundation, typically requiring thousands to tens of thousands of labeled samples for each domain. The annotation process not only requires deep involvement from domain experts, which is extremely costly, but the annotation quality also directly affects model performance. However, in practice, it is difficult to guarantee the consistency and accuracy of annotations, which has become a major bottleneck in knowledge graph construction.

[0014] Third, the knowledge coverage is limited and the timeliness is poor. Methods based on fixed data sources have inherent limitations; even without considering the efficiency of knowledge extraction, there is a clear upper limit to the knowledge that can be extracted. Due to the slow updates of data sources, these methods cannot reflect the latest knowledge changes in a timely manner and are difficult to effectively integrate multi-source heterogeneous knowledge, causing the constructed knowledge graph to gradually become outdated and affecting its application value in practical applications. Summary of the Invention

[0015] To address the shortcomings of current knowledge extraction technologies, such as limitations imposed by predefined schemas, weak cross-domain adaptability, and high costs of manual intervention, this invention provides a knowledge extraction method based on large language models and dynamic pattern induction. This method overcomes the aforementioned deficiencies in existing knowledge extraction methods, enables cross-domain adaptive knowledge extraction, and can automatically identify entities and relationships in different domains without the need for predefined schemas. This significantly reduces the cost of manual intervention and improves the completeness and timeliness of knowledge extraction.

[0016] The knowledge extraction method of this invention, based on a large language model and dynamic pattern induction, first obtains source data from the target application scenario. The source data is unstructured text. Then, knowledge extraction is performed on the source data to obtain a hierarchical knowledge graph. The method of this invention includes the following steps:

[0017] Step 1: Extract knowledge relation triples from the source data using a large language model, including:

[0018] (1) The text preprocessing module preprocesses the source data, removes irrelevant language content and special characters, and unifies the text format;

[0019] (2) Different types of knowledge relationships are extracted through a three-stage progressive process, including: First, the entity-entity relationship extraction module uses the prompt word P_EE to guide the large language model to identify entity pairs and semantic relationships between entities in the text; second, the entity-event relationship extraction module uses the prompt word P_EV to guide the large language model to identify events in the text and the entities involved in those events; finally, the event-event relationship extraction module uses the prompt word P_VV to guide the large language model to identify temporal or causal relationships between events in the text; the prompt words P_EE, P_EV, and P_VV each contain four parts: task description, constraints, output format specifications, and example demonstration; among them,

[0020] The constraints of the prompt word P_EE include: entities must be concrete nouns, excluding pronouns; the length of the relation description shall not exceed 10 words; each entity pair shall retain only the most important relation;

[0021] The constraints of the prompt word P_EV include: the event must be presented as an independent and complete sentence; the role of the entity in the event must be clearly marked; an event can be associated with multiple entities;

[0022] The constraints of the prompt word P_VV include relation type constraints: temporal relations: before, after, at the same time; causal relations: because, as a result.

[0023] (3) The post-processing module performs format repair on the extracted knowledge relation triples to ensure the structural integrity of the triples and stores the knowledge relation triples;

[0024] Step 2 involves conceptual generalization, abstracting the extracted entities, events, and relationships into generalized semantic types, including:

[0025] (1) The concept generation module generates at least three concept phrases at different levels of abstraction for each entity by using a large language model and combining the entity’s context; it generates at least three concept phrases at different levels of abstraction for each event and relationship based on its own text description, and the generated concept phrases avoid repeating the original text; all generated concept phrases form the initial concept pool.

[0026] (2) The hierarchical clustering module clusters the concept phrases of entities, events and relations in the initial concept pool and generates general concepts using a large language model. This includes: First, generating a domain-specific definition for each concept phrase in the initial concept pool, converting the concept phrase and the domain-specific definition into semantic embedding vectors and concatenating them to obtain the embedding vector of the concept phrase; then, using the k-means clustering algorithm to cluster the embedding vectors of the concept phrases and perform semantic aggregation; finally, for each aggregated concept cluster, using the large language model to combine all concept phrases, domain definitions, domain context and clustering semantic features within the cluster to generate a core label and cluster summary, and establishing an association between each entity, event and relation extracted in step 1 and the concept cluster to form a hierarchical knowledge graph with concept clusters as upper-level semantic nodes and entities, events and relations as lower-level instance nodes.

[0027] In step 2, the hierarchical clustering module uses the k-means clustering algorithm to cluster concept phrases. The objective function for clustering is as follows:

[0028] ;

[0029] in, A cluster set, containing clusters Clusters, For the first Cluster The center vector, For the first Embedding vectors of concept phrases; through iterative optimization, concept phrases with the same semantics are clustered into the same cluster; number of clusters. The optimal value is determined using the contour coefficient formula.

[0030] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0031] (1) The method of this invention can automatically identify entities and relationships in different domains without predefined schemas, without requiring a large amount of labeled data, significantly reducing reliance on domain experts, greatly reducing the cost of manual intervention, and shortening the knowledge graph construction cycle from several months to several days, thus greatly improving construction efficiency. The method of this invention supports rapid migration to new domains, fundamentally solving the domain adaptability problem of traditional methods.

[0032] (2) The method of this invention realizes multi-type triplet extraction based on prompt engineering in stages. It constructs a dedicated prompt word extraction system for three knowledge relationship types: entity-entity, entity-event, and event-event. It gives precise extraction rules for each relationship type. The extraction tasks are executed sequentially from static entity relationship to dynamic event relationship. Then, the output quality is guaranteed by format verification and automatic repair mechanism. The staged progressive extraction effectively solves the problem that traditional knowledge extraction methods cannot fully cover complex knowledge relationships. It optimizes the limitation of pipeline method with single relationship type and provides a complete knowledge triplet foundation for knowledge graph construction.

[0033] (3) In addition to generating basic abstract concepts through a large language model, the method of this invention introduces a dynamic concept clustering method with semantic embedding and mathematical optimization. It performs semantic aggregation on the concept embedding vectors and automatically determines the optimal number of clusters through the contour coefficient formula. This can effectively address the problem of inaccurate manually predefined cluster numbers, realize the automatic transformation from discrete concepts to structured schemas, improve the scientificity and accuracy of concept induction, and avoid the problems of excessive splitting or redundant merging.

[0034] (4) The method of the present invention realizes a multi-level concept generation mechanism that integrates contextual information. It adopts a differentiated concept abstraction strategy for the three types of graph elements: entities, events, and relationships. In addition to generating concepts based on their own textual features, the method also pays attention to the contextual information of the entity's neighboring nodes in the knowledge graph. When generating core labels for concept clusters, it comprehensively considers the domain definition, semantic features, and domain context of all concepts within the cluster, and simultaneously generates cluster summaries to explain the core semantics. This ensures the semantic accuracy and hierarchical richness of the generated concepts, reduces the concept redundancy rate, and lays the foundation for building a hierarchical knowledge system with a "concept-instance" structure.

[0035] (5) The method of this invention can improve the completeness and timeliness of knowledge extraction. By leveraging the massive knowledge built into the large language model, it effectively breaks through the limitations of fixed data sources, captures the latest domain knowledge dynamics, significantly improves the knowledge coverage and update speed, and ensures the continuous effectiveness of the knowledge graph. By constructing a dynamic hierarchical knowledge system, this invention can automatically abstract specific triples into general semantic concepts, which not only enhances the interpretability of knowledge but also improves the reasoning ability of knowledge. Attached Figure Description

[0036] Figure 1 This is a flowchart of the knowledge extraction method based on large language models and dynamic pattern induction of the present invention. Detailed Implementation

[0037] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0038] This invention addresses the core pain points of traditional knowledge extraction methods, which are limited by predefined patterns, have weak cross-domain adaptability, and have high costs for manual intervention. It proposes an efficient knowledge extraction method based on large language models and dynamic pattern induction. This method fully leverages the inherent advantages of large language models in deep semantic understanding, open-domain knowledge coverage, and context modeling. It integrates prompting engineering technology with a clustering-based dynamic pattern induction mechanism. The aim is to break through the dependence of traditional methods on fixed rules, i.e., schemas, and achieve automated, high-precision extraction and structured construction of knowledge from unstructured texts in different domains.

[0039] The overall framework of the knowledge extraction method based on large language models and dynamic pattern induction implemented in this invention embodiment is as follows: Figure 1 As shown, the core consists of two logically interconnected key stages: The first stage is triple extraction, which guides the large language model to automatically identify entities and semantic relationships between entities in the text through a carefully designed prompting strategy, generating standardized knowledge triples, including entity-entity triples, entity-event triples, and event-event triples, ensuring the integrity and accuracy of basic knowledge units; The second stage is concept induction, which performs semantic association analysis and hierarchical aggregation on the extracted discrete triples based on mathematical clustering algorithms, dynamically inducing structured knowledge modules with a unified conceptual system, realizing the automated, high-precision extraction and structured construction of knowledge in unstructured text.

[0040] This invention first acquires source data from the target application scenario under study. This source data includes, but is not limited to, domain requirement documents, internal knowledge bases, and textual content crawled from relevant web pages. The method of this invention primarily performs knowledge extraction from unstructured text. Then, the following two stages are executed.

[0041] The first stage involves triple extraction. A carefully designed prompting strategy guides the large language model to automatically identify entities and semantic relationships between them in the text, generating standardized knowledge triples to ensure the completeness and accuracy of the basic knowledge units. This stage includes the execution of text preprocessing, entity-entity relationship extraction, entity-event relationship extraction, event-event relationship extraction, and post-processing modules.

[0042] Step 11: The text preprocessing module performs refined preprocessing on the input raw text corpus, including:

[0043] (1) Language filtering: Identify and retain target language text, and filter out irrelevant language content;

[0044] (2) Document segmentation: For long documents that exceed the context window limit of a large language model (e.g., 8192 tokens), a sliding window strategy is used for segmentation to ensure semantic integrity. The window size is set to 80% of the model limit, and the overlap area is set to 20% to avoid the loss of key information at the segmentation boundaries;

[0045] (3) Format standardization: unify text format, remove special characters, and retain semantically relevant punctuation marks.

[0046] Step 12: Execute the multi-type triple extraction process. This invention employs cue word engineering and contextual learning, guiding a large language model to progressively extract different types of knowledge relationships in three stages using carefully designed cue words. In cue word engineering, the design of different cue words and examples will affect the results; this embodiment provides a finalized cue word template.

[0047] Step 121: Extract entity-entity relationships. Use the cue word P_EE to guide the large language model to identify entity pairs and their relationships in the text. The structure of the cue word P_EE is as follows:

[0048] (1) Task description: Identify all entity pairs and their semantic relationships from a given text.

[0049] (2) Constraints:

[0050] 1. The entity must be a concrete noun, excluding pronouns such as "he" or "it";

[0051] 2. The description of the relationship should be concise and clear, with a length not exceeding 10 words;

[0052] 3. Each entity pair retains only the most important relationship.

[0053] (3) Output format: JSON array, each element is {"head":"Entity 1", "relation":"relation", "tail":"Entity 2"}.

[0054] (4) Example demonstration: Provide 3-5 standard triplet examples.

[0055] The output format is (e1, r, e2), where e1 and e2 are entity nodes, and r is the relation type. For example, if the input text is "Shijiazhuang is the capital city of Hebei Province, located in the northern part of the North China Plain", the output triples would be [{"head":"Shijiazhuang", "relation":"is the capital city", "tail":"Hebei"}, {"head":"Shijiazhuang", "relation":"located in", "tail":"northern part of the North China Plain"}].

[0056] Step 122: Extract entity-event relationships. Use the cue word P_EV to guide the large language model to capture the participation relationships between entities and events. The structure of the cue word P_EV is as follows:

[0057] (1) Task description: Identify the events in the text and the entities involved in those events.

[0058] (2) Constraints:

[0059] 1. Events must be presented in the form of independent and complete sentences;

[0060] 2. Clearly indicate the role of the entity in the event, such as subject, object, time or place;

[0061] 3. An event can be associated with multiple entities.

[0062] (3) Output format: JSON array containing the event field and the entities array. Each element of the entity array is represented as {"entity":"entity", "role":"role"}.

[0063] (4) Example demonstration: Provide standard examples of entity-event extraction.

[0064] The output format is (e, r, v) or (v, r, e), where v is the event node, focusing on the interaction between static entities and dynamic events. For example, if the input text is "In 2023, OpenAI released the GPT-4 model, attracting global attention," the output triplet would be [{"event":"OpenAI released the GPT-4 model", "entities":[{"entity":"OpenAI", "role":"subject"}, {"entity":"GPT-4", "role":"object"}, {"entity":"2023", "role":"time"}]}].

[0065] Step 123: Extract event-event relationships. Use the cue word P_VV to guide the large language model to identify temporal or causal relationships between events. The cue word P_VV is used to limit the temporal and causal relationship types to the large language model. The structure of the cue word P_VV is as follows:

[0066] (1) Task description: Identify the temporal or causal relationships between events;

[0067] (2) Relationship type constraints:

[0068] 1. Temporal relationships: before, after, at the same time;

[0069] 2. Cause and effect: because, as a result.

[0070] (3) Output format: Each element in the JSON array is represented as {"event1":"Event 1", "relation":"Relation type", "event2":"Event 2"}.

[0071] (4) Special handling: Allow more detailed event descriptions and appropriately relax the generation length limit.

[0072] (5) Example demonstration: Provide standard examples of event extraction.

[0073] The output format is (v1, r, v2), where v1 and v2 are event nodes, and r is the time sequence or causal relationship. For example, if the input data is "Due to the continuous surge in computing power demand, the global production capacity demand for AI chips has increased. Various manufacturers subsequently launched a new round of expansion plans," the output triplet would be [{"event1":"Continuous surge in computing power demand", "relation":"because", "event2":"Increased global production capacity demand for AI chips"}, {"event1":"Increased global production capacity demand for AI chips", "relation":"before", "event2":"Manufacturers launched a new round of expansion plans"}].

[0074] This invention implements a dedicated prompt word extraction system based on prompt engineering for three knowledge relationship types: entity-entity, entity-event, and event-event. It employs a phased, progressive extraction approach, designing structured prompt words (P_EE, P_EV, P_VV) that include task descriptions, constraints, output format specifications, and example demonstrations. Precise extraction rules are provided for each relationship type, and extraction tasks are executed sequentially from static entity relationships to dynamic event relationships. Output quality is ensured through format verification and automatic repair mechanisms. This approach effectively addresses the problem of traditional knowledge extraction methods failing to comprehensively cover complex knowledge relationships, optimizes the limitations of pipelined methods with their single relationship type, and provides a complete knowledge triplet foundation for knowledge graph construction.

[0075] Step 13: The post-processing module performs post-processing and storage on the extracted knowledge relation triples.

[0076] The model output at all stages strictly follows the JSON format. Subsequent post-processing steps, such as parsing and correcting format errors, ensure the integrity of the triple structure. The extracted results, along with the original text and metadata, are then serialized and stored in batches as JSON files to provide structured input for the subsequent concept summarization stage.

[0077] The second stage involves concept induction. The core objective is to abstract the specific entities, events, and relationships obtained in the triple extraction stage into generalized semantic types, thereby constructing a dynamic schema that does not require manual pre-definition. This endows the knowledge graph with cross-domain adaptability and hierarchical semantic organization capabilities. This process primarily relies on the semantic abstraction capabilities of large-scale language models. The technical approach of concept induction will revolve around the core logic of "concept generation - hierarchical clustering," relying on the semantic understanding and knowledge dependency capture capabilities of LLM to achieve the transformation from discrete concepts to a structured schema. This stage mainly includes a concept generation module and a hierarchical clustering module.

[0078] Step 21: The concept generation module uses LLM to abstract specific graph elements into multi-dimensional general concepts.

[0079] For entities LLM combines the contextual information of its neighboring nodes to generate at least three concept phrases at different levels of abstraction, ensuring that the concepts fit the semantic roles of the entities in the graph. For example, if the input entity is "Peking University", the output concept phrases can be "higher education institution", "comprehensive research university", or "educational organization".

[0080] Regarding the event With Relationship This process directly generates at least three abstract phrases at different abstraction levels based on its own text description. All concepts must avoid repeating the original element text and cover as many abstraction levels as possible. For example, if the input event is "the company announces a layoff plan," the output concept phrases might be ["organizational restructuring event," "human resource adjustment," "corporate strategic change"]. While the concept pool generated by this process is dynamically adaptable, it is prone to redundancy in domain-specific scenarios, such as synonymous expressions for "emperor" and "monarch" within the same domain, and semantic dispersion (failure to aggregate cognate concepts across documents).

[0081] Step 22: The hierarchical clustering module takes over the concept pool generated by the concept generation module and solves the problems of redundancy and semantic dispersion through mathematical clustering and LLM-driven optimization.

[0082] First, for each concept phrase in the initial concept pool (entities, events, and relationships), LLM generates a domain-specific definition for each concept phrase based on the current application scenario. Then, using a text embedding model, the concept phrase and its domain-specific definition are transformed into high-dimensional semantic embedding vectors and concatenated as the embedding vector for that concept phrase, laying the data foundation for clustering. Second, the k-means clustering algorithm is used to achieve compact aggregation of semantically similar concepts. The optimization objective of the k-means clustering algorithm is to minimize the sum of the squared distances between all concept embedding vectors within a cluster and the cluster center. The mathematical expression is:

[0083] ;

[0084] in, The set representing all concept phrase clusters. Indicates the first Clusters, For the first Embedding vectors of concept phrases For the first Cluster The center vector, The squared Euclidean distance between a single concept phrase and the cluster center is given. Through iterative optimization of the k-means clustering function, redundant concepts are assigned to the same cluster, achieving initial semantic aggregation.

[0085] Due to the number of clusters in k-means clustering As a hyperparameter, the method of this invention further introduces a contour coefficient formula to automatically determine the optimal... The profile coefficient is set to ensure that the clustering quality balances intra-cluster cohesion and inter-cluster separation. The formula for the profile coefficient is as follows:

[0086] ;

[0087] in, The total number of concept phrases participating in the clustering; For the first The average intra-cluster distance of concept phrases Used to measure intra-cluster compactness; the smaller the value, the more semantically consistent the cluster. It is the first The average distance from each concept phrase to the nearest node in its cluster is used to measure inter-cluster separation; a higher value indicates a more significant semantic difference. The overall formula selects the optimal value by maximizing the mean of the silhouette coefficient. Avoid excessive splitting or redundant merging.

[0088] Finally, for each optimized concept cluster, LLM combines all concept phrases, domain definitions, domain context, and clustering semantic features within the cluster to generate a concise and representative core label. Simultaneously, a cluster summary is generated, systematically describing the cluster's core semantics, coverage, and role in domain knowledge. Subsequently, each cluster is associated with the original entities, relations, and event nodes extracted in the triple extraction stage, ensuring that each original node maps to its corresponding high-level concept cluster. Ultimately, using concept clusters as upper-level semantic nodes and original entities, relations, and events as lower-level instance nodes, combined with the domain dependencies between clusters, a hierarchical knowledge graph with clear levels and semantic coherence is constructed. This preserves the detailed information of the original data while achieving the structuring and abstraction of knowledge through high-level concept clusters, providing efficient semantic support for subsequent knowledge retrieval, reasoning, and other tasks.

[0089] This invention employs a dynamic concept clustering method combining semantic embedding and mathematical optimization in the concept induction stage. Besides generating basic abstract concepts through a large language model, it additionally introduces text embedding technology and a dual mathematical optimization mechanism. The k-means clustering algorithm is used to semantically aggregate concept embedding vectors, and the optimal cluster size k is automatically determined using a silhouette coefficient formula. This coefficient considers both intra-cluster cohesion and inter-cluster separation, effectively addressing the problem of inaccurate manually predefined cluster sizes. This improvement achieves automatic transformation from discrete concepts to a structured schema, enhancing the scientific rigor and accuracy of concept induction and avoiding problems of excessive splitting or redundant merging.

[0090] This invention implements a multi-level concept generation mechanism that integrates contextual information during the concept generation stage, employing differentiated concept abstraction strategies for three types of graph elements: entities, events, and relationships. For entity nodes, in addition to generating concepts based on their own textual features, it also considers the contextual information of the entity's neighboring nodes in the knowledge graph, requiring the generation of at least three concept phrases at different abstract levels for each entity. When generating core labels for concept clusters, it comprehensively considers the domain definitions, semantic features, and domain context of all concepts within the cluster, and simultaneously generates cluster summaries to elucidate the core semantics. This design ensures the semantic accuracy and hierarchical richness of the generated concepts, reduces concept redundancy, and lays the foundation for constructing a hierarchical knowledge system.

[0091] The method of this invention can be implemented as a knowledge extraction system using computer programs and hardware. This system may include a triple extraction module and a concept induction module. The input is source data of the target application scenario, and the output is a knowledge graph of that application scenario. Correspondingly, the triple extraction module includes a text preprocessing module, an entity-entity relationship extraction module, an entity-event relationship extraction module, an event-event relationship extraction module, and a post-processing module; the concept induction module includes a concept generation module and a hierarchical clustering module. The functions of each module are as described in the steps above and will not be repeated here.

[0092] The method of this invention was tested using the WebNLG dataset, a benchmark dataset in the field of natural speech processing. The purpose of the experiment was to extract a knowledge graph from this dataset. Comparison methods included the generative relation extraction model GENIE, the two-stage or multi-stage relation extraction framework EDC, and the method of this invention. Comparative experiments were conducted on triplet accuracy, using three evaluation criteria: Partial, Strict, and Exact. The Partial criterion allows candidates to partially match the reference triplet and does not restrict element types; the Strict criterion requires candidates to completely match the reference triplet and have the same element types; and the Exact criterion requires candidates to be completely identical to the reference triplet, ignoring differences in element types. Experiments revealed that on the WebNLG dataset, under the Partial criterion, the F1 score of the proposed method was 0.762, close to GENIE's 0.767, and significantly better than EDC's 0.728, indicating that the proposed method effectively captures the core information of triples. Under the Strict criterion, the F1 score of the proposed method was 0.72, on par with GENIE, and far exceeding EDC's 0.672, indicating that the proposed method has an advantage in triple normalization. Under the Exact criterion, the F1 score of the proposed method was 0.739, surpassing GENIE's 0.723, indicating that the proposed method has the most prominent advantage in accurate matching capability. Experimental results show that the overall performance of the proposed method on this dataset is better than EDC, and it matches or surpasses GENIE on more stringent metrics, demonstrating significant optimization effects.

[0093] It should be noted that the method of this invention is not a simple improvement on existing knowledge extraction techniques, but rather proposes a complete and original technical framework from triple extraction to dynamic pattern induction, addressing the core challenge of "constructing cross-domain knowledge graphs without predefined schemas." This invention solves the fundamental technical pain points. The fundamental reason for the poor performance of existing technologies lies in three major bottlenecks: (1) strong reliance on manually predefined schemas, resulting in poor cross-domain adaptability; (2) the need for a large amount of labeled data, leading to high construction costs; and (3) the extracted knowledge remains at the instance level, lacking abstract and inductive capabilities. This invention directly addresses these three bottlenecks through a "multi-type triple phase extraction method based on prompting engineering" and a "dynamic concept induction mechanism based on cluster optimization." By fully utilizing the semantic understanding capabilities of large language models, it achieves zero-label, cross-domain knowledge extraction from the source, and automatically constructs a dynamic schema through mathematical clustering, laying the foundation and prerequisite for subsequent knowledge graph applications.

[0094] The triplet extraction method of this invention is not a simple "single prompt" or "uniform template," but rather employs three dedicated prompt words (P_EE, P_EV, P_VV) to design structured guidance strategies for different relationship types, achieving comprehensive coverage of complex knowledge relationships. This classification approach is the key to achieving the ability to simultaneously capture static entity relationships, entity-event participation relationships, and temporal causal relationships between events. Alternative solutions, such as a single prompt word plus post-processing classification, lead to relationship type confusion and decreased extraction quality, failing to achieve the same knowledge completeness, thus highlighting the irreplaceable nature of the method of this invention.

[0095] The conceptual induction method of this invention is not a simple "keyword extraction" or "manual classification," but rather achieves automatic transformation from discrete concepts to a structured schema through a triple mechanism of learnable text embedding representation, k-means clustering objective function, and silhouette coefficient adaptive optimization. By minimizing the sum of squared intra-cluster distances through the k-means objective function, semantically similar concepts are ensured to cluster tightly. The optimal number of clusters k is automatically determined through the silhouette coefficient formula, avoiding the subjectivity of manually setting parameters. This dual mathematical optimization mechanism is an efficient way to achieve the ability to "dynamically adapt to concept systems in different domains." Alternative solutions, such as a fixed number of clusters plus a single clustering algorithm, only lead to excessive splitting or redundant merging, failing to fundamentally build a high-quality schema, thus highlighting the significant advantages of the method in this invention.

[0096] Except for the technical features described in the specification, all other technologies are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this invention to avoid redundancy and unnecessary limitation. The embodiments described above do not represent all embodiments consistent with this application. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this invention are still within the protection scope of this invention.

[0097] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

Claims

1. A knowledge extraction method based on large language models and dynamic pattern induction, which first obtains source data from the target application scenario, wherein the source data is unstructured text, characterized in that... The method extracts knowledge from the source data through the following steps: Step 1: Extract knowledge relation triples from the source data using a large language model, including: (1) The text preprocessing module preprocesses the source data, removes irrelevant language content and special characters, and unifies the text format; (2) Different types of knowledge relationships are extracted through a three-stage progressive process, including: First, the entity-entity relationship extraction module uses the prompt word P_EE to guide the large language model to identify entity pairs and semantic relationships between entities in the text; second, the entity-event relationship extraction module uses the prompt word P_EV to guide the large language model to identify events in the text and the entities involved in those events; finally, the event-event relationship extraction module uses the prompt word P_VV to guide the large language model to identify temporal or causal relationships between events in the text; the prompt words P_EE, P_EV, and P_VV each contain four parts: task description, constraints, output format specifications, and example demonstration; among them, The constraints of the prompt word P_EE include: entities must be concrete nouns, excluding pronouns; the length of the relation description shall not exceed 10 words; each entity pair shall retain only the most important relation; The constraints of the prompt word P_EV include: the event must be presented as an independent and complete sentence; the role of the entity in the event must be clearly marked; an event can be associated with multiple entities; The constraints of the prompt word P_VV include relation type constraints: temporal relations: before, after, at the same time; causal relations: because, as a result; (3) The post-processing module performs format repair on the extracted knowledge relation triples to ensure the structural integrity of the triples and stores the knowledge relation triples; Step 2 involves conceptual generalization, abstracting the extracted entities, events, and relationships into generalized semantic types, including: (1) The concept generation module generates at least three concept phrases at different levels of abstraction for each entity by using a large language model and combining the entity’s context; it generates at least three concept phrases at different levels of abstraction for each event and relationship based on its own text description, and the generated concept phrases avoid repeating the original text; all generated concept phrases form the initial concept pool. (2) The hierarchical clustering module clusters the concept phrases of entities, events and relations in the initial concept pool and generates general concepts using a large language model. This includes: First, generating a domain definition for each concept phrase in the initial concept pool, converting the concept phrase and the domain definition into semantic embedding vectors and concatenating them to obtain the embedding vector of the concept phrase; then, using the k-means clustering algorithm to cluster the embedding vectors of the concept phrases and perform semantic aggregation; finally, for each aggregated concept cluster, using the large language model to combine all concept phrases, domain definitions, domain context and clustering semantic features within the cluster to generate a core label and cluster summary, and establishing an association between each entity, event and relation extracted in step 1 and the concept cluster to form a hierarchical knowledge graph with concept clusters as upper-level semantic nodes and entities, events and relations as lower-level instance nodes.

2. The method according to claim 1, characterized in that, In step 1, the extracted knowledge relationship triples are output as JSON arrays. For entity-entity relationships, each element in the JSON array is represented as {"head":"entity1", "relation":"relation", "tail":"entity2"}. For entity-event relationships, the JSON array contains an event field "event" and an entity array "entities", with each element in the entity array represented as {"entity":"entity", "role":"role"}. For event-event relationships, each element in the JSON array is represented as {"event1":"event1", "relation":"relation type", "event2":"event2"}.

3. The method according to claim 1, characterized in that, In step 2, the hierarchical clustering module uses the k-means clustering algorithm to cluster concept phrases. The objective function for clustering is as follows: ； in, A cluster set, containing clusters Clusters, For the first Cluster The center vector, For the first The embedding vectors of concept phrases are obtained; through iterative optimization, concept phrases with the same semantics are clustered into the same cluster; Cluster number The optimal value is determined by solving the contour coefficient formula, as follows: ； in, The total number of concept phrases participating in the clustering. For the first The average intra-cluster distance of concept phrases It is the first The average distance from each concept phrase to the nearest node in the cluster.

Images